How to decommission nodes/blacklist nodes

HDFS

Put following config in conf/hdfs-site.xml:

<property>
  <name>dfs.hosts.exclude</name>
  <value>/full/path/of/host/exclude/file</value>
</property>

Use following command to ask HDFS to re-read host exclude file and decommission nodes accordingly.

./bin/hadoop dfsadmin -refreshNodes

MapReduce

Put following config in conf/mapred-site.xml

<property>
  <name>mapred.hosts.exclude</name>
  <value>/full/path/of/host/exclude/file</value>
</property>

Use following command to ask Hadoop MapReduce to reconfigure nodes.

./bin/hadoop mradmin -refreshNodes

Whitelist/Recommission

Also you can "whitelist" nodes. In other words, you can specify which nodes are allowed to connect to namenode/jobtracker. 

HDFS

Put following config in conf/hdfs-site.xml:

<property>
  <name>dfs.hosts</name>
  <value>/full/path/to/whitelisted/node/file</value>
</property>

Use following command to ask Hadoop to refresh node status to based on configuration.

./bin/hadoop dfsadmin -refreshNodes

MapReduce

Put following config in conf/mapred-site.xml

<property>
  <name>mapred.hosts</name>
  <value>>/full/path/to/whitelisted/node/file</value>
</property>

Use following command to ask Hadoop MapReduce to reconfigure nodes.

./bin/hadoop mradmin -refreshNodes

 

Support of -mradmin was added in 0.21.0. See JIRA issue https://issues.apache.org/jira/browse/HADOOP-5643 for details.