Friday, December 02, 2011

Memory allocation settings in Hadoop

Edit file conf/mapred-site.xml to change amount of memory allocated to sorting:

<property> 
    <name>io.sort.mb</name>
    <value>300</value>
</property>

Edit file conf/mapred-site.xml to change amount of memory allocated to each map/reduce task:

<property>
    <name>mapred.child.java.opts</name>
    <value>-Xmx800m</value>
</property>

Edit file conf/hadoop-env.sh to change amount of memory allocated to Hadoop daemons:

export HADOOP_HEAPSIZE=1000

No comments: