Thursday, May 12, 2011

Hadoop datanode version control

Sometimes, when you upgrade your Hadoop, you may get following error in your namenode log:

Incompatible build versions: namenode BV = Unknown; datanode BV =

./common-0.21.0/src/saveVersion.sh generates package-info.java which includes version information.  The content looks like

@HadoopVersionAnnotation(version="0.21.1-SNAPSHOT", revision="1", branch="",
                         user="username", date="Mon Nov 15 12:28:49 EST 2010",
                         url="your_domain/path",
                         srcChecksum="a1aeb15b4854808d152989ba76f90fac")

saveVersion.sh is executed when you build Hadoop using ant.  It is specified in build.xml (target "init").

In Java code, class org.apache.hadoop.util.VersionInfo manages version.  It gets version information from package-info.java generated by saveVersion.sh.

In class org.apache.hadoop.hdfs.server.datanode.DataNode, method handshake checks whether build version is equal.  How build version is calculated is shown below.

public static String getBuildVersion(){
  return VersionInfo.getVersion() +
  " from " + VersionInfo.getRevision() +
  " by " + VersionInfo.getUser() +
  " source checksum " + VersionInfo.getSrcChecksum();
}

So, the quick solution is that you upgrade all installations of Hadoop on different nodes.

No comments: