Thursday 19 January 2012

Common Error While Formatting Hdfs

Typical Error 
ERROR namenode.NameNode: java.io.IOException: Cannot create directory /app/hadoop/tmp/dfs/name/current
    at org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.clearDirectory(Storage.java:297)
    at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:1219)
    at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:1238)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1038)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1145)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1162)

How to solve it :
sudo chown 755 /app/hadoop/tmp


Now we create the directory and set the required ownerships and permissions:

$ sudo mkdir -p /app/hadoop/tmp
$ sudo chown hduser:hadoop /app/hadoop/tmp
# ...and if you want to tighten up security, chmod from 755 to 750...
$ sudo chmod 750 /app/hadoop/tmp

If you forget to set the required ownerships and permissions, you will see a java.io.IOException when you try to format the name node in the next section).

Success Case:

*******************************************/
 INFO util.GSet: VM type       = 64-bit
 INFO util.GSet: 2% max memory = 17.77875 MB
 INFO util.GSet: capacity      = 2^21 = 2097152 entries
 INFO util.GSet: recommended=2097152, actual=2097152
 INFO namenode.FSNamesystem: fsOwner=hduser
 INFO namenode.FSNamesystem: supergroup=supergroup
 INFO namenode.FSNamesystem: isPermissionEnabled=true
 INFO namenode.FSNamesystem: dfs.block.invalidate.limit=100
 INFO namenode.FSNamesystem: isAccessTokenEnabled=false accessKeyUpdateInterval=0 min(s), accessTokenLifetime=0 min(s)
 INFO namenode.NameNode: Caching file names occuring more than 10 times
 INFO common.Storage: Image file of size 112 saved in 0 seconds.
 INFO common.Storage: Storage directory /app/hadoop/tmp/dfs/name has been successfully formatted.
 INFO namenode.NameNode: SHUTDOWN_MSG:



refer:

1 comment:

  1. Hi, Great.. Tutorial is just awesome..It is really helpful for a newbie like me..
    I am a regular follower of your blog. Really very informative post you shared here.
    Kindly keep blogging. If anyone wants to become a Front end developer learn from Javascript Training in Chennai .
    or Javascript Training in Chennai.
    Nowadays JavaScript has tons of job opportunities on various vertical industry. ES6 Training in Chennai

    ReplyDelete