Saturday, March 2, 2019

Hadoop HDFS - Disk Usage



--Disk space reserved for non HDFS - hdfs-site.xml



dfs.datanode.du.reserved



--HDFS Balancer



sudo -su hdfs



hdfs balancer



hdfs balancer -threshold 5



--Creating user directories



sudo -u hdfs hadoop fs -mkdir /user/deepak.dubey

sudo -u hdfs hadoop fs -chown deepak.dubey:deepak.dubey /user/deepak.dubey



--Disk Usage



hadoop fs -du /user



hadoop fs -du -h /user



hadoop fs -du -h -s /user