1. 程式人生 > >hadoop hdfs設置quota

hadoop hdfs設置quota

計算 寫入 repl locks pac user clas 兩個 ati

quota分為兩種:

1. 目錄下的文件數限制

2. 目錄下的空間大小

//設置文件數
hdfs dfsadmin -setQuota 1000000 /user/jenkin
//設置空間大小
hdfs dfsadmin -setSpaceQuota 100g /user/jenkin

//查看quota
hadoop fs -count -q /user/jenkin

[[email protected] ~]# hadoop fs -count -q /user/jenkin
     1000000          999182    107374182400    100052379973           26          792         2440600809 /user/jenkin

列對應:
QUOTA REMAINING_QUOTA SPACE_QUOTA REMAINING_SPACE_QUOTA DIR_COUNT FILE_COUNT

計算公式:  

QUOTA – (DIR_COUNT + FILE_COUNT) = REMAINING_QUOTA

SPACE_QUOTA – CONTENT_SIZE = REMAINING_SPACE_QUOTA

space quota 的設定所看的不是上傳的檔案大小,是寫入 HDFS 的所有 block size,假設一個檔案要寫入兩個blocks,在core-site.xml設定dfs.block.size為64MB,dfs.replication為3,那所需空間:2*64MB*3=384MB

  

hadoop hdfs設置quota