解決java連線Hbase程式卡住執行也不報錯的問題
阿新 • • 發佈：2018-12-25
問題描述：
最近在hadoop叢集上搭建好Hbase，用hbase shell進行增刪改查操作都沒有問題，但是用java API來對hbase進行操作卻出現一個很奇怪的問題，就是執行java程式後沒有任何反應，程式也不報錯。
首先貼出我的測試程式碼：
執行程式，發現程式並沒有輸出查到的表內的值，奇怪的是程式不報任何異常，也不產生輸出，所以應該是卡住了，除錯程式發現，程式會卡在import java.io.IOException; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.Path; import org.apache.hadoop.hbase.HBaseConfiguration; import org.apache.hadoop.hbase.client.Get; import org.apache.hadoop.hbase.client.HTable; import org.apache.hadoop.hbase.client.Result; import org.apache.hadoop.hbase.client.ResultScanner; import org.apache.hadoop.hbase.client.Scan; import org.apache.hadoop.hbase.util.Bytes; public class Test { private static Configuration conf=HBaseConfiguration.create(); static { conf=HBaseConfiguration.create(); //註釋部分如果在本地測試不需要加，要是遠端連線Hbase需要制定hbase地址 // conf.set("hbase.master", "master:60000"); // conf.set("hbase.zookeeper.quorum","slave，slave2,slave3"); System.out.println(conf.get("hbase.master")); } public static void main(String[] args) throws IOException { // TODO Auto-generated method stub HTable table = new HTable(conf,"test1"); Scan s = new Scan(); ResultScanner rs = table.getScanner(s); for(Result res:rs) { System.out.println(res); } table.close(); } }
ResultScanner rs = table.getScanner(s);這句話，應該是呼叫HTable的方法問題，檢視zookeeper日誌後發現，程式一直在嘗試連線hbase，但是timeout,也就是連線超時，程式不斷嘗試，所以會出現卡住的現象。
而且檢視Hbase-master日誌發現沒有任何寫入，說明程式根本還沒執行到要用到hbase的時候就已經卡住了，也側面印證了我的Hbase配置是沒有問題的。所以我判斷是java連線Hbase過程出了問題。
接下來的幾天我一直在查，但是一直沒有解決，但是又一次我執行程式卡住之後，沒退出程式就走了，等很久我回來時，發現程式報錯了！（應該是長時間嘗試連線，達到了設定的timeout最大值）
報錯如下：
log4j:WARN No appenders could be found for logger (org.apache.hadoop.security.Groups). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. Exception in thread "main" java.net.SocketTimeoutException: callTimeout=1200000, callDuration=19813388: row '' on table 'TEST1' at null at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:143) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:91) at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:289) at org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:190) at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:185) at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:111) at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:780) at hbasetest.Test.main(Test.java:33) Caused by: java.io.IOException: com.google.protobuf.ServiceException: java.lang.NoClassDefFoundError: com/yammer/metrics/core/Gauge at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:300) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRowOrBefore(ProtobufUtil.java:1564) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRowOrBefore(ProtobufUtil.java:1540) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1337) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1179) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1136) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:977) at org.apache.hadoop.hbase.client.RegionServerCallable.prepare(RegionServerCallable.java:74) at org.apache.hadoop.hbase.client.ScannerCallable.prepare(ScannerCallable.java:125) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:114) ... 7 more Caused by: com.google.protobuf.ServiceException: java.lang.NoClassDefFoundError: com/yammer/metrics/core/Gauge at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1709) at org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1750) at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.get(ClientProtos.java:31612) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRowOrBefore(ProtobufUtil.java:1560) ... 15 more Caused by: java.lang.NoClassDefFoundError: com/yammer/metrics/core/Gauge at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1689) ... 18 more Caused by: java.lang.ClassNotFoundException: com.yammer.metrics.core.Gauge at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 19 more
發現是找不到包的異常，因為我之前都是按照網上很多帖子去導的jar包，很多帖子寫的依賴包都大同小異，沒什麼差別，後來通過大量搜尋，找到了問題，是程式缺少了一個叫：“metrics-core-2.2.0.jar”的包（此jar包可在$HBASE_HOME/lib下找到），build path該包，在執行程式，執行成功：
master:60000
log4j:WARN No appenders could be found for logger (org.apache.hadoop.security.Groups).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
keyvalues={1/personinfo:name/1462411628085/Put/vlen=6/mvcc=0, 1/personinfo:sex/1462411628085/Put/vlen=3/mvcc=0}
keyvalues={2/personinfo:name/1462411628085/Put/vlen=6/mvcc=0, 2/personinfo:sex/1462411628085/Put/vlen=3/mvcc=0}
keyvalues={3/personinfo:name/1462411625239/Put/vlen=6/mvcc=0}
keyvalues={4/personinfo:name/1462411623526/Put/vlen=6/mvcc=0, 4/personinfo:sex/1462411623526/Put/vlen=3/mvcc=0}
keyvalues={5/personinfo:name/1462411628403/Put/vlen=6/mvcc=0, 5/personinfo:sex/1462411628403/Put/vlen=3/mvcc=0}
keyvalues={6/personinfo:name/1462411628403/Put/vlen=6/mvcc=0}