1. 程式人生 > >解決sqoop報錯Invalid number; item = ITEM_UNICODE

解決sqoop報錯Invalid number; item = ITEM_UNICODE

lur cloudera ble not null edm ive pan ext 使用報錯

報錯棧:

java.sql.SQLException: Invalid number; item = ITEM_UNICODE
    at com.intersys.jdbc.SysList.getInt(SysList.java:1735)
    at com.intersys.jdbc.CacheResultSet.getInt(CacheResultSet.java:247)
    at org.apache.sqoop.lib.JdbcWritableBridge.readInteger(JdbcWritableBridge.java:52)
    at com.cloudera.sqoop.lib.JdbcWritableBridge.readInteger(JdbcWritableBridge.java:
53) at QueryResult.readFields(QueryResult.java:246) at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:244) at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:556) at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:
80) at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145) at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:
787) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)

問題的根本原因是字段值類型異常,本來應該是數字型,裏面卻存了unicode字符。使用報錯信息查不到相關資料,另外增加參數也不能解決:-D mapreduce.map.failures.maxpercent=1, query裏面增加: and Specimen is not null 也不能解決。(Specimen是報錯字段)。

靠譜的解決辦法是找出這行異常數據,在數據庫中糾正或刪除。如果想臨時解決,可以把這個字段在導入時轉換類型為字符:--map-column-java Specimen=String。

解決sqoop報錯Invalid number; item = ITEM_UNICODE