最近在做通過sqoop 將oracle資料庫當中的資料匯入的HDFS上面，但是當我序列的時候是沒有一點問題的。但是為了達到叢集當中資源的額最大的使用率。想讓匯入資料做成並行去處理。
在做並行的時候，有時候是好的，有時候就出錯，這樣不穩定的系統真的頭大。出現的問題如下：

8/10/29 15:01:03 ERROR manager.SqlManager: Error executing statement: java.sql.SQLRecoverableException: IO Error: Connection reset
java.sql.SQLRecoverableException: IO Error: Connection reset
  at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:
 
682)
  at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:711)
  at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:385)
  at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:30)
  at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:
 
558)
  at java.sql.DriverManager.getConnection(DriverManager.java:571)
  at java.sql.DriverManager.getConnection(DriverManager.java:215)
  at org.apache.sqoop.manager.OracleManager.makeConnection(OracleManager.java:329)
  at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:
 
52)
  at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:763)
  at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:786)
  at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:289)
  at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:260)
  at org.apache.sqoop.manager.SqlManager.getColumnTypesForQuery(SqlManager.java:253)
  at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:336)
  at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1858)
  at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1657)
  at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:106)
  at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:494)
  at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:621)
  at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
  at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
  at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
  at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
  at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
  at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
Caused by: java.net.SocketException: Connection reset
  at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:118)
  at java.net.SocketOutputStream.write(SocketOutputStream.java:159)
  at oracle.net.ns.DataPacket.send(DataPacket.java:209)
  at oracle.net.ns.NetOutputStream.flush(NetOutputStream.java:215)
  at oracle.net.ns.NetInputStream.getNextPacket(NetInputStream.java:302)
  at oracle.net.ns.NetInputStream.read(NetInputStream.java:249)
  at oracle.net.ns.NetInputStream.read(NetInputStream.java:171)
  at oracle.net.ns.NetInputStream.read(NetInputStream.java:89)
  at oracle.jdbc.driver.T4CSocketInputStreamWrapper.readNextPacket(T4CSocketInputStreamWrapper.java:123)
  at oracle.jdbc.driver.T4CSocketInputStreamWrapper.read(T4CSocketInputStreamWrapper.java:79)
  at oracle.jdbc.driver.T4CMAREngineStream.unmarshalUB1(T4CMAREngineStream.java:426)
  at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:390)
  at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:249)
  at oracle.jdbc.driver.T4CTTIoauthenticate.doOSESSKEY(T4CTTIoauthenticate.java:435)
  at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:580)
  ... 25 more
18/10/29 15:01:03 ERROR tool.ImportTool: Import failed: java.io.IOException: No columns to generate for ClassWriter
  at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1663)
  at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:106)
  at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:494)
  at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:621)
  at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
  at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
  at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
  at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
  at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
  at org.apache.sqoop.Sqoop.main(Sqoop.java:252)

Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.ShellMain], exit code [1]

看到上面的問題，第一反應應該是oracle 驅動的問題，然後就修改oracle的驅動包,然後從ojdbc6.jar的jar包開始替換，然後換成ojdbc7.jar但是問題依然是存在的。然後就FQ到國外的網站。

然後檢視到相同的問題。提供的解決方法

 上麥這段話的意思是，當我們在執行map任務的時候，在主機上缺少一個快速隨機產生的裝置。然後去了sqoop的官網看到也是同樣的這句話：

Solution: This problem occurs primarily due to the lack of a fast random number generation device on the host where the map tasks execute. On typical Linux systems this can be addressed by setting the following property in the java.security file:

在這裡在我們的執行腳本當中加入這兩句話：

export HADOOP_OPTS=-Djava.security.egd=file:/dev/../dev/urandom
sqoop import -D mapred.child.java.opts="-Djava.security.egd=file:/dev/../dev/urandom"

至此我的問題得到了解決。整個並行的sqoop指令碼執行成功。

下面是參考的文章:

http://stackoverflow.com/questions/2327220/oracle-jdbc-intermittent-connection-issue/

https://community.oracle.com/thread/943911?tstart=0&messageID=3793101

https://sqoop.apache.org/docs/1.4.6/SqoopUserGuide.html#_oracle_connection_reset_errors