生產環境 inpu lai home throws next erl 當前 例子

這一年的項目中，有大量的場景需要Java 進程調用 Linux的bash shell 腳本實現相關功能。

從之前的項目中拷貝的相關模塊和網上的例子來看，有個別的“陷阱”造成調用shell 腳本在某些特殊的場景下，有一些奇奇怪怪的bug。

大家且聽我一一道來。

先看看網上搜索到的例子：

[java] view plain copy

1. package someTest;
3. import java.io.BufferedReader;
4. import java.io.IOException;
5. import java.io.InputStreamReader;
7. public class ShellTest {
9. public static void main(String[] args) {
10. InputStreamReader stdISR = null;
11. InputStreamReader errISR = null;
12. Process process = null;
13. String command = "/home/Lance/workspace/someTest/testbash.sh";
14. try {
15. process = Runtime.getRuntime().exec(command);
16. int exitValue = process.waitFor();
18. String line = null;
20. stdISR = new InputStreamReader(process.getInputStream());
21. BufferedReader stdBR = new BufferedReader(stdISR);
22. while ((line = stdBR.readLine()) != null) {
23. System.out.println("STD line:" + line);
24. }
26. errISR = new InputStreamReader(process.getErrorStream());
27. BufferedReader errBR = new BufferedReader(errISR);
28. while ((line = errBR.readLine()) != null) {
29. System.out.println("ERR line:" + line);
30. }
31. } catch (IOException | InterruptedException e) {
32. e.printStackTrace();
33. } finally {
34. try {
35. if (stdISR != null) {
36. stdISR.close();
37. }
38. if (errISR != null) {
39. errISR.close();
40. }
41. if (process != null) {
42. process.destroy();
43. }
44. } catch (IOException e) {
45. System.out.println("正式執行命令：" + command + "有IO異常");
46. }
47. }
48. }
49. }

testbash.sh

[plain] view plain copy

1. #!/bin/bash
3. echo `pwd`

輸出結果為：

[plain] view plain copy

1. STD line:/home/Lance/workspace/someTest

Java在執行Runtime.getRuntime().exec(command)之後，Linux會創建一個進程，該進程與JVM進程建立三個管道連接，標準輸入流、標準輸出流、標準錯誤流。

上述代碼，依次讀取標準輸出流和標準錯誤流，在shell給出“退出信號”後，做了相應的清理工作。

對於一般場景來說，這段代碼可以湊合用了。但是，在實際場景中，會有以下幾個“陷阱”。

一. 當標準輸出流或標準錯誤流非常龐大的時候，會出現調用waitFor方法卡死的bug。

真實的環境中，當標準輸出在10000行左右的時候，就會出現卡死的情況。

原因分析：假設linux進程不斷向標準輸出流和標準錯誤流寫數據，而JVM卻不讀取，數據會暫存在linux緩存區，當緩存區存滿之後導致該進程無法繼續寫數據，會僵死，導致java進程會卡死在waitFor()處，永遠無法結束。

解決方式：由於標準輸出和錯誤輸出都會向Linux緩存區寫數據，而腳本如何輸出這兩種流是Java端不能確定的。為了不讓shell腳本的子進程卡死，這兩種輸出需要分別讀取，而且不能互相影響。所以必須新開兩個線程來進行讀取。

我開始的實現如下：

[java] view plain copy

1. package someTest;
3. import java.io.BufferedReader;
4. import java.io.IOException;
5. import java.io.InputStream;
6. import java.io.InputStreamReader;
7. import java.util.LinkedList;
8. import java.util.List;
10. public class CommandStreamGobbler extends Thread {
12. private InputStream is;
14. private String command;
16. private String prefix = "";
18. private boolean readFinish = false;
20. private boolean ready = false;
22. private List<String> infoList = new LinkedList<String>();
24. CommandStreamGobbler(InputStream is, String command, String prefix) {
25. this.is = is;
26. this.command = command;
27. this.prefix = prefix;
28. }
30. public void run() {
31. InputStreamReader isr = null;
32. try {
33. isr = new InputStreamReader(is);
34. BufferedReader br = new BufferedReader(isr);
35. String line = null;
36. ready = true;
37. while ((line = br.readLine()) != null) {
38. infoList.add(line);
39. System.out.println(prefix + " line: " + line);
40. }
41. } catch (IOException ioe) {
42. System.out.println("正式執行命令：" + command + "有IO異常");
43. } finally {
44. try {
45. if (isr != null) {
46. isr.close();
47. }
48. } catch (IOException ioe) {
49. System.out.println("正式執行命令：" + command + "有IO異常");
50. }
51. readFinish = true;
52. }
53. }
55. public InputStream getIs() {
56. return is;
57. }
59. public String getCommand() {
60. return command;
61. }
63. public boolean isReadFinish() {
64. return readFinish;
65. }
67. public boolean isReady() {
68. return ready;
69. }
71. public List<String> getInfoList() {
72. return infoList;
73. }
74. }

[java] view plain copy

1. package someTest;
3. import java.io.IOException;
4. import java.io.InputStreamReader;
6. public class ShellTest {
8. public static void main(String[] args) {
9. InputStreamReader stdISR = null;
10. InputStreamReader errISR = null;
11. Process process = null;
12. String command = "/home/Lance/workspace/someTest/testbash.sh";
13. try {
14. process = Runtime.getRuntime().exec(command);
16. CommandStreamGobbler errorGobbler = new CommandStreamGobbler(process.getErrorStream(), command, "ERR");
17. CommandStreamGobbler outputGobbler = new CommandStreamGobbler(process.getInputStream(), command, "STD");
19. errorGobbler.start();
20. // 必須先等待錯誤輸出ready再建立標準輸出
21. while (!errorGobbler.isReady()) {
22. Thread.sleep(10);
23. }
24. outputGobbler.start();
25. while (!outputGobbler.isReady()) {
26. Thread.sleep(10);
27. }
29. int exitValue = process.waitFor();
30. } catch (IOException | InterruptedException e) {
31. e.printStackTrace();
32. } finally {
33. try {
34. if (stdISR != null) {
35. stdISR.close();
36. }
37. if (errISR != null) {
38. errISR.close();
39. }
40. if (process != null) {
41. process.destroy();
42. }
43. } catch (IOException e) {
44. System.out.println("正式執行命令：" + command + "有IO異常");
45. }
46. }
47. }
48. }

到此為止，解決了Java卡死shell腳本的情況。再說說，第二種可能。

二. 由於shell腳本的編寫問題，當其自身出現僵死的情況，上述代碼出現Java代碼被僵死的Shell腳本阻塞住的情況。

原因分析：由於shell腳本也是人寫的，難免會出現失誤。在Java調用shell腳本時，無論是Debug場景還是生產環境，都發生過shell腳本意外僵死反過來卡死Java相關線程的情況。典型的表現為：shell腳本長時間運行，標準輸出和錯誤輸出沒有任何輸出（包括結束符），操作系統顯示shell腳本在正常運行或僵死，沒有退出信號。

解決方式：上述代碼中，至少有三處會導致線程阻塞，包括標準輸出和錯誤輸出這線程的BufferedReader的readline方法，以及Process的waitFor方法。解決這個問題的核心有兩個，1.避免任何Java線程被阻塞住，因為一旦被IO阻塞住，線程將處於內核態，主線程沒有任何辦法強制結束相關子線程。2.添加一個簡單的超時機制，超時後回收相應的線程資源，並結束調用過程。

演示代碼中，我改寫了testshell.sh，寫一個沒有任何輸出的死循環模擬shell卡死的情況。

[plain] view plain copy

1. #!/bin/bash
3. while true;do
4. a=1
5. sleep 0.1
6. done

[java] view plain copy

1. package someTest;
3. import java.io.BufferedReader;
4. import java.io.IOException;
5. import java.io.InputStream;
6. import java.io.InputStreamReader;
7. import java.util.LinkedList;
8. import java.util.List;
10. public class CommandStreamGobbler extends Thread {
12. private InputStream is;
14. private String command;
16. private String prefix = "";
18. private boolean readFinish = false;
20. private boolean ready = false;
22. // 命令執行結果,0:執行中 1:超時 2:執行完成
23. private int commandResult = 0;
25. private List<String> infoList = new LinkedList<String>();
27. CommandStreamGobbler(InputStream is, String command, String prefix) {
28. this.is = is;
29. this.command = command;
30. this.prefix = prefix;
31. }
33. public void run() {
34. InputStreamReader isr = null;
35. BufferedReader br = null;
36. try {
37. isr = new InputStreamReader(is);
38. br = new BufferedReader(isr);
39. String line = null;
40. ready = true;
41. while (commandResult != 1) {
42. if (br.ready() || commandResult == 2) {
43. if ((line = br.readLine()) != null) {
44. infoList.add(line);
45. } else {
46. break;
47. }
48. } else {
49. Thread.sleep(100);
50. }
51. }
52. } catch (IOException | InterruptedException ioe) {
53. System.out.println("正式執行命令：" + command + "有IO異常");
54. } finally {
55. try {
56. if (br != null) {
57. br.close();
58. }
59. if (isr != null) {
60. isr.close();
61. }
62. } catch (IOException ioe) {
63. System.out.println("正式執行命令：" + command + "有IO異常");
64. }
65. readFinish = true;
66. }
67. }
69. public InputStream getIs() {
70. return is;
71. }
73. public String getCommand() {
74. return command;
75. }
77. public boolean isReadFinish() {
78. return readFinish;
79. }
81. public boolean isReady() {
82. return ready;
83. }
85. public List<String> getInfoList() {
86. return infoList;
87. }
89. public void setTimeout(int timeout) {
90. this.commandResult = timeout;
91. }
92. }

[java] view plain copy

1. package someTest;
3. public class CommandWaitForThread extends Thread {
5. private Process process;
6. private boolean finish = false;
7. private int exitValue = -1;
9. public CommandWaitForThread(Process process) {
10. this.process = process;
11. }
13. public void run() {
14. try {
15. this.exitValue = process.waitFor();
16. } catch (InterruptedException e) {
17. e.printStackTrace();
18. } finally {
19. finish = true;
20. }
21. }
23. public boolean isFinish() {
24. return finish;
25. }
27. public void setFinish(boolean finish) {
28. this.finish = finish;
29. }
31. public int getExitValue() {
32. return exitValue;
33. }
35. }

[java] view plain copy

1. package someTest;
3. import java.io.IOException;
4. import java.io.InputStreamReader;
5. import java.util.Date;
7. public class ShellTest {
9. public static void main(String[] args) {
10. InputStreamReader stdISR = null;
11. InputStreamReader errISR = null;
12. Process process = null;
13. String command = "/home/Lance/workspace/someTest/testbash.sh";
14. long timeout = 10 * 1000;
15. try {
16. process = Runtime.getRuntime().exec(command);
18. CommandStreamGobbler errorGobbler = new CommandStreamGobbler(process.getErrorStream(), command, "ERR");
19. CommandStreamGobbler outputGobbler = new CommandStreamGobbler(process.getInputStream(), command, "STD");
21. errorGobbler.start();
22. // 必須先等待錯誤輸出ready再建立標準輸出
23. while (!errorGobbler.isReady()) {
24. Thread.sleep(10);
25. }
26. outputGobbler.start();
27. while (!outputGobbler.isReady()) {
28. Thread.sleep(10);
29. }
31. CommandWaitForThread commandThread = new CommandWaitForThread(process);
32. commandThread.start();
34. long commandTime = new Date().getTime();
35. long nowTime = new Date().getTime();
36. boolean timeoutFlag = false;
37. while (!commandIsFinish(commandThread, errorGobbler, outputGobbler)) {
38. if (nowTime - commandTime > timeout) {
39. timeoutFlag = true;
40. break;
41. } else {
42. Thread.sleep(100);
43. nowTime = new Date().getTime();
44. }
45. }
46. if (timeoutFlag) {
47. // 命令超時
48. errorGobbler.setTimeout(1);
49. outputGobbler.setTimeout(1);
50. System.out.println("正式執行命令：" + command + "超時");
51. }else {
52. // 命令執行完成
53. errorGobbler.setTimeout(2);
54. outputGobbler.setTimeout(2);
55. }
57. while (true) {
58. if (errorGobbler.isReadFinish() && outputGobbler.isReadFinish()) {
59. break;
60. }
61. Thread.sleep(10);
62. }
63. } catch (IOException | InterruptedException e) {
64. e.printStackTrace();
65. } finally {
66. if (process != null) {
67. process.destroy();
68. }
69. }
70. }
72. private boolean commandIsFinish(CommandWaitForThread commandThread, CommandStreamGobbler errorGobbler, CommandStreamGobbler outputGobbler) {
73. if (commandThread != null) {
74. return commandThread.isFinish();
75. } else {
76. return (errorGobbler.isReadFinish() && outputGobbler.isReadFinish());
77. }
78. }
79. }

在以上的代碼中，為了防止線程被阻塞，要點如下：

1. 在CommandStreamGobbler裏，bufferedReader在readLine()之前，先用ready()看一下當前緩沖區的情況，請特別註意ready()描述，這個方法是非阻塞的。

[java] view plain copy

1. boolean java.io.BufferedReader.ready() throws IOException
3. Tells whether this stream is ready to be read. A buffered character stream is ready if the buffer is not empty, or if the underlying character stream is ready.
5. Returns:
6. True if the next read() is guaranteed not to block for input, false otherwise. Note that returning false does not guarantee that the next read will block.

2.在一個新線程commandThread中，調用process對象的waitFor()從而避免主線程卡死，主線程的最後會執行finally塊中的process.destory()保證commandThread正常退出。

以上的兩點改進，保證了Java在調用shell腳本過程互不被對方卡死的機制。

三.在執行shell腳本過程中，可能會添加參數，通常在終端中，我們使用“ ”（空格）把參數隔開。

為了區分空格是作為參數分隔符，還是參數的一部分。調用exec方法有特別的註意事項。

[java] view plain copy

1. String command = "/home/Lance/workspace/someTest/testbash.sh ‘hello world‘";
3. process = Runtime.getRuntime().exec(command);

等價於

[java] view plain copy

1. List<String> commandList = new LinkedList<String>();
2. commandList.add("/home/Lance/workspace/someTest/testbash.sh");
3. commandList.add("hello world");
4. String[] commands = new String[commandList.size()];
5. for (int i = 0; i < commandList.size(); i++) {
6. commands[i] = commandList.get(i);
7. }
9. process = Runtime.getRuntime().exec(commands);

好了，今天介紹到這裏。

Java 調用 shell 腳本詳解