此前瞭解過Apache Drill的設計原理，借鑑與Google的Dremel的一個開源實現；而cloudera貌似在此基礎上構造了impala.

最近從apache官網上看到Drill原始碼可以下載了，分析一下。

http://www.apache.org/dyn/closer.cgi/incubator/drill/drill-1.0.0-m1-incubating/

程式碼結構：

包括:src,sqlparser,exec,distribution,contrib,common,sample-data.

檢視INSTALL.cd要求如下：

Java 7+

protoc 2.5.x compiler 【google的Protobuffer compiler 2.5.x】

Maven 3.0+

原始碼可以直接下載；然後通過mvn clean install來進行安裝。

將drill-1.0.0xxx.tar.gz解壓縮到/drill/目錄下

直接執行./sqlline，開始下載所需的包

最後開始報錯

[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
cat: .classpath: 沒有那個檔案或目錄
Exception in thread "main" java.lang.NoClassDefFoundError: sqlline/SqlLine
Caused by: java.lang.ClassNotFoundException: sqlline.SqlLine

 看來還算需要根據規定一步步來處理。

執行命令

mvn install [開始下載依賴的版本]

檢測Java版本不行，升級或者安裝Java 7

[[email protected] jdk1.7]# pwd
/drill/jdk1.7
修改/etc/profile

[[email protected] jdk1.7]# vi /etc/profile
[[email protected] jdk1.7]# source /etc/profile
[[email protected] jdk1.7]# 

然後執行mvn install

發現錯誤缺少profobuf的complier

根據Drill的Install.md中依賴內容為protoc 2.5.x complier

接下來準備：

安裝protobuf，下載http://code.google.com/p/protobuf/

http://code.google.com/p/protobuf/downloads/detail?name=protobuf-2.5.0.tar.gz

對應的protobuf-2.5.0.tar.gz

tar zxvf protobuf-2.5.0.tar.gz
cd protobuf-2.5.0
./configure
make
make check
make install
安裝結束。
驗證：
檢視是否安裝成功：protoc --version

[[email protected] protobuf-2.5.0]# protoc --version
libprotoc 2.5.0
[[email protected] protobuf-2.5.0]# 

接下來繼續:

mvn install

最後存在錯誤如下：

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.15:test (default-test) on project common: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:2.15:test failed: The forked VM terminated without saying properly goodbye. VM crash or System.exit called ?
[ERROR] Command was/bin/sh -c cd /drill/drill/common && /drill/jdk1.7/jre/bin/java -XX:MaxDirectMemorySize=4096M org.apache.maven.surefire.booter.ForkedBooter /drill/drill/common/target/surefire/surefire5854881226327277986tmp /drill/drill/common/target/surefire/surefire_02131510559628541214tmp
[ERROR] -> [Help 1]

測試通過

 git clone https://github.com/apache/incubator-drill.git
下載原始碼

[[email protected] incubator-drill]# pwd
/drill/incubator-drill
[[email protected] incubator-drill]# ls
common   distribution  header      KEYS     NOTICE   protocol   sample-data  sqlline    src     tools
contrib  exec          INSTALL.md  LICENSE  pom.xml  README.md  sandbox      sqlparser  target
[[email protected] incubator-drill]# 
執行測試：

    cd incubator-drill
    mvn clean install

編譯失敗存在錯誤：

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Invalid maximum direct memory size: -XX:MaxDirectMemorySize=4096M
The specified size exceeds the maximum representable size.
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

分析pom.xml檔案中

<plugin>
          <artifactId>maven-surefire-plugin</artifactId>
          <version>2.15</version>
          <configuration>
            <argLine>-XX:MaxDirectMemorySize=4096M </argLine>
          </configuration>
        </plugin>

分配給CentOS6.3的記憶體只有1128M

嘗試修改-XX:MaxDirectMemorySize=896M

重新進行mvn install,原來的錯誤已經不存從

執行到後面錯誤包括：

[INFO] exec/Java Execution Engine ........................ FAILURE [10:23.114s]
[INFO] SQL Parser ........................................ SKIPPED
[INFO] contrib/sqlline ................................... SKIPPED
[INFO] Packaging and Distribution Assembly ............... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 13:31.608s
[INFO] Finished at: Sat Nov 30 23:58:45 CST 2013
[INFO] Final Memory: 41M/146M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.15:test (default-test) on project java-exec: There are test failures.
[ERROR] 
[ERROR] Please refer to /drill/drill/exec/java-exec/target/surefire-reports for the individual test results.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :java-exec
[[email protected] drill]# 

存在test failures

Tests in error: 
  TestSimpleFunctions.testSubstring:122 » OutOfMemory Direct buffer memory
  TestSimpleFunctions.testIsNull:68 » OutOfMemory Direct buffer memory
  TestComparisonFunctions.testIntNullable:126->runTest:66 » OutOfMemory Direct b...
  TestComparisonFunctions.testInt:82->runTest:66 » OutOfMemory Direct buffer mem...
  TestComparisonFunctions.testBigIntNullable:136->runTest:66 » OutOfMemory Direc...
  TestComparisonFunctions.testBigInt:93->runTest:66 » OutOfMemory Direct buffer ...
  TestComparisonFunctions.testFloat4:104->runTest:66 » OutOfMemory Direct buffer...
  TestComparisonFunctions.testFloat8:115->runTest:66 » OutOfMemory Direct buffer...
  TestAgg.twoKeyAgg:92->doTest:63 » OutOfMemory Direct buffer memory
  TestAgg.oneKeyAgg:69->doTest:63 » OutOfMemory Direct buffer memory
  TestDistributedFragmentRun.oneBitOneExchangeOneEntryRun:50 »  test timed out a...
  TestDistributedFragmentRun.twoBitOneExchangeTwoEntryRun:106 »  test timed out ...
  TestDistributedFragmentRun.oneBitOneExchangeTwoEntryRun:69 »  test timed out a...
  TestDistributedFragmentRun.oneBitOneExchangeTwoEntryRunLogical:87 »  test time...
  TestMergeJoin.orderedEqualityLeftJoin:124 » NullPointer
  TestMergeJoin.orderedEqualityInnerJoin:180 » OutOfMemory Direct buffer memory
  TestMergeJoin.orderedEqualityMultiBatchJoin:232 » OutOfMemory Direct buffer me...
  TestMergeJoin.simpleEqualityJoin:72 » OutOfMemory Direct buffer memory
  TestEndianess.testLittleEndian:33 » OutOfMemory Direct buffer memory

相關例項：

testLittleEndian(org.apache.drill.exec.memory.TestEndianess)  Time elapsed: 0.735 sec  <<< ERROR!
java.lang.OutOfMemoryError: Direct buffer memory
    at java.nio.Bits.reserveMemory(Bits.java:658)
    at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
    at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:306)
    at io.netty.buffer.PoolArenaL$DirectArena.newChunk(PoolArenaL.java:381)
    at io.netty.buffer.PoolArenaL.allocateNormal(PoolArenaL.java:144)
    at io.netty.buffer.PoolArenaL.allocate(PoolArenaL.java:133)
    at io.netty.buffer.PoolArenaL.allocate(PoolArenaL.java:95)
    at io.netty.buffer.PooledByteBufAllocatorL.newDirectBuffer(PooledByteBufAllocatorL.java:236)
    at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:132)
    at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:123)
    at org.apache.drill.exec.memory.DirectBufferAllocator.buffer(DirectBufferAllocator.java:35)
    at org.apache.drill.exec.memory.TestEndianess.testLittleEndian(TestEndianess.java:33)

-------------------

在Cloudera的CDH4.3的虛擬機器下，安裝jdk1.7.0_45和protoc-2.5.0

然後嘗試多次

mvn e install最終完成安裝編譯

測試執行；

./sqlline -u jdbc:drill:schema=parquet-local -n admin -p admin

首先掃描是否有更新，如果沒有更新，啟動命令列

測試命令

SELECT 
      _MAP['R_REGIONKEY'] AS region_key, 
      _MAP['R_NAME'] AS name, _MAP['R_COMMENT'] AS comment
    FROM "sample-data/region.parquet";