Hive架構簡圖

Hive架構簡圖 hive原理與原始碼分析-hive原始碼架構與理論

Hive3.x安裝準備工作

詳細參見：Hive3.x 安裝與debug

1 Hive命令列提交查詢

SELECT deptno, count(deptname) as deptno_cnt from hive3_test.depts group by deptno;

執行棧

下面簡要分析關鍵路徑程式碼

2 CliDriver接收查詢

public int processCmd(String cmd) {
  ...
  processLocalCmd(String cmd,
 
 CommandProcessor proc, CliSessionState ss) {
      ret = (ReExecDriver)qp.run(cmd).getResponseCode();
      這裡run呼叫Driver::run
      //public CommandProcessorResponse run(String command) //見下文
      ret = processLocalCmd(cmd, proc, ss);//待單獨分析
    }
  ...
  
}

3 ReExecDriver compileAndRespond查詢

public
 
 CommandProcessorResponse run(String command) {
    CommandProcessorResponse r0 = compileAndRespond(command);
    private void runInternal(String command, boolean alreadyCompiled){
    }
}

compileAndRespond中呼叫Driver compile查詢

public CommandProcessorResponse compileAndRespond(String command, boolean
 
 cleanupTxnList){
   private void compileInternal(String command, boolean deferClose){
	     Driver::compile(command, true, deferClose);
    }
}

4 Driver compile請求

private void compile(String command, boolean resetTaskIds, boolean deferClose) throws CommandProcessorResponse {
   ......
   //ParseUtils.parse 見command  sql解析為AST
   try {
        tree = ParseUtils.parse(command, ctx);
      } catch (ParseException e) {
        parseError = true;
        throw e;
      } finally {
        hookRunner.runAfterParseHook(command, parseError);
      }
  ......

  //語法分析 Do semantic analysis and plan generation
  BaseSemanticAnalyzer sem = SemanticAnalyzerFactory.get(queryState, tree);
  sem.analyze(tree, ctx);

  //檢測語法合法性 validate the plan
  sem.validate();
  .....
  // 邏輯執行計劃
  plan = new QueryPlan(queryStr, sem, perfLogger.getStartTime(PerfLogger.DRIVER_RUN), queryId,
        queryState.getHiveOperation(), schema);
}

4.1 ParseUitls SQL AST解析

ParseUitls
/** Parses the Hive query. */
public static ASTNode parse(
    String command, Context ctx, String viewFullyQualifiedName) throws ParseException {
  ParseDriver pd = new ParseDriver();
  ASTNode tree = pd.parse(command, ctx, viewFullyQualifiedName);
  tree = findRootNonNullToken(tree);
  handleSetColRefs(tree);
  return tree;
}

ParseUitls 調起HiveParser

ParseDriver
public ASTNode parse(String command, Context ctx, String viewFullyQualifiedName){

HiveParser parser = new HiveParser(tokens);
...
ASTNode tree = (ASTNode) r.getTree();
}

ASTNode的結構如下

4.2 BaseSemanticAnalyzer語法分析

接上文sem.analyze(tree, ctx);語法分析

會先進入calciteplanner進行查詢優化
CalcitePlanner待續
public void analyzeInternal(ASTNode ast) throws SemanticException {
    if (runCBO) {
      super.analyzeInternal(ast, new PlannerContextFactory() {
        @Override
        public PlannerContext create() {
          return new PreCboCtx();
        }
      });
    } else {
      super.analyzeInternal(ast);
    }
}