1. 程式人生 > >hive 2.1.0安裝

hive 2.1.0安裝

前言:
本文主要是從如下blog中學習得來:

http://blog.csdn.net/cuihaolong/article/details/52038543

http://fengshulin.iteye.com/blog/2320359

1.準備:

 java 1.7

hadoop 2.6.4

2.準備hive和mysql安裝包:

3.解壓hive:

下載安裝包後用tar -xzvf命令解壓,放在/usr/local/hadoop目錄下。

4. 設定Hive環境變數

    vim ~/.bashrc  
      
    export HIVE_HOME=/usr/local/hadoop/apache-hive-2.1.0-bin 
    export PATH=$JAVA_HOME/bin:$PATH:$HADOOP_HOME/bin:$HIVE_HOME/bin  
      
    source ~/.bashrc  

4. Hive的配置檔案
    cp hive-default.xml.template hive-site.xml  
    cp hive-log4j.properties.template hive-log4j.properties  
    cp hive-exec-log4j.properties.template hive-exec-log4j.properties  
    cp hive-env.xml.template hive-env.xml

5. 修改hive配置檔案。

5.1 修改hive-env.sh(新增)

HADOOP_HOME=/home/hadoop
export HIVE_CONF_DIR=/home/hive/conf    

5.2修改 hive-site.xml (新增下面部分)

<property>
   <name>javax.jdo.option.ConnectionURL</name>
   <value>jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true</value>
   <description>JDBC connect string for a JDBC metastore</description>
</property>
<property>
  <name>javax.jdo.option.ConnectionDriverName</name>
  <value>com.mysql.jdbc.Driver</value>
  <description>Driver class name for a JDBC metastore</description>
</property>
<property>
  <name>javax.jdo.option.ConnectionUserName</name>
  <value>hive</value>
  <description>username to use against metastore database</description>
</property>
<property>
  <name>javax.jdo.option.ConnectionPassword</name>
  <value>hive</value>
  <description>password to use against metastore database</description>
</property>
6.修改 hive-site.xml 替換${system:java.io.tmpdir} 和 ${system:user.name}為/home/hadoop/hive/tmp/,否則會報如下錯誤 
    Logging initialized using configuration in file:/usr/local/hive/conf/hive-log4j2.properties Async: true  
    Exception in thread "main" java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D  
        at org.apache.hadoop.fs.Path.initialize(Path.java:205)  
        at org.apache.hadoop.fs.Path.<init>(Path.java:171)  
        at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:631)  
        at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:550)  
        at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:518)  
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:705)  
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:641)  
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)  
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)  
        at java.lang.reflect.Method.invoke(Method.java:498)  
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)  
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)  
    Caused by: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D  
        at java.net.URI.checkPath(URI.java:1823)  
        at java.net.URI.<init>(URI.java:745)  
        at org.apache.hadoop.fs.Path.initialize(Path.java:202)  
        ... 12 more  


7.MySQL

7.1安裝MySQL

Hive預設使用derby資料庫儲存元資料,但是該資料庫不適用於生產環境,這邊使用MySQL作為元資料的儲存資料庫。
所以需要先安裝好MySQL

直接安裝mysql,可以用  sudo apt-get install mysql-server mysql-client  來安裝

7.2建立MySQL使用者(新建mysql使用者,使用者名稱是hive,密碼也是hive,對應 hive-site.xml檔案裡面的使用者密碼設定)

create user 'hive' identified by 'hive';
grant all privileges on *.* to 'hive' with grant option;
flush privileges;
create database hive;

7.3拷貝MySQL驅動檔案

下載地址:http://dev.mysql.com/downloads/connector/j/ ,解壓後拷貝其中的mysql-connector-Java-5.1.40-bin.jar到hive的lib資料夾下。

8.啟動Hive

先開啟Hadoop叢集和mysql,再執行hive

啟動mysql:

/etc/init.d/mysql start
開啟hive
./bin/hive

9.啟動Hive報錯(如無請忽略)

    Caused by: MetaException(message:Hive metastore database is not initialized. Please use schematool (e.g. ./schematool -initSchema -dbType ...) to create the schema.  
    If needed, don't forget to include the option to auto-create the underlying database in your JDBC connection string (e.g. ?createDatabaseIfNotExist=true for mysql))  
    這是由於沒有初始化Hive元資料的資料庫,預設情況下,Hive的元資料儲存在了內嵌的derby資料庫裡  
      
    (1)先刪除 ../hive/bin目錄下的metastore_db資料夾
    (2)執行如下命令方可  
    ./bin/schematool -initSchema -dbType derby  

10.測試Hive
    create table table_name (  
      id                int,  
      dtDontQuery       string,  
      name              string  
    );  
    show tables;