1. 程式人生 > >linux中資料倉庫工具hive簡介及安裝部署詳解

linux中資料倉庫工具hive簡介及安裝部署詳解

簡介: Apache Hive是一個建立在Hadoop架構之上的資料倉庫。它能夠提供資料的精煉,查詢和分析。 hive是基於Hadoop的一個數據倉庫工具,可以將結構化的資料檔案對映為一張資料庫表,並提供簡單的SQL查詢功能,可以將SQL語句轉換為MapReduce任務進行執行。其優點是學習成本低,可以通過類SQL語句快速實現簡單的MapReduce統計,不必開發專門的MapReduce應用,十分適合資料倉庫的統計分析。 安裝及部署: 前期準備: hadoop、mysql的安裝,這裡就不詳細說了,詳情見我的專欄 https://blog.csdn.net/CowBoySoBusy/column/info/28102

hive安裝包下載地址: http://archive.cloudera.com/cdh5/cdh/5/hive-1.1.0-cdh5.7.0.tar.gz

安裝:

#解壓
tar -zxvf hive-1.1.0-cdh5.7.0.tar.gz 
#複製hive解壓到你定的位置
cp -r hive-1.1.0-cdh5.7.0.tar.gz /home/zq

在這裡插入圖片描述

mysql驅動包匯入: 把mysql驅動包放置到$HIVE_HOME\lib目錄下

配置環境變數:

vi /etc/profile
export HIVE_HOME=/usr/local/hive
export PATH=$PATH:$HIVE_HOME/bin
source
/etc/profile

hive-env.sh配置:

cd /home/zq/hive-1.1.0-cdh5.7.0/conf
#生成hive-env.sh檔案
cp -r hive-env.sh.template hive-env.sh
#配置
vi hive-env.sh
--------------------------------------------
	
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-amd64
export HADOOP_HOME=/home/zq/hadoop-2.6.0-cdh5.7.0
export HIVE_HOME=/home/zq/hive-1.1.0-cdh5.7.0
# HADOOP_HOME=${bin}/../../hadoop
# Hive Configuration Directory can be controlled by: export HIVE_CONF_DIR=$HIVE_HOME/conf # Folder containing extra ibraries required for hive compilation/execution can be controlled by: export HIVE_AUX_JARS_PATH=/home/zq/hive-1.1.0-cdh5.7.0/lib/*

hive-site.xml配置:

#生成hive-site.xml檔案
cp -r hive-default.xml.template hive-site.xml 
#配置
vi hive-site.xml
--------------------------------------------
<configuration>
	 <property>
    <name>hive.exec.local.scratchdir</name>
    <value>/home/zq/hive-1.1.0-cdh5.7.0/tmp</value>
    <description>Local scratch space for Hive jobs</description>
  </property>
	 <property>
    <name>hive.downloaded.resources.dir</name>
    <value>/home/zq/hive-1.1.0-cdh5.7.0/tmp</value>
    <description>Temporary local directory for added resources in the remote file system.</description>
  </property>
	 <property>
    <name>hive.querylog.location</name>
    <value>/home/zq/hive-1.1.0-cdh5.7.0/tmp</value>
    <description>Location of Hive run time structured log file</description>
  </property>
  <property>
    <name>hive.exec.scratchdir</name>
    <value>/home/zq/hive-1.1.0-cdh5.7.0/tmp</value>
  </property>
  <property>
    <name>hive.metastore.warehouse.dir</name>
    <value>/home/zq/hive-1.1.0-cdh5.7.0/warehouse</value>
    <description>location of default database for the warehouse</description>
  </property>
   <property>
  <name>hive.metastore.uris</name>
  <value>zq://127.0.0.1:9083</value>
  <description>Thrift URI for the remote metastore. Used by metastore client to connect to remote metastore.</description>
</property>
<property> 
   <name>hive.hwi.listen.port </name> 
   <value>9999</value> 
   <description>This is the port the Hive Web Interface will listen on </description> 
</property> 
<property>
    <name>hive.querylog.location</name>
    <value>/home/zq/hive-1.1.0-cdh5.7.0/log</value>
    <description>Location of Hive run time structured log file</description>
  </property>
  <property> 
   <name>datanucleus.fixedDatastore </name> 
   <value>false</value> 
</property> 
<property> 
   <name>datanucleus.autoCreateSchema </name> 
   <value>true</value> 
</property> 
 <property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true&amp;characterEncoding=UTF-8&amp;useSSL=false</value>
  </property>

  <property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
  </property>

  <property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>hive</value>
    <description>Username to use against metastore database</description>
  </property>

  <property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>hive</value>
  </property>
</configuration>

啟動hive: (1)啟動hadoop:start-all.sh (2)啟動mysql:service mysql start (3)啟動hive 直接輸入hive即可

hive --service hwi

然後開啟http://zq:9999/hwi/就能看到Hive的web頁面。