1. 程式人生 > >CentOS 7部署Hive 3.1.0

CentOS 7部署Hive 3.1.0

一、環境

軟體 版本
Hadoop 3.1.1
Hive 3.1.0
MySQL 8.0.11
JDK 1.8

二、安裝Hive

Hive只要在一臺Hadoop叢集的伺服器上安裝即可,安裝到/usr/local/hive目錄下。執行如下命令:

mkdir -p /usr/local/hive
tar -zxvf apache-hive-3.1.0-bin.tar.gz -C /usr/local/hive

接下來配置環境變數,執行命令:vim /etc/profile,開啟profile檔案,在檔案末尾新增Hive配置:

export HIVE_HOME=/usr/local/hive/apache-hive-3.1.0-bin
export PATH=$PATH:$HIVE_HOME/bin

配置好Hive環境變數後,執行命令:source /etc/profile,使環境配置生效。

三、配置Hive

執行命令:cd /usr/local/hive/apache-hive-3.1.0-bin/conf,進入Hive配置目錄,執行如下命令複製配置檔案:

cp hive-env.sh.template hive-env.sh
cp hive-default.xml.template hive-site.xml
cp hive-log4j2.properties.template hive-log4j2.properties

1、配置hive-env.h

配置java,hadoop,hive目錄,如下:

export JAVA_HOME=/usr/local/java/jdk1.8.0_181
export HADOOP_HOME=/usr/local/hadoop/hadoop-3.1.1
export HIVE_HOME=/usr/local/hive/apache-hive-3.1.0-bin
export HIVE_CONF_DIR=$HIVE_HOME/conf
export HIVE_AUX_JARS_PATH=$HIVE_HOME/lib/*

2、配置hive-site.xml

配置MySQL地址、登入名、密碼、驅動等。
下載MySQL驅動並複製到lib目錄下:cp mysql-connector-java-8.0.11.jar /usr/local/hive/apache-hive-3.1.0-bin/lib。
配置如下:

<property>
    <name>system:java.io.tmpdir</name>
    <value>/usr/local/hive/apache-hive-3.1.0-bin/tmp</value>
</property>
<property>
    <name>system:user.name</name>
    <value>${user.name}</value>
</property>
<property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>root</value>
    <description>Username to use against metastore database</description>
</property>
<property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>mysql</value>
    <description>password to use against metastore database</description>
</property>
<property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://192.168.0.199:3306/hive?createDatabaseIfNotExist=true&amp;characterEncoding=UTF-8&amp;useSSL=false</value>
    <description>
        JDBC connect string for a JDBC metastore.
        To use SSL to encrypt/authenticate the connection, provide database-specific SSL flag in the connection URL.
        For example, jdbc:postgresql://myhost/db?ssl=true for postgres database.
    </description>
</property>
<property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.cj.jdbc.Driver</value>
    <description>Driver class name for a JDBC metastore</description>
</property>
<property>
    <name>hive.metastore.schema.verification</name>
    <value>false</value>
    <description>
        Enforce metastore schema version consistency.
        True: Verify that version information stored in is compatible with one from Hive jars.  Also disable automatic
        schema migration attempt. Users are required to manually migrate schema after Hive upgrade which ensures
        proper metastore schema migration. (Default)
        False: Warn if the version information stored in metastore doesn't match with one from in Hive jars.
    </description>
</property>

3、配置hive-log4j2.properties

新建並配置日誌目錄:

property.hive.log.dir = /usr/local/hive/apache-hive-3.1.0-bin/logs

四、啟動Hive

在啟動Hive前,先在MySQL下建立Hive資料庫,執行如下命令:
schematool -dbType mysql -initSchema
建立好資料庫後,執行命令:hive,啟動並進入hive,如下圖所示:
在這裡插入圖片描述

五、驗證Hive

hive常用命令:

show databases;																	//檢視資料庫
create database database_name;													//新建資料庫	
use database_name;																//使用資料庫
drop database database_name;													//刪除資料庫
show tables;																	//檢視資料表
show tables in database_name;													//檢視資料庫下的表
create table ht_big_data (id int, name string)  row format delimited fields terminated by '\t';		//建表
show create table table_name;																		//檢視建表語句
load data local inpath 'bigdata.txt' overwrite into table ht_big_data;								//載入資料到表中
load data inpath '/hdfs/hive/bigdata.txt' into table ht_big_data;									//從hdfs中追加資料到hive表
select * from table_name;																			//檢視資料