【一】hive安裝(遠端metastore)
前期:請先安裝jdk和hadoop和mysql
環境ubuntu16.04
下載
http://mirrors.tuna.tsinghua.edu.cn/apache/hive/
rz上傳安裝包到伺服器
解壓
tar -zxvf apache-hive-2.3.3-bin.tar.gz
修改名字檔名字
mv apache-hive-2.3.3-bin hive
配置環境變數
vi ~/.bashrc
export HIVE_HOME=/app/hive
export PATH=$PATH:$HIVE_HOME/bin
載入配置檔案
source ~/.bashrc
檢視是否安裝好
hive -version
配置
cd /app/hive/conf
修改配置檔案hive-site.xml
(如果沒有這個檔案則複製一個
cp hive-default.xml.template hive-site.xml
)
元資料放在遠端mysql中
vi hive-site.xml
<property> <name>hive.metastore.warehouse.dir</name> <value>hdfs://hadoopcluster/user/hive/warehouse</value> <description>hadoop是HA,否則用hdfs://node1:9000/user/hive/warehouse</description> </property> <property> <name>hive.exec.scratchdir</name> <value>hdfs://hadoopcluster/hive/scratchdir</value> <description>Scratch space for Hive jobs</description> </property> <property> <name>javax.jdo.option.ConnectionUserName</name> <value>root</value> </property> <property> <name>javax.jdo.option.ConnectionPassword</name> <value>mysql對應使用者的密碼</value> </property> <property> <name>javax.jdo.option.ConnectionURL</name> <value>jdbc:mysql://192.168.2.103:3306/hive?createDatabaseIfNotExist=true&useSSL=true</value> </property> <property> <name>javax.jdo.option.ConnectionDriverName</name> <value>com.mysql.jdbc.Driver</value> </property> <property> <name>hive.metastore.uris</name> <value>thrift://node1:9083</value> </property>
<property>
<name>hive.metastore.schema.verification</name>
<value>false</value>
</property>
拷貝mysql的驅動到hive的lib中
cd /app/hive/lib
修改配置檔案hive-env.sh
cd /app/hive/conf
(如果沒有則拷貝:cp hive-env.sh.template hive-env.sh)
vi hive-env.sh
HADOOP_HOME=/app/hadoop/hadoop-2.9.0
進入hive互動介面
hive
報錯
解決辦法在hive-site.xml中加入
<property>
<name>system:java.io.tmpdir</name>
<value>/app/hive/tmpdir</value>
</property>
<property>
<name>system:user.name</name>
<value>hive</value>
</property>
報錯
FAILED: SemanticException org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Hive metastore database is not initialized. Please use schematool (e.g. ./schematool -initSchema -dbType ...) to create the schema. If needed, don't forget to include the option to auto-create the underlying database in your JDBC connection string (e.g. ?createDatabaseIfNotExist=true for mysql))
解決方法:初始化schema資訊
cd /app/hive/bin
schematool -initSchema -dbType mysql
測試
create database hive;
use hive;