大資料BigData之如何配置hive連線mysql,把mysql作為元資料庫?
阿新 • • 發佈:2018-12-20
如何配置hive連線mysql,把mysql作為元資料庫?
文章目錄
1.增加mysql資料庫的連線配置
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true& useSSL=false</value>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
<description>Driver class name for a JDBC metastore</description>
</property>
<property>
<name> javax.jdo.option.ConnectionUserName</name>
<value>root</value>
<description>username to use against metastore database</description>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>password</value>
< description>password to use against metastore database</description>
</property>
2.修改臨時資料夾的路徑
找到以下2個配置,並改為正確的路徑:
<property>
<name>hive.exec.local.scratchdir</name>
<value>/home/lee/iotmp</value>
<description>Local scratch space for Hive jobs</description>
</property>
<property>
<name>hive.downloaded.resources.dir</name>
<value>/home/lee/iotmp</value>
<description>Temporary local directory for added resources in the remote file system.</description>
</property>
這裡因為我當前使用者是hduser,所以我在hduser的目錄下建立一個iotmp資料夾,並授權:
$ mkdir -p /home/lee/iotmp
$ chmod -R 775 /home/lee/iotmp
3.修改 hive-config.sh
進入到 hive 的 bin 目錄下 …/hive/bin/
$ vi hive-config.sh
在該檔案的最前面加入以下配置:
//你的JAVA_HOME
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
//你的HADOOP_HOME
export HADOOP_HOME=/usr/local/hadoop
//你的HIVE_HOME
export HIVE_HOME=/usr/local/hive
4.下載MySQL JDBC驅動
$ wget http://central.maven.org/maven2/mysql/mysql-connector-java/5.1.41/mysql-connector-java-5.1.41.jar
下載完後,把jar包複製到…/hive/lib目錄下面
5.在HDFS中建立目錄和設定許可權
啟動Hadoop,在Hadoop中建立hive需要用到的目錄並設定好許可權:
$ hadoop fs -mkdir /tmp
$ hadoop fs -mkdir -p /home/lee/hive/warehouse
$ hadoop fs -chmod g+w /tmp
$ hadoop fs -chmod g+w /home/lee/hive/warehouse
6.初始化元資料庫
進入 你的hive lib 目錄,我的是在/home/lee/hive/lib,初始化Hive元資料對應的MySQL資料庫:
$ schematool -initSchema -dbType mysql