大數據之---Hive全網最詳細的編譯tar及部署終極篇
| RHEL6 | 角色 | jdk |
| hadoop-2.8.1.tar.gz | ||
| apache-maven-3.3.9 | mysql-5.1 | |
| xx.xx.xx.xx ip地址 | NN | hadoop01 |
| xx.xx.xx.xx ip地址 | DN | hadoop02 |
| xx.xx.xx.xx ip地址 | DN | hadoop03 |
| xx.xx.xx.xx ip地址 | DN | hadoop04 |
| xx.xx.xx.xx ip地址 | DN | hadoop05 |
本次涉及偽分布式部署只是要主機hadoop01，軟件安裝參考偽分布式部署終極篇
2、安裝jdk
mkdir /usr/java && cd /usr/java/
tar -zxvf /tmp/server-jre-7u80-linux-x64.tar.gz
echo 'export JAVA_HOME=/usr/java/jdk1.7.0_80'>>/etc/profile
source /etc/profile
3、安裝maven
cd /usr/local/
unzip /tmp/apache-maven-3.3.9-bin.zip
chown root: /usr/local/apache-maven-3.3.9 -R
echo 'export MAVEN_HOME=/usr/local/apache-maven-3.3.9'>>/etc/profile
echo 'export MAVEN_OPTS="-Xms256m -Xmx512m"'>>/etc/profile
source /etc/profile
--JDK 和maven部署可參考---大數據之----部署安裝編譯打包hadoop終極篇
4、安裝mysql
yum -y install mysql-server mysql
/etc/init.d/mysqld start
chkconfig mysqld on
mysqladmin -u root password 123456
mysql -uroot -p123456
use mysql;
GRANT ALL PRIVILEGES ON *.* TO 'root'@'localhost' IDENTIFIED BY 'v123456' WITH GRANT OPTION;
GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY '123456' WITH GRANT OPTION;
update user set password=password('123456') where user='root';
delete from user where not (user='root') ;
delete from user where user='root' and password='';
drop database test;
DROP USER ''@'%';
flush privileges;
5、下載hive源碼包：
# http://archive.cloudera.com/cdh5/cdh/5/
# 根據cdh版本選擇對應hive軟件包：
# hive-1.1.0-cdh5.7.1-src.tar.gz
# 解壓後使用maven命令編譯成安裝包
6、編譯:
cd /tmp/
tar -xf hive-1.1.0-cdh5.7.1-src.tar.gz
cd /tmp/hive-1.1.0-cdh5.7.1
mvn clean package -DskipTests -Phadoop-2 -Pdist
# 編譯生成的包在以下位置：
# packaging/target/apache-hive-1.1.0-cdh5.7.1-bin.tar.gz
7、安裝編譯生成的Hive包，然後測試
cd /usr/local/
tar -xf /tmp/apache-hive-1.1.0-cdh5.7.1-bin.tar.gz
ln -s apache-hive-1.1.0-cdh5.7.1-bin hive
chown -R hadoop:hadoop apache-hive-1.1.0-cdh5.7.1-bin
chown -R hadoop:hadoop hive
echo 'export HIVE_HOME=/usr/local/hive'>>/etc/profile
echo 'export PATH=$HIVE_HOME/bin:$PATH'>>/etc/profile
8、更改環境變量
su - hadoop
cd /usr/local/hive
cd conf
1、hive-env.sh
cp hive-env.sh.template hive-env.sh&&vi hive-env.sh
HADOOP_HOME=/usr/local/hadoop
2、hive-site.xml
vi hive-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://localhost:3306/vincent_hive?createDatabaseIfNotExist=true</value>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>root</value>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>vincent</value>
</property>
</configuration>
9、拷貝mysql驅動包到$HIVE_HOME/lib
# 上方的hive-site.xml使用了java的mysql驅動包
# 需要將這個包上傳到hive的lib目錄之下
# 解壓 mysql-connector-java-5.1.45.zip 對應的文件到目錄即可
cd /tmp
unzip mysql-connector-java-5.1.45.zip
cd mysql-connector-java-5.1.45
cp mysql-connector-java-5.1.45-bin.jar /usr/local/hive/lib/
未拷貝有相關報錯：
The specified datastore driver ("com.mysql.jdbc.Driver") was not found in the CLASSPATH.
Please check your CLASSPATH specification,
and the name of the driver.
大數據之---Hive全網最詳細的編譯tar及部署終極篇