CentOS 7.3 CDH 5.10.0 Druid0.12.4安裝記錄
CentOS 7.3 CDH 5.10.0安裝記錄
0. 集群規劃
192.167.1.247 realtime247 realtime+hadoopdata
192.167.1.248 broker248 broker
192.167.1.249 history249 middleware+history+hadoopdata
192.167.1.250 history250 middleware+history+hadoopdata
192.167.1.251 master251 Mysql+coordinator+overlord+hadoopMaster
1. 環境準備
安裝第三方依賴包：（所有節點，註意一定要安裝)
yum install -y chkconfig python bind-utils psmisc libxslt zlib sqlite cyrus-sasl-plain cyrus-sasl-gssapi fuse fuse-libs redhat-lsb
echo never > /sys/kernel/mm/transparent_hugepage/defrag
echo never > /sys/kernel/mm/transparent_hugepage/defrag
修改swappiness參數:
vi /etc/sysctl.conf 文件裏添加如下參數：
或者：
echo ‘vm.swappiness=0‘>> /etc/sysctl.conf
JDK 1.8 RPM包:
http://download.oracle.com/otn-pub/java/jdk/8u172-b11/a58eab1ec242421181065cdc37240b08/jdk-8u172-linux-x64.rpm?AuthParam=1527227488_e083c615628d8b91527508f00008dd6d
下載cloudera-manager，parcel：
https://archive.cloudera.com/cm5/cm/5/cloudera-manager-centos7-cm5.10.0_x86_64.tar.gz
http://archive.cloudera.com/cdh5/parcels/5.10.0/CDH-5.10.0-1.cdh5.10.0.p0.41-el7.parcel
http://archive.cloudera.com/cdh5/parcels/5.10.0/CDH-5.10.0-1.cdh5.10.0.p0.41-el7.parcel.sha1
http://archive.cloudera.com/cdh5/parcels/5.10.0/manifest.json
MYSQL安裝包：
https://cdn.mysql.com/archives/mysql-5.6/MySQL-server-5.6.30-1.el7.x86_64.rpm
https://cdn.mysql.com/archives/mysql-5.6/MySQL-client-5.6.30-1.el7.x86_64.rpm
https://cdn.mysql.com/archives/mysql-5.6/MySQL-shared-5.6.30-1.el7.x86_64.rpm
https://cdn.mysql.com/archives/mysql-5.6/MySQL-shared-compat-5.6.30-1.el7.x86_64.rpm
https://cdn.mysql.com/archives/mysql-5.6/MySQL-embedded-5.6.30-1.el7.x86_64.rpm
https://cdn.mysql.com/archives/mysql-5.6/MySQL-devel-5.6.30-1.el7.x86_64.rpm
或者
https://cdn.mysql.com/archives/mysql-5.6/MySQL-5.6.30-1.el7.x86_64.rpm-bundle.tar
jdbc連接包mysql-connector-java.jar:
http://dev.mysql.com/downloads/connector/j/
2. 每臺機器安裝JDK
2.1 檢查是否安裝了JDK： rpm -qa | grep jdk
2.2 如果已經安裝了JDK，則卸載: yum -y remove [查詢出來的包名]
yum -y remove java-1.7.0-openjdk-headless-1.7.0.111-2.6.7.8.el7.x86_64
yum -y remove java-1.8.0-openjdk-1.8.0.102-4.b14.el7.x86_64
yum -y remove java-1.7.0-openjdk-1.7.0.111-2.6.7.8.el7.x86_64
yum -y remove copy-jdk-configs-1.2-1.el7.noarch
yum -y remove java-1.8.0-openjdk-headless-1.8.0.102-4.b14.el7.x86_64
2.3 安裝下載下來的JDK RPM包:
rpm -ivh /home/workspace/zipped/jdk-8u172-linux-x64.rpm
2.4 設置JDK環境變量
vi /etc/profile
export JAVA_HOME=/usr/java/jdk1.8.0_172-amd64
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
source /etc/profile #環境變量生效
3. 配置網絡，修改機器名，關閉防火墻，關閉selinux，安裝NTP時間同步
3.1 配置固定IP地址（每臺機器，IP每臺自己選擇變化)
vi /etc/sysconfig/network-scripts/ifcfg-eth0
TYPE=Ethernet
BOOTPROTO=static
DEFROUTE=yes
NAME=eth0
UUID=7ac09286-c35b-4f15-a9ba-701c093832bf
DEVICE=eth0
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
IPV6_PRIVACY=no
ONBOOT=yes
DNS1=192.168.1.1
IPADDR=192.168.1.246
PREFIX=24
GATEWAY=192.168.1.1
3.2 修改機器名（在各臺機器上執行相應的腳本)
192.167.1.247
hostnamectl set-hostname realtime247
hostnamectl --static set-hostname realtime247
192.167.1.248
hostnamectl set-hostname broker248
hostnamectl --static set-hostname broker248
192.167.1.249
hostnamectl set-hostname history249
hostnamectl --static set-hostname history249
192.167.1.250
hostnamectl set-hostname history250
hostnamectl --static set-hostname history250
192.167.1.251
hostnamectl set-hostname master251
hostnamectl --static set-hostname master251
修改hosts文件
vi /etc/hosts
127.0.0.1 localhost
::1 localhost
192.168.1.247 realtime247
192.168.1.248 broker248
192.168.1.249 history249
192.168.1.250 history250
192.168.1.251 master251
3.3 關閉防火墻
systemctl stop firewalld #關閉防火墻
systemctl disable firewalld #禁用防火墻
systemctl status firewalld #查看防火墻狀態
3.4 selinux關閉(所有節點官方文檔要求)
vi /etc/selinux/config
SELINUX=disabled
#註意： 重啟才能生效
#重啟後檢查
sestatus -v
#SELinux status: disabled
3.5 NTP服務器設置(用於不同節點間實現時間同步)
3.5.1 yum -y install ntp #所有節點安裝時間同步服務
3.5.2 master節點更改時間
更改master的節點的時間同步服務器
vi /etc/ntp.conf
# 註釋掉所有server *.*.*的指向，新添加一條可連接的ntp服務器(百度一下ntp服務器，我選的是上海交大的)
server ntp.sjtu.edu.cn iburst
restrict default nomodify notrap
driftfile /var/lib/ntp/drift
restrict 127.0.0.1
restrict -6 ::1
includefile /etc/ntp/crypto/pw
keys /etc/ntp/keys
3.5.3 在其他節點上把ntp指向master服務器地址即可(/etc/ntp.conf下)
vi /etc/ntp.conf
server master251 iburst
3.5.4 啟動並設置開機啟動
systemctl start ntpd.service #啟動時間同步服務
systemctl enable ntpd.service #設置開機啟動
3.5.5 時間同步(主節點不做）
ntpdate -u master251
4. 設置SSH免密登陸
A） 每臺機器生成訪問秘鑰，復制到192.168.1.251:/home/workspace目錄下
192.168.1.247：
ssh-keygen -t rsa -P ‘‘ -f ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
scp ~/.ssh/authorized_keys 192.168.1.251:/home/workspace/authorized_keys247
rm -rf ~/.ssh/authorized_keys #刪除公鑰文件
192.168.1.248：
ssh-keygen -t rsa -P ‘‘ -f ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
scp ~/.ssh/authorized_keys 192.168.1.251:/home/workspace/authorized_keys248
rm -rf ~/.ssh/authorized_keys #刪除公鑰文件
192.168.1.249：
ssh-keygen -t rsa -P ‘‘ -f ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
scp ~/.ssh/authorized_keys 192.168.1.251:/home/workspace/authorized_keys249
rm -rf ~/.ssh/authorized_keys #刪除公鑰文件
192.168.1.250：
ssh-keygen -t rsa -P ‘‘ -f ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
scp ~/.ssh/authorized_keys 192.168.1.251:/home/workspace/authorized_keys250
rm -rf ~/.ssh/authorized_keys #刪除公鑰文件
192.168.1.251：
ssh-keygen -t rsa -P ‘‘ -f ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
cp ~/.ssh/authorized_keys /home/workspace/authorized_keys251
rm -rf ~/.ssh/authorized_keys #刪除公鑰文件
B） 在192.168.1.251上將所有的共鑰合並成一個公鑰文件
cat /home/workspace/authorized_keys247 >> /home/workspace/authorized_keys
cat /home/workspace/authorized_keys248 >> /home/workspace/authorized_keys
cat /home/workspace/authorized_keys249 >> /home/workspace/authorized_keys
cat /home/workspace/authorized_keys250 >> /home/workspace/authorized_keys
cat /home/workspace/authorized_keys251 >> /home/workspace/authorized_keys
C） 將合並後的公鑰文件復制到集群中的各個主機中
scp /home/workspace/authorized_keys 192.168.1.247:~/.ssh/
scp /home/workspace/authorized_keys 192.168.1.248:~/.ssh/
scp /home/workspace/authorized_keys 192.168.1.249:~/.ssh/
scp /home/workspace/authorized_keys 192.168.1.250:~/.ssh/
cp /home/workspace/authorized_keys ~/.ssh/ #因為目前在251主機中，所以使用的命令為cp而不是scp
D） 每臺機器：
chmod 600 ~/.ssh/authorized_keys #authorized_keys的訪問權限
chmod 600 ~/.ssh/
5. 安裝MYSQL
A) 查詢MYSQL是否安裝，如果安裝了，則卸載
rpm -qa | grep mariadb
rpm -qa | grep mysql
卸載命令：
# rpm -e [查詢出來的MYSQL包名] --nodeps
註意：如果卸載不了就強制卸載，--nodeps --force
yum -y remove [查詢出來的MYSQL包名]
rm -rf /var/lib/mysql
rm /etc/my.cnf
B） 安裝MYSQL
rpm -ivh MySQL-devel-5.6.30-1.el7.x86_64.rpm
rpm -ivh MySQL-shared-5.6.30-1.el7.x86_64.rpm
rpm -ivh MySQL-server-5.6.30-1.el7.x86_64.rpm
rpm -ivh MySQL-client-5.6.30-1.el7.x86_64.rpm
C) 設置MYSQL
C-1) systemctl start mysql #啟動MYSQL ##>>>MYSQL啟動後，默認其實己經生成了默認密碼，密碼保存在/root/.mysql_secret文件中
C-2) chkconfig mysql on #設置開機啟動
C-3) 修改MYSQL默認密碼
方法1：
mysql -uroot -p[初始化密碼] #cat /root/.mysql_secret 即可看到初始化密碼
SET PASSWORD=PASSWORD(‘root‘); （重置密碼）
方法2：
service mysql stop
mysqld_safe --user=mysql --skip-grant-tables --skip-networking&
mysql -u root mysql
mysql> use mysql;
mysql> UPDATE user SET Password=PASSWORD(‘root‘)where USER=‘root‘;
mysql> FLUSH PRIVILEGES;
mysql> quit
# service mysql restart
# mysql -u root –p
Enter password: root
mysql>SET PASSWORD=PASSWORD(‘root‘); （重置密碼）
mysql> use mysql; (此DB存放MySQL的各種配置信息)
Database changed
mysql> select host,user from user; (查看是否能夠查詢)
mysqladmin -uroot password ‘新密碼‘ #設置密碼
如果數據庫如法啟動，使用下面的腳本進行初始化：
mysql_install_db --user=mysql --ldata=/var/lib/mysql/
C-4) 設置遠程訪問
GRANT ALL PRIVILEGES ON *.* TO ‘新用戶‘@‘%‘ IDENTIFIED BY ‘新用戶的密碼‘ WITH GRANT OPTION;
FLUSH PRIVILEGES;
C-5) 復制jdbc mysql 訪問包到java lib目錄下
如果/usr/share/java/目錄不存在，執行
mkdir /usr/share/java/
cp mysql-connector-java-5.1.40-bin.jar /usr/share/java/mysql-connector-java.jar
或
cp mysql/mysql-connector-java-5.1.40-bin.jar /usr/java/jdk1.8.0_172-amd64/lib/mysql-connector-java.jar
C-6） 創建數據庫
create database hive DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database amon DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database hue DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database monitor DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database oozie DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
grant all on *.* to root@"%" Identified by "root";
6. 安裝Cloudera-Manager
A） 解壓cm tar包到指定目錄所有服務器都要(或者在主節點解壓好，然後通過scp到各個節點同一目錄下)
scp /home/workspace/zipped/cloudera-manager-centos7-cm5.10.0_x86_64.tar.gz 192.168.1.247:/home/workspace/zipped/
scp /home/workspace/zipped/cloudera-manager-centos7-cm5.10.0_x86_64.tar.gz 192.168.1.248:/home/workspace/zipped/
scp /home/workspace/zipped/cloudera-manager-centos7-cm5.10.0_x86_64.tar.gz 192.168.1.249:/home/workspace/zipped/
scp /home/workspace/zipped/cloudera-manager-centos7-cm5.10.0_x86_64.tar.gz 192.168.1.250:/home/workspace/zipped/
mkdir /opt/cloudera-manager
tar -axvf /home/workspace/zipped/cloudera-manager-centos7-cm5.10.0_x86_64.tar.gz -C /opt/cloudera-manager
B) 創建cloudera-scm用戶（所有節點）
useradd --system --home=/opt/cloudera-manager/cm-5.10.0/run/cloudera-scm-server --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm
C) 在主節點創建cloudera-manager-server的本地元數據保存目錄
mkdir /var/cloudera-scm-server
chown cloudera-scm:cloudera-scm /var/cloudera-scm-server
chown cloudera-scm:cloudera-scm /opt/cloudera-manager
D) 配置從節點cloudera-manger-agent指向主節點服務器(master251節點不做此操作)
vi /opt/cloudera-manager/cm-5.10.0/etc/cloudera-scm-agent/config.ini
將server_host改為CMS所在的主機名即master251
E) 主節點中創建parcel-repo倉庫目錄(只在master251節點操作)
mkdir -p /opt/cloudera/parcel-repo
ln -s /var/www/html/cdh/510/parcels/ parcel-repo
chown cloudera-scm:cloudera-scm /opt/cloudera/parcel-repo
#cp CDH-5.10.0-1.cdh5.10.0.p0.41-el7.parcel CDH-5.10.0-1.cdh5.10.0.p0.41-el7.parcel.sha1 manifest.json /opt/cloudera/parcel-repo
#mv CDH-5.10.0-1.cdh5.10.0.p0.41-el7.parcel.sha1 CDH-5.10.0-1.cdh5.10.0.p0.41-el7.parcel.sha #把CDH-5.10.0-1.cdh5.10.0.p0.41-el7.parcel.sha1 後綴要把1去掉
F) 所有節點創建parcels目錄(一定要在所有節點操作）
#解釋：Clouder-Manager將CDHs從主節點的/opt/cloudera/parcel-repo目錄中抽取出來，分發解壓激活到各個節點的/opt/cloudera/parcels目錄中】
mkdir -p /opt/cloudera/parcels #創建parcels目錄
chown cloudera-scm:cloudera-scm /opt/cloudera/parcels #給cloudera-scm授權目錄讀寫權限
G）初始腳本配置數據庫scm_prepare_database.sh(在主節點master251上操作)
#mysql -u root -proot #使用root用戶登陸mysql
#create database cloudera DEFAULT CHARSET utf8 COLLATE utf8_general_ci; #創建cloudera數據庫
grant all on *.* to root@"%" Identified by "root"; #授權
/opt/cloudera-manager/cm-5.10.0/share/cmf/schema/scm_prepare_database.sh mysql -hlocalhost -uroot -proot --scm-host master251 cloudera root root
說明：這個腳本就是用來創建和配置CMS需要的數據庫的腳本。各參數是指：
mysql：數據庫用的是mysql，如果安裝過程中用的oracle，那麽該參數就應該改為oracle。
-hlocalhost：數據庫建立在localhost主機上面。也就是主節點上面。
-uroot：root身份運行mysql。-123456：mysql的root密碼是***。
--scm-host hadoop1：CMS的主機，一般是和mysql安裝的主機是在同一個主機上。
最後三個參數是：數據庫名，數據庫用戶名，數據庫密碼。
H） 主節點上啟動cloudera-scm-server
cp /opt/cloudera-manager/cm-5.10.0/etc/init.d/cloudera-scm-server /etc/init.d/cloudera-scm-server #將cloudera-scm-server復制到開機啟動目錄
chkconfig cloudera-scm-server on #設置cloudera-scm-server開機啟動
vi /etc/init.d/cloudera-scm-server #修改cloudera-scm-server服務啟動位置
#CMF_DEFAULTS=${CMF_DEFAULTS:-/etc/default}改為
CMF_DEFAULTS=/opt/cloudera-manager/cm-5.10.0/etc/default
service cloudera-scm-server start #啟動cloudera-scm-server服務
#同時為了保證在每次服務器重啟的時候都能啟動cloudera-scm-server，應該在開機啟動腳本vi /etc/rc.local中加入命令：
service cloudera-scm-server restart
I） 啟動cloudera-scm-agent所有節點
mkdir /opt/cloudera-manager/cm-5.10.0/run/cloudera-scm-agent #創建cloudera-scm-agent運行目錄
cp /opt/cloudera-manager/cm-5.10.0/etc/init.d/cloudera-scm-agent /etc/init.d/cloudera-scm-agent #將cloudera-scm-agent復制到開機啟動目錄
chkconfig cloudera-scm-agent on #設置cloudera-scm-agent開機啟動
vi /etc/init.d/cloudera-scm-agent #修改cloudera-scm-agent服務啟動位置
#CMF_DEFAULTS=${CMF_DEFAULTS:-/etc/default}改為
CMF_DEFAULTS=/opt/cloudera-manager/cm-5.10.0/etc/default
service cloudera-scm-agent start #啟動cloudera-scm-agent服務
#同時為了保證在每次服務器重啟的時候都能啟動cloudera-scm-agent，應該在開機啟動腳本vi /etc/rc.local中加入命令：
service cloudera-scm-agent restart
Master:
mkdir /var/log/cloudera-scm-server
chown cloudera-scm:cloudera-scm /var/log/cloudera-scm-server
chmod g+w /var/log/cloudera-scm-server
Agent:
mkdir /var/log/cloudera-scm-agent
chown cloudera-scm:cloudera-scm /var/log/cloudera-scm-agent
chmod g+w /var/log/cloudera-scm-agent
7. 搭建本地yum源
A) 安裝軟件
yum install -y yum-utils createrepo #master機器執行
yum install -y httpd # master機器執行 安裝apache2
systemctl start httpd.service #master機器執行 啟動apache服務
systemctl enable httpd.service #master機器執行 設置開機啟動
systemctl start httpd.service #啟動
systemctl stop httpd.service #停止
systemctl restart httpd.service #重啟
systemctl enable httpd.service #開機啟動
systemctl disable httpd.service #開機不啟動
B) 下載CDH rpm包和Parcel包
下載地址：
http://archive.cloudera.com/cm5/repo-as-tarball/5.10.0/cm5.10.0-centos7.tar.gz #CM
http://archive.cloudera.com/cdh5/repo-as-tarball/5.10.0/ #CDH
mkdir -p /var/www/html/cdh/510
tar -zxvf /home/workspace/zipped/cm5.10.0-centos7.tar.gz -C /var/www/html/cdh/510 #解壓到本地www站點地址 rpms
mv /opt/cloudera/parcel-repo/CDH-5.10.0-1.cdh5.10.0.p0.41-el7.parcel /var/www/html/cdh/510
mv /opt/cloudera/parcel-repo/CDH-5.10.0-1.cdh5.10.0.p0.41-el7.parcel.sha /var/www/html/cdh/510
mv /opt/cloudera/parcel-repo/manifest.json /var/www/html/cdh/510
chmod -R 755 /var/www/html/* #授權文件夾權限
C) 本地yum源
yum install yum-utils createrepo #安裝repo util
mv /home/lenmom/cdh5/ /home/lenmom/cdh5/ # /home/lenmom/cdh5/ contains parcels and rpms.
createrepo /var/www/html/cdh/510/rpms # /var/www/html/cdh/510/rpms rpms.
chmod -R 755 /var/www/html/* #授權文件夾權限
vi /etc/yum.repos.d/cdh510.repo 寫入如下內容
[cdh510]
name=repo
baseurl=http://master251/cdh/510/rpms/
enabled=true
gpgcheck=false
分發到所有agent節點:
scp /etc/yum.repos.d/cdh510.repo realtime247:/etc/yum.repos.d/
scp /etc/yum.repos.d/cdh510.repo broker248:/etc/yum.repos.d/
scp /etc/yum.repos.d/cdh510.repo history249:/etc/yum.repos.d/
scp /etc/yum.repos.d/cdh510.repo history250:/etc/yum.repos.d/
保存後，執行以下兩條命令
yum clean all #清除yum緩存
rm -Rf /var/cache/yum/x86_64
yum update #更新yum緩存
yum makecache #所有節點執行
8. 安裝：
查看端口占用：
netstat -anp|grep 7180 #7180是cloudera-server host的端口號，查看某個端口是否被占用
lsof -i:7180 #7180是cloudera-server host的端口號，查看某個端口是否被占用
瀏覽器
http://192.168.1.251:7180/ 開始安裝過程（等待服務器啟動需要一段時間，耐心等待）
遠程pacel全部刪除，只留一個，地址為：
http://192.168.1.251/cdh/510/parcels/
註意：安裝過程中，如果出現“主機運行狀況不良”，是因為節點上次安裝沒有成功，需要刪除cm_guid文件才能再次安裝。其次要註意master分配的資源是否足夠，
有可能是因為master資源不夠，心跳無法及時響應，導致報告主機運行狀況不良。解決辦法如下：
1. rm -f /opt/cloudera-manager/cm-5.10.0/lib/cloudera-scm-agent/cm_guid
service cloudera-scm-agent restart
2. 增加master節點的內存和cpu資源，讓他能夠及時響應心跳檢測服務。
CentOS 7.3 CDH 5.10.0 Druid0.12.4安裝記錄