CDH6.0.1詳細安裝步驟及問題解決
CDH6.0.1詳細安裝步驟及常見問題解決
Cloudera Manager能夠大大減輕配置hadoop叢集的工作量,安裝好以後能夠快速熟悉Hadoop生態系統常用的一些元件,如HBase, Hive, Spark, Oozie, Impala, Hue等等。筆者在安裝Cloudera Manager過程中遇到了很多麻煩,最終在折騰了一個星期以後終於配置成功。現在將配置步驟和過程中遇到的一些問題的解決辦法拿出來與大家一起分享。
系統環境
• 作業系統:3臺 CentOS 7 x64
• Cloudera Manager:6.0.1
• CDH: 6.0.1
VM IP Adress | Centos_HostName | Disk Space | Physical Memory | Role |
---|---|---|---|---|
192.168.1.101 | hadoop01 | 40GB | 16GB | Cloudera Manager Server + Cloudera Manager Agent |
192.168.1.102 | hadoop02 | 40GB | 4GB | Cloudera Manager Agent |
192.168.1.106 | hadoop06 | 40GB | 8GB | Cloudera Manager Agent |
- 最初是想搭建一個5個節點的Hadoop叢集的,後來發現Cloudera Manager佔用記憶體特別多,無奈一臺PC只能安裝一個CentOS;
- 安裝CentOS系統時磁碟空間最好選成40GB,預設的20GB不夠用。以後擴充套件很麻煩;
- 預設2GB的swap空間不夠用,需要擴充套件成4GB;
一、配置網路環境
安裝完CentOS後第一步就是配置網路。
-
編輯/etc/sysconfig/network-scripts/ifcfg-eth0配置檔案,修改其中以下粗體標示的專案:
vi /etc/sysconfig/network-scripts/ifcfg-eth0
TYPE=Ethernet
BOOTPROTO=static
DEFROUTE=yes
PEERDNS=yes
PEERROUTES=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
IPV6_FAILURE_FATAL=no
NAME=eth0
UUID=3719cf03-5d05-454b-903f-e1e26ad34fd6
DEVICE=eth0
ONBOOT=yes
IPADDR=192.168.1.101
NETWORK=192.168.1.1
NETSTAT=255.255.255.0
GATEWAY=192.168.1.1
DNS1=222.186.160.70
DNS2=114.114.114.114
其它主機的IP地址做相應的更改。 -
編輯/etc/default/grub檔案,在GRUB_CMD_LINE_LINUX="“項中,插入"net.ifnames=0 biosdevname=0”(與原有項之間用空格隔開),儲存退出。
-
呼叫命令grub2-mkconfig -o /boot/grub2/grub.cfg更新配置,然後直接reboot重啟。
這樣之後再用ifconfig -a命令檢視時,就有eth0網絡卡存在了。 -
更改hostname
hostnamectl set-hostname hadoop01 -
修改hosts檔案
vi /etc/hosts192.168.1.101 hadoop01
192.168.1.102 hadoop02
192.168.1.106 hadoop06 -
關閉防火牆
sed -i ‘s/SELINUX=enforcing/SELINUX=disabled/g’ /etc/selinux/config
setenforce 0
systemctl stop firewalld.service
systemctl disable firewalld.service -
配置免密登入:
在每臺主機上生成rsa公鑰和私鑰
ssh-keygen -t rsa
cd .ssh
cat id_rsa.pub >> authorized_keys
登入其他主機,將其他主機的公鑰檔案內容都拷貝到hadoop01主機上的authorized_keys檔案中,命令如下:
[[email protected] .ssh]# ssh-copy-id -i hadoop01
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed – if you are prompted now it is to install the new keys
[email protected]’s password:
Number of key(s) added: 1
Now try logging into the machine, with: “ssh ‘hadoop01’”
and check to make sure that only the key(s) you wanted were added.
在每臺主機上重複以上步驟,使任意兩臺主機間都能相互連通。
至此,免密碼登入已經設定完成。
-
更改系統時間:
進入home目錄,修改.bash_profile
#setup time region
TZ=‘Asia/Shanghai’
export TZ
或者: tzselect將當前時間和日期寫入BIOS,避免重啟後失效
hwclock –w另外還要進入CentOS的選單System Tools–> Settings --> Date & Time中,將時區修改為Asia/Shanghai。
-
配置NTP同步各臺伺服器的時間
9.1主節點配置在配置之前,先使用ntpdate手動同步一下時間,免得本機與對時中心時間差距太大,使得ntpd不能正常同步。這裡選用65.55.56.206作為對時中心,ntpdate -u ntp1.aliyun.com
vi /etc/ntp.conf
#Hosts on local network are less restricted.
取消下面一行的註釋,並將其中IP和掩碼修改為真實環境IP和掩碼,此行配置為允許ntp客戶端連線的配置
restrict 192.168.1.0 mask 255.255.255.0 nomodify notrap
找到server 0.centos.pool.ntp.org iburst,並將所有server配置進行註釋
新增下面兩行內容
server 127.127.1.0 # local clock
fudge 127.127.1.0 stratum 10啟動NTP服務
設定後,重啟ntpd服務,用ntpstat來檢查效果【大概幾秒鐘就可以看到效果】
#systemctl restart ntpd
#ntpstatvi /etc/sysconfig/ntpdate
#Set to ‘yes’ to sync hw clock after successful ntpdate
SYNC_HWCLOCK=yes9.2 Agent節點配置
修改/etc/ntp.conf
對該檔案進行以下內容的修改:
1、註釋所有restrict和server配置
2、新增下面一行,需要修改以下的IP為NTP伺服器的IP(在本案例中就是server主機的IP)
server 192.168.1.101將所有主機的ntp服務設定為開機自動啟動
chkconfig ntpd on
systemctl enable ntpd.service
二、 配置163 yum源
-
下載repo檔案
cd /etc/yum.repos.d/
wget http://mirrors.163.com/.help/CentOS7-Base-163.repo -
備份並替換系統的repo檔案
mv CentOS-Base.repo CentOS-Base.repo.bak
cp CentOS7-Base-163.repo CentOS-Base.repo -
執行yum源更新命令
yum clean all
yum makecache
yum updateyum install chkconfig python bind-utils psmisc libxslt zlib sqlite cyrus-sasl-plain cyrus-sasl-gssapi fuse fuse-libs redhat-lsb –y
yum install httpd mod_ssl openssl-devel python-psycopg2 –y
yum install MySQL-python
yum install perl
yum install autoconf
三、安裝MySQL
-
檢視並解除安裝CentOS系統自帶的MySQL和Mariadb
rpm -qa | grep MySQL
rpm -qa | grep mariadb
rpm -e --nodeps MySQL-python-1.2.5-1.el7.x86_64
rpm -e --nodeps mariadb-libs-5.5.60-1.el7_5.x86_64 -
下載並上傳以下rpm包到主節點
MySQL-client-5.6.42-1.el7.x86_64.rpm
MySQL-devel-5.6.42-1.el7.x86_64.rpm
MySQL-server-5.6.42-1.el7.x86_64.rpm -
在主節點上執行以下命令安裝MySQL
rpm -ivh MySQL* -
安裝完成後啟動MySQL:
service mysql start
systemctl start mysql.service -
修改MySQL的配置檔案
vi /etc/my.cnf[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
transaction-isolation = READ-COMMITTED
#Disabling symbolic-links is recommended to prevent assorted security risks;
#to do so, uncomment this line:
symbolic-links = 0
key_buffer_size = 32M
max_allowed_packet = 32M
thread_stack = 256K
thread_cache_size = 64
query_cache_limit = 8M
query_cache_size = 64M
query_cache_type = 1
max_connections = 550
#expire_logs_days = 10
#max_binlog_size = 100M
#log_bin should be on a disk with enough free space.
#Replace ‘/var/lib/mysql/mysql_binary_log’ with an appropriate path for your
#system and chown the specified folder to the mysql user.
log_bin=/var/lib/mysql/mysql_binary_log
#In later versions of MySQL, if you enable the binary log and do not set
#a server_id, MySQL will not start. The server_id must be unique within
#the replicating group.
server_id=1
binlog_format = mixed
read_buffer_size = 2M
read_rnd_buffer_size = 16M
sort_buffer_size = 8M
join_buffer_size = 8M
#InnoDB settings
innodb_file_per_table = 1
innodb_flush_log_at_trx_commit = 2
innodb_log_buffer_size = 64M
innodb_buffer_pool_size = 4G
innodb_thread_concurrency = 8
innodb_flush_method = O_DIRECT
innodb_log_file_size = 512M
[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
sql_mode=STRICT_ALL_TABLES -
獲取系統初始密碼:
[[email protected] ~]# cat /root/.mysql_secret
#The random password set for the root user at Tue Dec 18 20:37:59 2018 (local time): pP4xR06U9nPyWrhA登入MySQL:
[[email protected] ~]# mysql -uroot -ppP4xR06U9nPyWrhA
Warning: Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.6.42SET PASSWORD FOR ‘root’@‘localhost’ = PASSWORD(‘xxxxxxx’)
grant all on . to [email protected]"%" identified by " xxxxxxx "
設定MySQL開機自啟動
systemctl enable mysql.service
初始化MySQL
/usr/bin/mysql_secure_installation
下載mysql-connector-java-5.1.46.tar.gz並解壓
在解壓目錄下找到mysql-connector-java.jar, 複製到所有主機的/usr/share/java目錄下cd mysql-connector-java-5.1.46
cp mysql-connector-java-5.1.46.jar /usr/share/java/mysql-connector-java.jar[[email protected] usr]# ls -l /usr/share/java/mysql-connector-java.jar
-rw-r–r-- 1 root root 1004838 Dec 19 20:45 /usr/share/java/mysql-connector-java.jar下載Oracle JDK包oracle-j2sdk1.8-1.8.0+update141-1.x86_64.rpm上傳到所有主機並安裝
下載地址:https://archive.cloudera.com/cm6/6.0.1/redhat7/yum/RPMS/x86_64/rpm -ivh oracle-j2sdk1.8-1.8.0+update141-1.x86_64.rpm
-
Creating Databases for Cloudera Software
需要建的庫有
scm、amon、rman、hue、metastore、sentry、nav、navms、oozie
CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON scm.* TO ‘scm’@’%’ IDENTIFIED BY ‘[email protected]’;
CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON rman.* TO ‘rman’@’%’ IDENTIFIED BY ‘[email protected]’;
CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON hue.* TO ‘hue’@’%’ IDENTIFIED BY ‘[email protected]’;
CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON metastore.* TO ‘metastore’@’%’ IDENTIFIED BY ‘[email protected]’;
CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON sentry.* TO ‘sentry’@’%’ IDENTIFIED BY ‘[email protected]’;
CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON nav.* TO ‘nav’@’%’ IDENTIFIED BY ‘[email protected]’;
CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON navms.* TO ‘navms’@’%’ IDENTIFIED BY ‘[email protected]’;
CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON oozie.* TO ‘oozie’@’%’ IDENTIFIED BY ‘[email protected]’;
-
初始化cloudera manager資料庫
/opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm scm
四、下載並安裝Cloudera Manager
-
下載地址: https://archive.cloudera.com/cm6/6.0.1/redhat7/yum/RPMS/x86_64/
1.1. cloudera-manager-daemons-6.0.1-610811.el7.x86_64.rpm
1.2. cloudera-manager-server-6.0.1-610811.el7.x86_64.rpm
1.3. cloudera-manager-agent-6.0.1-610811.el7.x86_64.rpm在主節點hadoop01伺服器上安裝1,2,3;
在Agent伺服器上安裝1,3; -
在主節點hadoop01上執行:
systemctl start cloudera-scm-server檢視日誌:
tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log
顯示如下表示服務已經正常起來
INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server.
-
在所有Agent節點hadoop01, hadoop02, hadoop06上執行:
配置Agent的Server指向
vi /etc/cloudera-scm-agent/config.ini[General]
#Hostname of the CM server.
server_host=hadoop01[Security]
#Use TLS and certificate validation when connecting to the CM server.
use_tls=1啟動cloudera-scm-agent服務:
systemctl status cloudera-scm-agent
systemctl start cloudera-scm-agentAgent Log檔案位置:/var/log/cloudera-scm-agent
cloudera-scm-agent.logtail -f /var/log/cloudera-scm-agent/cloudera-scm-agent.log
五、配置CDH
Cloudera Manager Server啟動起來以後就可以登入Web介面進行CDH的安裝配置工作了。
在瀏覽器中輸入 http://192.168.1.101:7180
就可以看到如下的登入介面
使用者名稱和密碼預設都為admin
點選登入按鈕後就會看到選擇TLS認證方式的介面,在此我們採用第1種方式,讓cloudera manager自動配置TLS認證。
在主節點hadoop01機器上執行:
JAVA_HOME=/usr/java/jdk1.8.0_181-amd64 /opt/cloudera/cm-agent/bin/certmanager --location /opt/cloudera/CMCA setup --configure-services
重啟cloudera manager server,然後重新登入,就會看到如下介面
點選繼續,就會看到選擇主機的介面
輸入要安裝CDH的主機IP地址,點選搜尋
選中搜索到的主機,點選繼續
在此我們不做任何改動,直接點選繼續
我們前面已經手動安裝了Oracle-j2sdk,所以這裡不用再做勾選,直接點選繼續
輸入root使用者的密碼,點選繼續
執行一段時間以後又遇到了如下的錯誤:主機執行狀況不良。
在網上搜索到如下解決辦法,應用後問題得到解決:
Cloudera 安裝Parcel提示主機執行狀況不良
解決方法
刪除agent目錄下面的cm_guid檔案,並重啟失敗節點的agent服務。
[[email protected] ~]# cd /var/lib/cloudera-scm-agent/
[[email protected] cloudera-scm-agent]# ll
total 32
-rw------- 1 root root 2 Dec 28 14:27 active_parcels.json
drwxr-xr-x 2 cloudera-scm cloudera-scm 4096 Dec 28 23:01 agent-cert
-rw-r–r-- 1 root root 36 Dec 28 14:25 cm_guid
-rw------- 1 root root 13813 Dec 29 07:10 response.avro
-rw-r–r-- 1 root root 36 Dec 28 14:25 uuid
[[email protected] cloudera-scm-agent]# rm -rf cm_guid
[[email protected] cloudera-scm-agent]# systemctl restart cloudera-scm-agent
在兩臺報錯的主機上將cm_guid刪除後,返回安裝介面,發現已經自動進行到如下介面開始繼續安裝
等待一段時間後CDH parcel已經在所有主機上啟用
點選繼續,進入檢查主機正確性的介面
出現3個黃色警告,我們一 一解決。
- Cloudera 建議將 /proc/sys/vm/swappiness 設定為最大值 10。當前設定為 30。
Cloudera 建議將 /proc/sys/vm/swappiness 設定為最大值 10。當前設定為 30。使用 sysctl 命令在執行時更改該設定並編輯 /etc/sysctl.conf,以在重啟後儲存該設定。您可以繼續進行安裝,但 Cloudera Manager 可能會報告您的主機由於交換而執行狀況不良。以下主機將受到影響:
檢視詳細資訊
hadoop[01-02, 06]
解決辦法:
臨時調整: sysctl vm.swappiness=10
永久調整: vi /etc/sysctl.conf
增加下面一行
vm.swappiness=10
- 已啟用透明大頁面壓縮,可能會導致重大效能問題。
已啟用透明大頁面壓縮,可能會導致重大效能問題。請執行“echo never > /sys/kernel/mm/transparent_hugepage/defrag”和“echo never > /sys/kernel/mm/transparent_hugepage/enabled”以禁用此設定,然後將同一命令新增到 /etc/rc.local 等初始化指令碼中,以便在系統重啟時予以設定。以下主機將受到影響:
檢視詳細資訊
hadoop[01-02, 06]
解決辦法:
echo never > /sys/kernel/mm/transparent_hugepage/defrag
echo never > /sys/kernel/mm/transparent_hugepage/enabled
修改/etc/rc.local將上面兩行新增進去
vi /etc/rc.local
- Psycopg2版本太低的問題
安裝pip
yum -y install epel-release
yum -y install python-pip
pip install --upgrade pip
yum install postgresql-devel
安裝psycopg2
下載psycopg2-2.7.6.1.tar.gz並解壓
[[email protected] psycopg2-2.7.6.1]# python setup.py build
[[email protected] psycopg2-2.7.6.1]# python setup.py install
重新執行檢查主機,所有專案檢查通過
點選繼續
我們選擇所有服務,點選繼續
點選按主機檢視按鈕,可以清楚的看到每臺主機上安裝的元件
我們採用預設設定,直接點選繼續
輸入先前建立的資料庫的名稱,使用者名稱和密碼,測試連線,全部通過,點選繼續
直接點選繼續開始安裝
等待一段時間後安裝完成
點選繼續,進入Summary介面
至此,CDH安裝順利完成,Cheers!
接下來讓我們一起看看安裝完成後的主介面