1 Cloudera安裝配置

1.1 安裝前注意事項

前期在fengkongj上操作，再將配置好的安裝包拷給fengkongb,fengkongc,fengkongd,fengkonge, fengkongf, fengkongg, fengkongh, fengkongi,
基本思路是：先在後臺配置好cm(cloudera management)、parcels，然後在頁面上操作cm，安裝zookeeper、yarn、spark、hbase等。
參考官方文件：http://www.cloudera.com/documentation/enterprise/latest/topics/cm_ig_install_path_c.html
 

1.2 安裝配置cm
1） 在http://archive.cloudera.com/cm5/cm/5/下載：cloudera-manager-el6-cm5.7.0_x86_64.tar.gz
2） 建立/opt/cloudera-manager目錄,上傳並將tar包解壓到該目錄下
3） 建立使用者

 useradd --system --home=/opt/cloudera-manager/cm-5.7.0/run/cloudera-scm-server --no-create-home --comment "Cloudera SCM User" cloudera-scm

4） 修改擁有者使用者

 chown -R
 
 cloudera-scm:cloudera-scm /opt/cloudera-manager

2） 按照官網的檔案格式下載原始檔, 上傳到linux.
目錄為/opt/cloudera/parcel-repo,資料夾擁有者要改為cloudera-scm：

 chown -R cloudera-scm:cloudera-scm /opt/cloudera

3） 並在/var/www/html上配置連結：

 ln -s /opt/cloudera/parcel-repo /var/www/html/CDH5.7.0

開啟瀏覽器，輸入192.168.89.10/CDH5.7.3，檢視效果：

4） 修改manifest.json,刪除剩下以下parcelName的字元簇:
…
“parcelName”: “CDH-5.4.0-1.cdh5.4.0.p0.27-el7.parcel”,
….
5） 修改sha1為sha:

mv CDH-5.5.2-1.cdh5.5.2.p0.4-el7.parcel.sha1 CDH-5.5.2-1.cdh5.5.2.p0.4-el7.parcel.sha

1.4 配置cdh庫和使用者
注：可選
create database amon DEFAULT CHARACTER SETutf8;
grant all on amon.* TO ’ amon’@’%’ IDENTIFIEDBY ‘amon_123’;
create database rman DEFAULT CHARACTER SETutf8;
grant all on rman.* TO ‘rman’@’%’ IDENTIFIEDBY ‘rman_123’;
create database metastore DEFAULT CHARACTERSET utf8;
grant all on metastore.* TO ‘hive’@’%’IDENTIFIED BY ‘hive_123’;
create database sentry DEFAULT CHARACTER SETutf8;
grant all on sentry.* TO ‘sentry’@’%’IDENTIFIED BY ‘sentry_123’;
create database nav DEFAULT CHARACTER SETutf8;
grant all on nav.* TO ‘nav’@’%’ IDENTIFIED BY’nav_123’;
create database navms DEFAULT CHARACTER SETutf8;
grant all on navms.* TO ‘navms’@’%’ IDENTIFIEDBY ‘navms_123’;
create database hue DEFAULT CHARACTER SETutf8;
grant all on hue.* TO ‘hue’@’%’ IDENTIFIED BY’hue_123’;
create database hive DEFAULT CHARACTER SETutf8;
grant all on hive.* TO ‘hive’@’%’ IDENTIFIEDBY ‘hive_123’;
create database oozie DEFAULT CHARACTER SETutf8;
grant all on oozie.* TO ‘oozie’@’%’ IDENTIFIEDBY ‘oozie_123’;
create database cdhreport DEFAULT CHARACTERSET utf8;
grant all on cdhreport.* TO ‘report’@’%’IDENTIFIED BY ‘report_123’;
create database cdhmonitor DEFAULT CHARACTERSET utf8;
grant all on cdhmonitor.* TO ‘monitor’@’%’IDENTIFIED BY ‘monitor_123’;

mysql> flush privileges;

1.5 mysql客戶端驅動

1） 下載http://dev.mysql.com/downloads/connector/j/5.1.html
2） 上傳mysql-connector-java-5.1.38.tar.gz
3） 解壓後將mysql-connector-java-5.1.38-bin.jar放入/opt/cloudera-manager/cm-5.7.0/share/cmf/lib
4） 也放一份在下面目錄

cp mysql-connector-java-5.1.38-bin.jar /usr/share/java/mysql-connector-java.jar

1.6 建立scm資料庫

vi /opt/cloudera-manager/cm-5.7.0/share/cmf/schema/scm_prepare_database.sh

JAVA_HOME=/opt/jdk1.8.0_60

 /opt/cloudera-manager/cm-5.7.0/share/cmf/schema/scm_prepare_database.sh mysql -h fengkonga -P 3306 -uroot -p123456 --scm-host fengkonga scm scm scm_123

注：輸出一下內容，說明成功：
…
Executing: /opt/jdk1.8.0_60/bin/java -cp/usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/opt/cloudera-manager/cm-5.7.3/share/cmf/schema/.
./lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /opt/cloudera-manager/cm-5.7.0/etc/cloudera-scm-server/db.propertiescom.cloudera.cmf.db.[main] DbCommandExecutor INFOSuccessfullyconnected to database.
All done,your SCM database is configured correctly!

1.7 Cloudera agent安裝

fengkongb,fengkongc,fengkongd, fengkonge, fengkongf, fengkongg, fengkongh, fengkongi:   

vi /opt/cloudera-manager/cm-5.7.0/etc/cloudera-scm-agent/config.ini

…
server_host=192.168.89.10
…

fengkongj:

sudo scp -r cloudera fengkongb:/opt
sudo scp -rcloudera-manager fengkongb:/opt
sudo scp -r cloudera fengkongc:/opt
sudo scp -rcloudera-manager fengkongc:/opt
sudo scp -r cloudera fengkongd:/opt
sudo scp -rcloudera-manager fengkongd:/opt
sudo scp -r cloudera fengkonge:/opt
sudo scp -rcloudera-manager fengkonge:/opt
sudo scp -r cloudera fengkongf:/opt
sudo scp -rcloudera-manager fengkongf:/opt
sudo scp -r cloudera fengkongg:/opt
sudo scp -rcloudera-manager fengkongg:/opt
sudo scp -r cloudera fengkongh:/opt
sudo scp -rcloudera-manager fengkongh:/opt
sudo scp -r cloudera fengkongi:/opt
sudo scp -r cloudera-managerfengkongi:/opt

fengkongb,fengkongc,fengkongd, fengkonge, fengkongf, fengkongg, fengkongh, fengkongi:

useradd --system--home=/opt/cloudera-manager/cm-5.7.0/run/cloudera-scm-server --no-create-home --comment "Cloudera SCM User" cloudera-scm
chown -R cloudera-scm:cloudera-scm /opt/cloudera-manager
chown -R cloudera-scm:cloudera-scm /opt/cloudera

1.8 Cloudera server啟動

fengkongj：

/opt/cloudera-manager/cm-5.7.0/etc/init.d/cloudera-scm-server start
tail -f /opt/cloudera-manager/cm-5.7.0/log/cloudera-scm-server/cloudera-scm-server.log

注：server端在初始化，看日誌知道出現以下字樣表明初始化完成：
。。。
2016-03-23 11:17:50,333 INFOWebServerImpl:org.mortbay.log: jetty-6.1.26.cloudera.4
2016-03-23 11:17:50,334 INFOWebServerImpl:org.mortbay.log: Started [email protected]:7180
2016-03-23 11:17:50,334 INFOWebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server.

1.9 Cloudera agent啟動

/opt/cloudera-manager/cm-5.7.0/etc/init.d/cloudera-scm-agent start
 tail -f /opt/cloudera-manager/cm-5.7.0/log/cloudera-scm-agent/cloudera-scm-agent.log

注：如果啟動失敗，檢查/opt/cloudera-manager/cm-5.7.0/run目錄下是否有cloudera-scm-agent資料夾，沒有則新建：

mkdir cloudera-scm-agent
chown cloudera-scm:cloudera-scm cloudera-scm-agent/

1.10 Cdh倚賴包安裝

yum -y install bind-utils psmisc libxsltzlib sqlite cyrus-sasl-plain cyrus-sasl-gssapi fuse portmap fuse-libsredhat-lsb

1.11 訪問Cloudera管理臺WEB
瀏覽器輸入：
192.168.89.10:7180
預設：
使用者：admin
密碼：admin
出現登入介面則表示cm安裝成功。