1. 程式人生 > >oozie(1):oozie安裝步驟

oozie(1):oozie安裝步驟

一、準備工作

1.準備軟體

oozie和ext2.2

2.Hadoop

3.mysql

建資料庫

mysql> create database oozie;

二、安裝和配置

1.上傳解壓oozie安裝包

tar -zxvf oozie-4.1.0-cdh5.7.0.tar.gz -C /opt/modules/

2.修改hadoop裡面的core-site.xml

(1)修改core-site.xml 

#root替換為自己使用者名稱
<property>
	<name>hadoop.proxyuser.root.hosts</name>
	<value>*</value>
</property>
<property>
	<name>hadoop.proxyuser.root.groups</name>
	<value>*</value>
</property>

(2)重啟hadoop叢集服務

sbin/start-dfs.sh
sbin/start-yarn.sh

啟動history
sbin/mr-jobhistory-daemon.sh start historyserver

3.生成libext目錄,並拷貝相關的jar包

(1)oozie安裝目錄下

mkdir libext

(2)oozie安裝目錄同級目錄下解壓oozie-hadooplibs-4.1.0-cdh5.7.0.tar.gz

tar -zxvf oozie-hadooplibs-4.1.0-cdh5.7.0.tar.gz -C /opt/modules/

     解壓縮後,在/opt/modules/oozie-4.1.0-cdh5.7.0目錄下會增加hadooplibs包!

(3)拷貝解壓出來的jar包到{OOZIE_HOME}/libext

cp hadooplibs/hadooplib-2.6.0-cdh5.7.0.oozie-4.1.0-cdh5.7.0/* /opt/modules/oozie-4.1.0-cdh5.7.0/libext/

(4)拷貝ext-2.2.zip到{OOZIE_HOME}/libext

cp /opt/software/ext-2.2.zip libext/

(5)拷貝mysql的驅動包到{OOZIE_HOME}/libext裡面

cp /opt/modules/hive-1.1.0-cdh5.7.0/lib/mysql-connector-java-5.1.27-bin.jar /opt/modules/oozie-4.1.0-cdh5.7.0/libext/

4.修改oozie配置檔案 {OOZIE_HOME}/conf/oozie-site.xml

<property>
	<name>oozie.service.JPAService.jdbc.driver</name>
	<value>com.mysql.jdbc.Driver</value>
</property>
<property>
	<name>oozie.service.JPAService.jdbc.url</name>
	<value>jdbc:mysql://hadoop:3306/oozie</value>
</property>
<property>
	<name>oozie.service.JPAService.jdbc.username</name>
	<value>root</value>
</property>
<property>
	<name>oozie.service.JPAService.jdbc.password</name>
	<value>123456</value>
</property>
** 關聯HDFS
<property>
	<name>oozie.service.HadoopAccessorService.hadoop.configurations</name>
	<value>*=/opt/modules/hadoop-2.6.0-cdh5.7.0/etc/hadoop</value>
</property>

5、依次執行以下命令

(1)在{OOZIE_HOME},生產web工程(war)

bin/oozie-setup.sh prepare-war

(2)如果報錯

unzip: command not found

是因為centOS預設沒有unzip解壓縮命令,所以解決是安裝unzip

yum install -y unzip zip

(3)上傳包到HDFS

bin/oozie-setup.sh sharelib create \
-fs hdfs://hadoop:8020 \
-locallib oozie-sharelib-4.1.0-cdh5.7.0-yarn.tar.gz

(4)生成資料庫

bin/oozie-setup.sh db create -run -sqlfile oozie.sql

結果:

setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"

Validate DB Connection
DONE
Check DB schema does not exist
DONE
Check OOZIE_SYS table does not exist
DONE
Create SQL schema
DONE
Create OOZIE_SYS table
DONE

Oozie DB has been created for Oozie version '4.1.0-cdh5.7.0'

The SQL commands have been written to: oozie.sql

(5)檢測成功生成的資料庫

(a)mysql資料庫中

use oozie;
show tables;
結果
mysql> show tables;
+------------------------+
| Tables_in_oozie        |
+------------------------+
| BUNDLE_ACTIONS         |
| BUNDLE_JOBS            |
| COORD_ACTIONS          |
| COORD_JOBS             |
| OOZIE_SYS              |
| OPENJPA_SEQUENCE_TABLE |
| SLA_EVENTS             |
| SLA_REGISTRATION       |
| SLA_SUMMARY            |
| VALIDATE_CONN          |
| WF_ACTIONS             |
| WF_JOBS                |
+------------------------+
12 rows in set (0.00 sec)

(b)oozie.sql檔案:剛剛建立的oozie資料庫中資料表的所有sql語句

cat /opt/modules/oozie-4.1.0-cdh5.7.0/oozie.sql
結果:
...
CREATE INDEX I_WF_CTNS_PENDING_AGE ON WF_ACTIONS (pending_age);
CREATE INDEX I_WF_CTNS_STATUS ON WF_ACTIONS (status);
CREATE INDEX I_WF_CTNS_WF_ID ON WF_ACTIONS (wf_id);
CREATE INDEX I_WF_JOBS_END_TIME ON WF_JOBS (end_time);
CREATE INDEX I_WF_JOBS_EXTERNAL_ID ON WF_JOBS (external_id);
CREATE INDEX I_WF_JOBS_LAST_MODIFIED_TIME ON WF_JOBS (last_modified_time);
CREATE INDEX I_WF_JOBS_PARENT_ID ON WF_JOBS (parent_id);
CREATE INDEX I_WF_JOBS_STATUS ON WF_JOBS (status);

create table OOZIE_SYS (name varchar(100), data varchar(100))
insert into OOZIE_SYS (name, data) values ('db.version', '3')
insert into OOZIE_SYS (name, data) values ('oozie.version', '4.1.0-cdh5.7.0')

6.啟動服務

bin/oozied.sh start

7.測試前端頁面oozie

http://hadoop:11000/oozie/

  

8.oozie停止指令碼

bin/oozie-stop.sh