cdh版本的hue安裝配置部署以及集成hadoop hbase hive mysql等權威指南
hue下載地址：https://github.com/cloudera/hue
hue學習文檔地址：http://archive.cloudera.com/cdh5/cdh/5/hue-3.7.0-cdh5.3.6/manual.html
我目前使用的是hue-3.7.0-cdh5.3.6
hue（HUE=Hadoop User Experience）
Hue是一個開源的Apache Hadoop UI系統，由Cloudera Desktop演化而來，最後Cloudera公司將其貢獻給Apache基金會的Hadoop社區，它是基於Python Web框架Django實現的。
通過使用Hue我們可以在瀏覽器端的Web控制臺上與Hadoop集群進行交互來分析處理數據，例如操作HDFS上的數據，運行MapReduce Job，執行Hive的SQL語句，瀏覽HBase數據庫等等。
hue特點：
能夠支持各個版本的hadoop
hue默認數據庫：sql lite
文件瀏覽器：對數據進行增刪改查
hue下載src包進行一次編譯，二次編譯，在這用的是已經一次編譯
hue部署：
1、下載依賴包：yum源安裝
sudo yum -y install ant asciidoc cyrus-sasl-devel cyrus-sasl-gssapi gcc gcc-c++ krb5-devel libtidy libxml2-devel libxslt-devel mvn mysql mysql-devel openldap-devel python-devel sqlite-devel openssl-devel
sudo yum install ant asciidoc cyrus-sasl-devel cyrus-sasl-gssapi cyrus-sasl-plain gcc gcc-c++ krb5-devel libffi-devel libxml2-devel libxslt-devel make mysql mysql-devel openldap-devel python-devel sqlite-devel gmp-devel
2、解壓hue tar包
tar -zxvf hue-3.7.0-cdh5.3.6.tar.gz -C /指定的目錄
3、二次編譯
進入hue目錄：執行make apps 會出現個build目錄
錯誤：（centos 7會有）
error: static declaration of ‘X509_REVOKED_dup’ follows non-static declaration
static X509_REVOKED * X509_REVOKED_dup(X509_REVOKED *orig) {
^
In file included from /usr/include/openssl/ssl.h:156:0,
from OpenSSL/crypto/x509.h:17,
from OpenSSL/crypto/crypto.h:30,
from OpenSSL/crypto/crl.c:3:
/usr/include/openssl/x509.h:751:15: note: previous declaration of ‘X509_REVOKED_dup’ was here
X509_REVOKED *X509_REVOKED_dup(X509_REVOKED *rev);
^
error: command ‘gcc‘ failed with exit status 1
給下面兩個刪掉：/usr/include/openssl/x509.h -》751、752行
X509_REVOKED *X509_REVOKED_dup(X509_REVOKED *rev);
X509_REQ *X509_REQ_dup(X509_REQ *req);
##必須刪掉，註釋不行
4、進入到hue-3.7.0-cdh5.3.6/desktop/conf
配置hue.ini文件:
secret_key=jFE93j;2[290-eiw.KEiwN2s3[‘d;/.q[eIW^y#e=+Iei*@Mn<qW5o
http_host=hadoop01.xningge.com
http_port=8888
time_zone=Asia/Shanghai
5、啟動hue
兩種方式
1-->cd build/env/bin---》./supervisor
2-->build/env/bin/supervisor
6、瀏覽器訪問hue
主機名+8888
創建用戶名和密碼
hue和hadoop的組件配置
1、hdfs的配置
在hdfs-site.xml中配置
<property>
<name>dfs.webhdfs.enabled</name>
<value>true</value>
</property>
在core-site.xml中配置
<property>
<name>hadoop.proxyuser.hue.hosts</name>
<value>*</value>
</property>
<property>
<name>hadoop.proxyuser.hue.groups</name>
<value>*</value>
</property>
2、重新啟動hdfs進程
sbin/start-yarn.sh
3、hue配置
[[hdfs_clusters]]
# HA support by using HttpFs
[[[default]]]
# Enter the filesystem uri
fs_defaultfs=hdfs://hadoop01.xningge.com:8020
# NameNode logical name.
## logical_name=
# Use WebHdfs/HttpFs as the communication mechanism.
# Domain should be the NameNode or HttpFs host.
# Default port is 14000 for HttpFs.
webhdfs_url=http://hadoop01.xningge.com:50070/webhdfs/v1
# This is the home of your Hadoop HDFS installation
hadoop_hdfs_home=/opt/modules/cdh/hadoop-2.5.0-cdh5.3.6
# Use this as the HDFS Hadoop launcher script
hadoop_bin=/opt/modules/cdh/hadoop-2.5.0-cdh5.3.6/bin
# Change this if your HDFS cluster is Kerberos-secured
## security_enabled=false
# Default umask for file and directory creation, specified in an octal value.
## umask=022
# Directory of the Hadoop configuration
hadoop_conf_dir=/opt/modules/cdh/hadoop-2.5.0-cdh5.3.6/etc/hadoop
[[yarn_clusters]]
[[[default]]]
# Enter the host on which you are running the ResourceManager
resourcemanager_host=hadoop01.xningge.com
# The port where the ResourceManager IPC listens on
resourcemanager_port=8032
# Whether to submit jobs to this cluster
submit_to=True
# Resource Manager logical name (required for HA)
## logical_name=
# Change this if your YARN cluster is Kerberos-secured
## security_enabled=false
# URL of the ResourceManager API
resourcemanager_api_url=http://hadoop01.xningge.com:8088
# URL of the ProxyServer API
proxy_api_url=http://hadoop01.xningge.com:8088
# URL of the HistoryServer API
history_server_api_url=http://hadoop01.xningge.com:19888
eg:此配置都是偽分布式模式
4、啟動hue服務
build/env/bin/supervisor
hue與hive配置
1、hive配置
在hive-site.xml配置
<property>
<name>hive.server2.thrift.bind.host</name>
<value>hadoop01.xningge.com</value>
</property>
<property>
<name>hive.metastore.uris</name>
<value>hadoop01.xningge.com:9083</value>
</property>
2、啟動hive服務
bin/hiveserver2 &
bin/hive --service metastore &
3 、hue配置
修改hue.ini文件
[beeswax]
# Host where HiveServer2 is running.
# If Kerberos security is enabled, use fully-qualified domain name (FQDN).
hive_server_host=hadoop01.xningge.com
# Port where HiveServer2 Thrift server runs on.
hive_server_port=10000
# Hive configuration directory, where hive-site.xml is located
hive_conf_dir=/opt/modules/cdh/hive-0.13.1-cdh5.3.6/conf
# Timeout in seconds for thrift calls to Hive service
server_conn_timeout=120
# Choose whether Hue uses the GetLog() thrift call to retrieve Hive logs.
# If false, Hue will use the FetchResults() thrift call instead.
## use_get_log_api=true
# Set a LIMIT clause when browsing a partitioned table.
# A positive value will be set as the LIMIT. If 0 or negative, do not set any limit.
## browse_partitioned_table_limit=250
# A limit to the number of rows that can be downloaded from a query.
# A value of -1 means there will be no limit.
# A maximum of 65,000 is applied to XLS downloads.
## download_row_limit=1000000
# Hue will try to close the Hive query when the user leaves the editor page.
# This will free all the query resources in HiveServer2, but also make its results inaccessible.
## close_queries=false
# Thrift version to use when communicating with HiveServer2
## thrift_version=5
hue與關系型數據庫配置
[librdbms]
# The RDBMS app can have any number of databases configured in the databases
# section. A database is known by its section name
# (IE sqlite, mysql, psql, and oracle in the list below).
[[databases]]
# sqlite configuration.
[[[sqlite]]] //註意這裏一定要取消註釋
# Name to show in the UI.
nice_name=SQLite
# For SQLite, name defines the path to the database.
name=/opt/modules/hue-3.7.0-cdh5.3.6/desktop/desktop.db
# Database backend to use.
engine=sqlite
# Database options to send to the server when connecting.
# https://docs.djangoproject.com/en/1.4/ref/databases/
## options={}
# mysql, oracle, or postgresql configuration.
##註意：這裏的數據不能改動，默認是hue的數據庫
[[[mysql]]] //註意這裏一定要取消註釋
# Name to show in the UI.
nice_name="My SQL DB"
# For MySQL and PostgreSQL, name is the name of the database.
# For Oracle, Name is instance of the Oracle server. For express edition
# this is ‘xe‘ by default.
name=sqoop//這個name是數據庫表名
# Database backend to use. This can be:
# 1. mysql
# 2. postgresql
# 3. oracle
engine=mysql
# IP or hostname of the database to connect to.
host=hadoop01.xningge.com
# Port the database server is listening to. Defaults are:
# 1. MySQL: 3306
# 2. PostgreSQL: 5432
# 3. Oracle Express Edition: 1521
port=3306
# Username to authenticate with when connecting to the database.
user=xningge
# Password matching the username to authenticate with when
# connecting to the database.
password=???
# Database options to send to the server when connecting.
# https://docs.djangoproject.com/en/1.4/ref/databases/
## options={}
hue與zookeeper配置
只需修改hue.ini文件
host_ports=hadoop01.xningge.com:2181
啟動zookeeper：
hue與oozie的配置
修改：hue.ini文件
[liboozie]
oozie_url=http://hadoop01.xningge.com:11000/oozie
如果沒有出來的：
修改：oozie-site.xml
<property>
<name>oozie.service.WorkflowAppService.system.libpath</name>
<value>/user/oozie/share/lib</value>
</property>
到oozie目錄下重新創建sharelib庫：
bin/oozie-setup.sh sharelib create -fs hdfs://hadoop01.xningge.com:8020 -locallib oozie-sharelib-4.0.0-cdh5.3.6-yarn.tar.gz
啟動oozie：bin/oozied.sh start
hue與hbase的配置
修改hue.ini文件:
hbase_clusters=(Cluster|hadoop01.xningge.com:9090)
hbase_conf_dir=/opt/cdh_5.3.6/hbase-0.98.6-cdh5.3.6/conf
修改hbase-site.xml,添加以下配置：
<property>
<name>hbase.regionserver.thrift.http</name>
<value>true</value>
</property>
<property>
<name>hbase.thrift.support.proxyuser</name>
<value>true</value>
</property>
啟動hbase：
bin/start-hbase.sh
bin/hbase-daemon.sh start thrift
##hbase完全分布式
hbase_clusters=(Cluster1|hostname:9090,Cluster2|hostname:9090,Cluster3|hostname:9090)
cdh版本的hue安裝配置部署以及集成hadoop hbase hive mysql等權威指南