大數據集群安裝步驟
阿新 • • 發佈：2018-05-30
Hadoop集群安裝環境
服務器配置：
CPU型號：Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz
CPU核素：16
內存：64GB
操作系統
版本：CentOS Linux release 7.5.1804 (Core)
主機列表：
192.168.0.101 node1
192.168.0.102 node2
192.168.0.103 node3
192.168.0.104 node4
192.168.0.105 node4
軟件路徑：/data/tools/
JAVA_HOME路徑：/opt/java # java為軟鏈接
Hadoop集群路徑：/data/bigdata/
軟件版本：
| 組件名 | 安裝包 | 說明 |
|---|---|---|
| JDK | jdk-8u162-linux-x64.tar.gz | 基礎環境 |
| zookeeper | zookeeper-3.4.12.tar.gz | Text |
| Hadoop | hadoop-2.7.6.tar.gz | Text |
| spark | spark-2.1.2-bin-hadoop2.7.tgz | Text |
| hbase | hbase-1.2.6-bin.tar.gz | Text |
| hive | apache-hive-2.3.3-bin.tar.gz | Text |
| kafka | kafka_2.11-1.1.0.tgz | Text |
| hue | hue-3.12.0.tgz | Text |
| flume | apache-flume-1.8.0-bin.tar.gz | Text |
常用命令
1、查看系統基本配置：
[root@node1 ~]# uname -a Linux node1 3.10.0-123.9.3.el7.x86_64 #1 SMP Thu Nov 6 15:06:03 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux [root@node1 ~]# cat /etc/redhat-release CentOS Linux release 7.5.1804 (Core) [root@node1 ~]# free -m total used free shared buffers cached Mem: 64267 2111 62156 16 212 1190 -/+ buffers/cache: 708 63559 Swap: 0 0 0 [root@node1 ~]# free -g total used free shared buffers cached Mem: 62 2 60 0 0 1 -/+ buffers/cache: 0 62 Swap: 0 0 0 [root@node1 ~]# lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 16 On-line CPU(s) list: 0-15 Thread(s) per core: 2 Core(s) per socket: 8 Socket(s): 1 NUMA node(s): 1 Vendor ID: GenuineIntel CPU family: 6 Model: 79 Model name: Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz Stepping: 1 CPU MHz: 2095.148 BogoMIPS: 4190.29 Hypervisor vendor: KVM Virtualization type: full L1d cache: 32K L1i cache: 32K L2 cache: 256K L3 cache: 20480K NUMA node0 CPU(s): 0-15 [root@node1 ~]# df -h 文件系統 容量 已用 可用 已用% 掛載點 /dev/sda2 100G 3.1G 97G 4% / devtmpfs 7.7G 0 7.7G 0% /dev tmpfs 7.8G 0 7.8G 0% /dev/shm tmpfs 7.8G 233M 7.5G 3% /run tmpfs 7.8G 0 7.8G 0% /sys/fs/cgroup /dev/sda1 500M 9.8M 490M 2% /boot/efi /dev/sda4 1.8T 9.3G 1.8T 1% /data tmpfs 1.6G 0 1.6G 0% /run/user/1000
大數據集群安裝步驟