利用最新的CentOS7.5，hadoop3.1,spark2.3.2搭建spark叢集
1. 橋接模式，靜態ip上外網：vi /etc/sysconfig/network-scripts/ifcfg-ens33
TYPE=EthernetPROXY_METHOD=noneBROWSER_ONLY=noBOOTPROTO=staticDEFROUTE=yesIPV4_FAILURE_FATAL=noIPV6INIT=yesIPV6_AUTOCONF=yesIPV6_DEFROUTE=yesIPV6_FAILURE_FATAL=noIPV6_ADDR_GEN_MODE=stable-privacyNAME=ens33UUID=292b882b-ebd2-4d03-9ce5-402da7858f6dDEVICE=ens33ONBOOT=yesIPADDR=192.168.2.124NETMASK=255.255.255.0GATEWAY=192.168.2.1DNS1=127.207.160.106DNS2=219.239.26.42
或者用下面的DNS也是可以的：DNS1=8.8.8.8DNS2=8.8.4.4
systemctl restart network
cat /etc/resolv.conf看一下nameserver 8.8.8.8nameserver 8.8.4.42. 關閉防火牆 systemctl stop firewalld.service #停止firewall systemctl disable firewalld.service #禁止firewall開機啟動
3. 禁止： vi /etc/sysconfig/selinux 改成disabled SELINUX=enforcing 改成： SELINUX=disabled4. 配置yum源 網路yum源 yum install wget cd /etc/yum.repos.d/wget http://mirrors.aliyun.com/repo/Centos-7.repomv CentOS-Base.repo CentOS-Base.repo.bakmv Centos-7.repo CentOs-Base.repoyum clean allyum makecacheyum update
5. 免密碼登陸：永久修改主機名：在三臺機器上分別執行下列語句：注意三臺機器的使用者名稱不同hostnamectl --static set-hostname spark1ssh-keygen -t rsacd /root/.sshcp id_rsa.pub authorized_keys然後從spakr1主機免密碼登陸spark2，spark3ssh-copy-id -i spark2ssh-copy-id -i spark3同理在spark2執行下列語句：ssh-copy-id -i spark1ssh-copy-id -i spark3同理在spark3執行下列語句：ssh-copy-id -i spark1ssh-copy-id -i spark2
6. hadoop叢集搭建：
未完待續。