1. 程式人生 > >centos7 docker 安裝hadoop 2.7.2記錄

centos7 docker 安裝hadoop 2.7.2記錄

docker常用命令:

執行映象:
<code>docker run -i -t centos  /bin/bash</code>

提交映象:
<code>docker commit 3a09b2588478 ubuntu:mynewimage</code>

掛載目錄:
docker run --privileged=true  -v /home/admin/Downloads:/mnt -i -t centos

複製檔案:
cp hostpath dockerpath



下載:

hadoop-2.7.2.tar.gz

jdk-7u67-linux-x64.tar.gz

解壓檔案:

<pre><code>tar xvzf hadoop-2.7.2.tar.gz
</code>
<span style="color:#3366ff;"><span style="color:#000000;"><strong>設定環境變數:</strong>
vi /etc/profile</span></span><code></code>檔案。在檔案末尾加入下面配置資訊:

export JAVA_HOME=/root/soft/jdk1.7.0_67
export CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export PATH=$PATH:$JAVA_HOME/bin
export HADOOP_HOME=/root/soft/hadoop-2.7.2
export HADOOP_CONFIG_HOME=$HADOOP_HOME/etc/hadoop
export PATH=$PATH:$HADOOP_HOME/bin
export PATH=$PATH:$HADOOP_HOME/sbin

使之立即生效
source /etc/profile

<strong>配置hadoop:</strong>
<pre><code>cd $HADOOP_CONFIG_HOME</code>
cp mapred-site.xml.template mapred-site.xml

[[email protected] hadoop]# cat core-site.xml                                                                                                    
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
    <property>
            <name>hadoop.tmp.dir</name>
            <value>/root/soft/hadoop-2.7.2/tmp</value>
            <description>A base for other temporary directories.</description>
    </property>

    <property>
            <name>fs.default.name</name>
            <value>hdfs://master:9000</value>
            <final>true</final>
            <description>The name of the default file system.  A URI whose
            scheme and authority determine the FileSystem implementation.  The
            uri's scheme determines the config property (fs.SCHEME.impl) naming
            the FileSystem implementation class.  The uri's authority is used to
            determine the host, port, etc. for a filesystem.</description>
    </property>
</configuration>


[
[email protected]
hadoop]# vi hdfs-site.xml
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
   <property>
        <name>dfs.replication</name>
        <value>2</value>
        <final>true</final>
        <description>Default block replication.
        The actual number of replications can be specified when the file is created.
        The default is used if replication is not specified in create time.
        </description>
    </property>

    <property>
        <name>dfs.namenode.name.dir</name>
        <value>/root/soft/hadoop-2.7.2/namenode</value>
        <final>true</final>
    </property>

    <property>
        <name>dfs.datanode.data.dir</name>
        <value>/root/soft/hadoop-2.7.2/datanode</value>
        <final>true</final>
    </property>

</configuration>


[[email protected] hadoop]# vi mapred-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
   <property>
        <name>mapred.job.tracker</name>
        <value>master:9001</value>
        <description>The host and port that the MapReduce job tracker runs
        at.  If "local", then jobs are run in-process as a single map
        and reduce task.
        </description>
    </property>
</configuration>


格式化 namenode,進入 hadoop/bin目錄下執行
hdfs namenode -format

安裝ssh
yum install openssh-server
yum -y install openssh-clients
生成訪問金鑰
[email protected]:/# cd ~/
[email protected]:~# ssh-keygen -t rsa -P '' -f ~/.ssh/id_dsa
[email protected]:~# cd .ssh
[email protected]:~/.ssh# cat id_dsa.pub >> authorized_keys

儲存映象副本

這裡我們將安裝好Hadoop的映象儲存為一個副本。

[email protected]:~# exit
[email protected]:~$ docker commit -m "hadoop-2.7.2 install" 8ef06706f88d centos:hadoop

檢視docker ip
docker inspect --format '{{.NetworkSettings.IPAddress}}' 33d36f3b5c4c

<pre><code></code><pre name="code" class="plain"><pre><code>docker run -d -ti --name master -h master centos:hadoop</code>

<pre><code>docker run -d -ti --name slave1 -h slave1 centos:hadoop</code>

<pre><code></code><pre name="code" class="plain"><pre><code>docker run -d -ti --name slave2 -h slave2 centos:hadoop</code>




相關推薦

centos7 docker 安裝hadoop 2.7.2記錄

docker常用命令: 執行映象: <code>docker run -i -t centos /bin/bash</code> 提交映象: <code>docker commit 3a09b2588478 ubuntu:mynew

CentOS7.0安裝配置hadoop2.7.0 資源準備 資源下載: hadoop-2.7.0.tar.gz 密碼:727y jdk-8u45-linux-x64.tar.gz 密碼:d8bm

CentOS7.0安裝配置hadoop2.7.0 資源準備 資源下載: 注意事項: 如果自己下載資源的話,注意hadoop,jdk,centos都應該是64位或者32位的,以免出現無法預料的錯誤,上面的資源都是64位的我是在mac下配置的,virtual box是ios x系統的,如果是其它系統的另

Hadoop環境 IDE配置(在eclipse中安裝hadoop-eclipse-plugin-2.7.3.jar插件)

map bubuko other 9.png 查看 3.2 div 集群 點擊 一、hadoop-eclipse-plugin-2.7.3.jar插件下載點擊下載 二、把插件放到eclipse的安裝目錄dropins下 三、eclipse上的配置 3.1 打開Wind

大資料基礎(五)從零開始安裝配置Hadoop 2.7.2+Spark 2.0.0到Ubuntu 16.04

raw to spark 0 install ubuntu 14.04.01 desktop x64 1 system基礎配置 《以下都是root模式》 1.3 root password sudo passwd root 1.5 root登入選項 a.在terminal下

Hadoop-2.7.2分散式安裝手冊

目錄 目錄 1 1. 前言 3 3. 部署 5 4. 約定 7

ubuntu 14.04 下單機安裝 hadoop 2.7.2+scala 2.11.8+spark 2.0偽分散式教程

一.安裝java 將java下載後手動解壓到/home/che資料夾下(可採用右擊壓縮包,單擊“提取”) 在終端(可用Ctrl+Alt+T快捷鍵開啟)中輸入: sudo gedit /etc/profile 在開啟的文字中新增: expor

Hadoop 2.7.2 + Sqoop2 安裝使用

dfs- mon form 64bit shell 架設 sqoop2 address bsp 1. 安裝環境 本次使用兩個節點,一個作為master,一個作為slave。兩個虛擬機使用virtualbox。操作系統為ubuntu 64bit server(Ubuntu

CentOS7下編譯Hadoop-2.7.2詳解

1.工具準備,最靠譜的是hadoop說明文件裡要求具備的那些工具。 到hadoop官網,點選source下載hadoop-2.7.2-src.tar.gz。 利用FileZilla上傳壓縮包到/opt/software/目錄下 在該目錄下,解壓到/opt/modu

hadoop 2.7.2使用maven編譯安裝

(1)依賴包: libssl-dev   zlib1g-dev  openssl  protobuf2.5 maven  jdk1.7 cmake openssh-server (2) mvn  clean package -DskipTests -Pdist,na

CentOS7.3 部署Haproxy 1.7.2

-- sts lob 末尾 裝包 backend iptables 失敗 fresh haproxy:http://www.haproxy.org/ 本文涉及haproxy的安裝,並做簡單配置。 一.環境準備 1. 操作系統 CentOS-7-x86_64-Everyt

CentOS7安裝 zookeeper-3.5.2 服務

stand enable pan ble imu alpha 日誌文件 logs req 1 [root@centos-linux src]# pwd 2 /usr/local/src 3 4 # 1、創建 /usr/local/src/zookeep

CentOS7 靜默安裝Oracle 11gR2(11.2.0.1)

文件的 sil swapon profile pytho ise sym 端口 啟動 最近有個項目需要遷移,項目中數據庫用的是oracle,期間折騰了好久,特此記錄。 一,下載Oracle 1,首先下載Oracle 11gR2,地址如下:http://www.oracle.

centos7 docker 安裝 mysql5.7.24 導入11G的sql

install 替換 yum 驗證 .json working mon uname 是否 Docker 要求 CentOS 系統的內核版本高於 3.10 ,查看本頁面的前提條件來驗證你的CentOS 版本是否支持 Docker 。通過 uname -r 命令查看你當前的內核

centos7 docker 安裝 mysql5.7.24 匯入11G的sql

先在CentOS7裡面安裝docker Docker 要求 CentOS 系統的核心版本高於 3.10 ,檢視本頁面的前提條件來驗證你的CentOS 版本是否支援 Docker 。通過 uname -r 命令檢視你當前的核心版本 uname -r 3.10.0-514.el7.x86_64 1、初步安

hadoop 2.7.2 + zookeeper 高可用叢集部署

一.環境說明 虛擬機器:vmware 11 作業系統:Ubuntu 16.04 Hadoop版本:2.7.2 Zookeeper版本:3.4.9 二.節點部署說明 三.Hosts增加配置 sudo gedit /etc/hosts wxzz-pc、wxzz-pc0、

Hadoop 2.7.2啟動WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform...

在/etc/profile檔案中配置如下環境變數 export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native export HADOOP_O

Centos7 & Docker & Jenkins & ASP.NET Core 2.0 自動化釋出和部署

寫在前面   Docker一直很火熱,一直想把原本的Jenkins自動部署工具搬到Docker上面,無奈今年一直忙於各種事情,遲遲未實施這個事情,正好迎來了dotnet core 2.0 的正式釋出,升級專案的同時,順便直接將Jenkins搬到Docker上。為什麼要寫這篇文章呢?因為找過相關的資料,大多數文

ubuntu搭建hadoop 2.7.2 Single Node Cluster及windows eclipse yarn提交Mapreduce筆記

環境配置 hadoop 版本:2.7.2 ubuntu 版本:lubuntu-16.04.1-desktop-amd64 java 版本:1.8.0_102 windows 版本:Microsoft Windows [版本 10.0.10586] eclipse 版本:Ne

Hadoop Shell命令(版本:2.7.2

將不 mis 無法 替換 text 系統 print facl hot Hadoop Shell命令(版本:2.7.2) 官方文檔2.7.2 官方中文1.0.4 概觀 appendToFile cat checksum chgrp chmod chown copyFro

docker centos7 原始碼安裝MySQL-5.7

增加使用者和目錄  [[email protected] /]# groupadd mysql [[email protected] /]# useradd -g mysql mysql [[email protected] /]# passwd mysql [[em