筆者是在已安裝好的偽分散式Hadoop環境下安裝的Spark。虛擬機器環境為：centos7。

Hadoop版本號為：

[centosm@centosm spark]$ hadoop version
Hadoop 2.7.2

一、Spark偽分散式安裝

1、確定hadoop環境已安裝好

3、解壓spark壓縮包並重命名解壓後的檔名為spark

     tar -zvxf  spark-2.1.1-bin-hadoop2.7.tgz 

4、修改配置檔案

     cd /usr/local/spark
     cp ./conf/spark-env.sh.template
 
 ./conf/spark-env.sh

     spark-env.sh 檔案後追加如下語句

vi spark-env.sh
export SPARK_DIST_CLASSPATH=$(/home/centosm/hadoopM/bin/hadoop classpath)

5、啟動spark，執行時出現下列問題

解決：

vi  spark-config.sh
export JAVA_HOME=/usr/java/jdk1.8.0_91

6、執行spark自帶的一些小例子，如下計算 π 的近似值

[centosm@centosm spark]$ ./bin/run-example SparkPi
 
 2>&1 | grep "Pi is roughly"
Pi is roughly 3.1428357141785708

7、進入spark互動模式

[[email protected] spark]$ ./bin/spark-shell            
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
 
 [jar:file:/home/centosm/spark/jars/slf4j-log4j12-1.7.16.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/centosm/hadoopM/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
17/06/22 14:51:48 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17/06/22 14:52:15 WARN metastore.ObjectStore: Failed to get database global_temp, returning NoSuchObjectException
Spark context Web UI available at http://192.168.22.125:4040
Spark context available as 'sc' (master = local[*], app id = local-1498114310652).
Spark session available as 'spark'.
Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /___/ .__/\_,_/_/ /_/\_\   version 2.1.1
      /_/

Using Scala version 2.11.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_91)
Type in expressions to have them evaluated.
Type :help for more information.

scala> 

在互動模式的情啟動的情況可以進入web頁面檢視相關資訊，如下：

8、yum 安裝sbt

[centosm@centosm test]$ curl https://bintray.com/sbt/rpm/rpm > bintray-sbt-rpm.repo
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   159    0   159    0     0     23      0 --:--:--  0:00:06 --:--:--    37
[centosm@centosm test]$
[centosm@centosm test]$ sudo mv bintray-sbt-rpm.repo /etc/yum.repos.d/
[root@centosm test]# yum install sbt
[centosm@centosm bin]$ sbt --version
Getting org.scala-sbt sbt 0.13.15  (this may take some time)...