1. 程式人生 > >安裝 Tensorflow

安裝 Tensorflow

wheel 但是 memory nump ado rtu rime new you

環境:Ubuntu 16.04 64bit

1. 安裝Anaconda

  Anaconda 是 Python 的一個科學計算發行版,內置了數百個Python經常會使用的庫,也包括了許多機器學習和數據挖掘依賴的庫,包括 Scikit-learn、NumPy、SciPy 和 Pandas等,其中可能有一些還是 Tensorflow 的依賴庫。

  Anaconda 提供了一個編譯好的環境可以直接安裝。同時 Anaconda 自動集成了最新版的MKL(Math Kernel Libaray)庫,這是Intel推出的底層數值計算庫,功能上包含了 BLAS(Basic Linear Algebra Software)等矩陣計算庫的功能,可以作為 NumPy、SciPy、Scikit-learn、NumExpr 等庫的底層依賴,加速這些庫的矩陣運算。

  簡單來說,Anaconda 是目前最好的科學計算的 Python 環境,方便了安裝,也提高了性能。

  安裝步驟:

  1) 在官網上下載(www.continuum.io/downloads)相應版本

  2) 在下載目錄下執行命令,例如: bash Anaconda3-4.3.1-Linux-x86_64.sh

  3) 回車確認,進入 Anaconda 的 License 文檔,可以按 q 跳過,然後輸入 yes 確認。下一步會讓輸入 Anaconda 的安裝路徑,可以按回車鍵使用默認路徑

4) 安裝完成,程序提示是否把 anaconda3 的 binary 路徑加入到 .bashrc,建議添加。這樣 python 命令就可以自動使用 Anaconda 的 Python 的環境了。

2. 安裝Tensorflow

  由於是在 Python 的環境下使用 Tensorflow,可以使用 Python 的默認包管理器安裝。如果 pip 的版本>9,可以直接運行:

pip install tensorflow
[email protected]:~$ pip install tensorflow
Collecting tensorflow
  Downloading tensorflow-1.1.0-cp36-cp36m-manylinux1_x86_64.whl (31.4MB)
    100% |?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..| 31
.4MB 48kB/s Requirement already satisfied: six>=1.10.0 in ./anaconda3/lib/python3.6/site-packages (from tensorflow) Collecting protobuf>=3.2.0 (from tensorflow) Downloading protobuf-3.3.0-cp36-cp36m-manylinux1_x86_64.whl (5.7MB) 100% |?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..| 5.7MB 242kB/s Requirement already satisfied: wheel>=0.26 in ./anaconda3/lib/python3.6/site-packages (from tensorflow) Requirement already satisfied: werkzeug>=0.11.10 in ./anaconda3/lib/python3.6/site-packages (from tensorflow) Requirement already satisfied: numpy>=1.11.0 in ./anaconda3/lib/python3.6/site-packages (from tensorflow) Requirement already satisfied: setuptools in ./anaconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg (from protobuf>=3.2.0->tensorflow) Installing collected packages: protobuf, tensorflow Successfully installed protobuf-3.3.0 tensorflow-1.1.0

================================分割線========================================

但是直接安裝時,由於一些編譯參數的原因,在使用 Tensorflow 時,會出現如下的warnings:

[email protected]:~$ python
>>> import tensorflow as tf
>>> hello = tf.constant(Hello, TensorFlow!)
>>> sess = tf.Session()
>>> print(sess.run(hello))

在我的機器上當執行到 sess = tf.Session() 時,會提示:

2017-05-25 07:33:08.176641: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasnt compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2017-05-25 07:33:08.176687: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasnt compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-05-25 07:33:08.176701: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasnt compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.

意思是說:你的機器上有這些指令集可以用,並且用了他們會加快你的 CPU 運行速度,但是你的 TensorFlow 在編譯的時候並沒有用到這些指令集。

如果覺得不好的話,可以嘗試用編譯源碼安裝解決。

編譯源碼安裝

1. 安裝 Bazel

  查看:https://bazel.build/versions/master/docs/install-ubuntu.html#install-on-ubuntu

  Bazel 是 Google 自家的編譯工具,使用它來編譯 Tensorflow :

  1) Add Bazel distribution URI as a package source (one time setup),執行命令

echo "deb [arch=amd64] http://storage.googleapis.com/bazel-apt stable jdk1.8" | sudo tee /etc/apt/sources.list.d/bazel.list
curl https://bazel.build/bazel-release.pub.gpg | sudo apt-key add -

  If you want to use the JDK 7, please replace jdk1.8 with jdk1.7 and if you want to install the testing version of Bazel, replace stable with testing.

  2) Install and update Bazel,執行命令

sudo apt-get update && sudo apt-get install bazel

  3) Once installed, you can upgrade to a newer version of Bazel with:

sudo apt-get upgrade bazel

2. 卸載已經安裝的 Tensorflow:

pip uninstall tensorflow 

3. 克隆 Tensorflow 倉庫:

git clone --recurse-submodules https://github.com/tensorflow/tensorflow 

4. 配置Tensorflow

運行configure腳本

[email protected]:~/share/tensorflow$ cd tensorflow/
[email protected]:~/share/tensorflow/tensorflow$ ./configure 
Please specify the location of python. [Default is /home/jingyg/anaconda3/bin/python]: 
Found possible Python library paths:
  /home/jingyg/anaconda3/lib/python3.6/site-packages
Please input the desired Python library path to use.  Default is [/home/jingyg/anaconda3/lib/python3.6/site-packages]

Using python library path: /home/jingyg/anaconda3/lib/python3.6/site-packages
Do you wish to build TensorFlow with MKL support? [y/N] 
No MKL support will be enabled for TensorFlow
Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native]: 
Do you wish to use jemalloc as the malloc implementation? [Y/n] 
jemalloc enabled
Do you wish to build TensorFlow with Google Cloud Platform support? [y/N] 
No Google Cloud Platform support will be enabled for TensorFlow
Do you wish to build TensorFlow with Hadoop File System support? [y/N] 
No Hadoop File System support will be enabled for TensorFlow
Do you wish to build TensorFlow with the XLA just-in-time compiler (experimental)? [y/N] 
No XLA support will be enabled for TensorFlow
Do you wish to build TensorFlow with VERBS support? [y/N] 
No VERBS support will be enabled for TensorFlow
Do you wish to build TensorFlow with OpenCL support? [y/N] 
No OpenCL support will be enabled for TensorFlow
Do you wish to build TensorFlow with CUDA support? [y/N] 
No CUDA support will be enabled for TensorFlow
Extracting Bazel installation...
.................
INFO: Starting clean (this may take a while). Consider using --async if the clean takes more than several minutes.
Configuration finished

可以根據自己的情況進行選擇。

5. 編譯(比較耗時):

bazel build -c opt --copt=-msse4.1 --copt=-msse4.2 --copt=-mavx //tensorflow/tools/pip_package:build_pip_package

Tips:最好先增加下 swap 的空間:

# 生成swap鏡像文件
sudo dd if=/dev/zero of=/mnt/1024Mb.swap bs=1M count=1024
# 對該鏡像文件格式化
sudo mkswap /mnt/1024Mb.swap
# 掛載該鏡像文件 
sudo swapon /mnt/1024Mb.swap

使用free -m 即可查看到swap空間已經增加成功

否則,在編譯過程中,可能出現內存不足的問題:
ERROR: /home/jingyg/share/tensorflow/tensorflow/tensorflow/core/kernels/BUILD:2339:1: C++ compilation of rule //tensorflow/core/kernels:cwise_op failed: gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -B/usr/bin -B/usr/bin -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 -D_FORTIFY_SOURCE=1 -DNDEBUG ... (remaining 134 argument(s) skipped): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 4.
virtual memory exhausted: Cannot allocate memory

6. 上述命令會生成一個叫做 build_pip_package 的腳本,按照如下命令運行這個腳本,在 /tmp/tensorflow_pkg 文件夾中創建pip的安裝包:

bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg 

7. 用pip命令安裝:

pip install /tmp/tensorflow_pkg/tensorflow-1.2.0rc0-cp36-cp36m-linux_x86_64.whl 

Tensorflow測試:

  退出 Tensorflow 目錄,如果不退出執行下面的命令,會出現 Failed to load the native TensorFlow runtime 的錯誤

[email protected]:~/share/tensorflow/tensorflow$ cd ..
[email protected]:~/share/tensorflow$ python
Python 3.6.0 |Anaconda 4.3.1 (64-bit)| (default, Dec 23 2016, 12:22:00) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf

Tips: ImportError: /home/jingyg/anaconda3/bin/../lib/libstdc++.so.6: version `GLIBCXX_3.4.21‘ not found

原因是 Anaconda 裏的 libgcc 版本低了

解決辦法,安裝最新的:

[email protected]:~/share/tensorflow$ conda install libgcc

安裝 Tensorflow