最全caffe安裝踩坑記錄（Anaconda,nvidia-docker,Linux編譯）
阿新 • • 發佈：2019-07-20
Anaconda,nvidia-docker,Linux三種方式安裝caffe
1.Anaconda安裝caffe
1.首先安裝anaconda
2.建立虛擬環境（python2.7）
conda create -n caffe python=2.7 anaconda
3.安裝caffe
# 然後下面二選一即可，安裝caffe conda install -c conda-forge caffe conda install -c conda-forge/label/broken caffe
4.注意：
# 如果出現numpy匯入錯誤，是因為 conda-forge中的numpy版本過低，可再輸入下面命令 pip install numpy --upgrade
5.測試
# 測試是否成功 source activate caffe python import caffe
2.nvidia-docker安裝caffe-gpu
注意：docker僅支援cpu nvidia-docker支援GPU
1.安裝nvidia-docker
wget -P /tmp https://github.com/NVIDIA/nvidia-docker/releases/download/v1.0.1/nvidia-docker_1.0.1-1_amd64.deb sudo dpkg -i /tmp/nvidia-docker*.deb
# 測試nvidia-docker run --rm nvidia/cuda nvidia-smi
2.拉取caffe-gpu映象
nvidia-docker pull bvlc/caffe:gpu
3.啟動一個容器
nvidia-docker run --rm -ti <映象名字> docker attach <容器id> # 已開啟的容器
# 退出容器 ctrl + d （容器不執行）或者ctrl q+p(容器後臺執行)
# 進入未執行的容器
docker ps -a # 檢視容器id
docker start <id>
docker attach <id># 容器改名
dockers rename 原id 新id
4.更新源
apt uptate
5.測試
#caffe安裝目錄預設在/opt/caffe,若想支援opencv，多gpu，請自行編譯，可參看第三種方式編譯安裝caffe
python import caffe
3.Linux編譯caffe，並支援多GPU
1.linux安裝git
# Centos yum install git # Ubuntu sudo apt-get install git
git clone https://github.com/BVLC/caffe.git
2.編譯前的Makefile.config配置
# 新建Makefile.config
mv Makefile.config.example Makefile.config
注意：
1 ## Refer to http://caffe.berkeleyvision.org/installation.html 2 # Contributions simplifying and improving our build system are welcome! 3 4 # cuDNN acceleration switch (uncomment to build with cuDNN). 5 USE_CUDNN := 1 6 7 # CPU-only switch (uncomment to build without GPU support). 8 # CPU_ONLY := 1 9 10 # uncomment to disable IO dependencies and corresponding data layers 11 # USE_OPENCV := 0 12 # USE_LEVELDB := 0 13 # USE_LMDB := 0 14 15 # uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary) 16 # You should not set this flag if you will be reading LMDBs with any 17 # possibility of simultaneous read and write 18 # ALLOW_LMDB_NOLOCK := 1 19 20 # Uncomment if you're using OpenCV 3 21 OPENCV_VERSION := 3 22 23 # To customize your choice of compiler, uncomment and set the following. 24 # N.B. the default for Linux is g++ and the default for OSX is clang++ 25 # CUSTOM_CXX := g++ 26 27 # CUDA directory contains bin/ and lib/ directories that we need. 28 CUDA_DIR := /usr/local/cuda 29 # On Ubuntu 14.04, if cuda tools are installed via 30 # "sudo apt-get install nvidia-cuda-toolkit" then use this instead: 31 # CUDA_DIR := /usr 32 33 # CUDA architecture setting: going with all of them. 34 # For CUDA < 6.0, comment the *_50 through *_61 lines for compatibility. 35 # For CUDA < 8.0, comment the *_60 and *_61 lines for compatibility. 36 # For CUDA >= 9.0, comment the *_20 and *_21 lines for compatibility. 37 CUDA_ARCH := -gencode arch=compute_30,code=sm_30 \ 38 -gencode arch=compute_35,code=sm_35 \ 39 -gencode arch=compute_50,code=sm_50 \ 40 -gencode arch=compute_52,code=sm_52 \ 41 -gencode arch=compute_60,code=sm_60 \ 42 -gencode arch=compute_61,code=sm_61 \ 43 -gencode arch=compute_61,code=compute_61 44 45 # BLAS choice: 46 # atlas for ATLAS (default) 47 # mkl for MKL 48 # open for OpenBlas 49 BLAS := atlas 50 # Custom (MKL/ATLAS/OpenBLAS) include and lib directories. 51 # Leave commented to accept the defaults for your choice of BLAS 52 # (which should work)! 53 # BLAS_INCLUDE := /path/to/your/blas 54 # BLAS_LIB := /path/to/your/blas 55 56 # Homebrew puts openblas in a directory that is not on the standard search path 57 # BLAS_INCLUDE := $(shell brew --prefix openblas)/include 58 # BLAS_LIB := $(shell brew --prefix openblas)/lib 59 60 # This is required only if you will compile the matlab interface. 61 # MATLAB directory should contain the mex binary in /bin. 62 # MATLAB_DIR := /usr/local 63 # MATLAB_DIR := /Applications/MATLAB_R2012b.app 64 65 # NOTE: this is required only if you will compile the python interface. 66 # We need to be able to find Python.h and numpy/arrayobject.h. 67 PYTHON_INCLUDE := /usr/include/python2.7 \ 68 /usr/lib/python2.7/dist-packages/numpy/core/include 69 # Anaconda Python distribution is quite popular. Include path: 70 # Verify anaconda location, sometimes it's in root. 71 # ANACONDA_HOME := $(HOME)/anaconda 72 # PYTHON_INCLUDE := $(ANACONDA_HOME)/include \ 73 # $(ANACONDA_HOME)/include/python2.7 \ 74 # $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include 75 76 # Uncomment to use Python 3 (default is Python 2) 77 # PYTHON_LIBRARIES := boost_python3 python3.5m 78 # PYTHON_INCLUDE := /usr/include/python3.5m \ 79 # /usr/lib/python3.5/dist-packages/numpy/core/include 80 81 # We need to be able to find libpythonX.X.so or .dylib. 82 PYTHON_LIB := /usr/lib 83 # PYTHON_LIB := $(ANACONDA_HOME)/lib 84 85 # Homebrew installs numpy in a non standard path (keg only) 86 # PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include 87 # PYTHON_LIB += $(shell brew --prefix numpy)/lib 88 89 # Uncomment to support layers written in Python (will link against Python libs) 90 WITH_PYTHON_LAYER := 1 91 92 # Whatever else you find you need goes here. 93 INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial 94 LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu/hdf5/serial 95 96 # If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies 97 # INCLUDE_DIRS += $(shell brew --prefix)/include 98 # LIBRARY_DIRS += $(shell brew --prefix)/lib 99 100 # NCCL acceleration switch (uncomment to build with NCCL) 101 # https://github.com/NVIDIA/nccl (last tested version: v1.2.3-1+cuda8.0) 102 USE_NCCL := 1 103 104 # Uncomment to use `pkg-config` to specify OpenCV library paths. 105 # (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.) 106 # USE_PKG_CONFIG := 1 107 108 # N.B. both build and distribute dirs are cleared on `make clean` 109 BUILD_DIR := build 110 DISTRIBUTE_DIR := distribute 111 112 # Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171 113 # DEBUG := 1 114 115 # The ID of the GPU that 'make runtest' will use to run unit tests. 116 TEST_GPUID := 0 117 118 # enable pretty build (comment to see full commands) 119 Q ?= @
1.第5行USE_CUDNN := 1（安裝和cuda匹配的cudnn）
2.第21行 OPENCV_VERSION := 3（安裝opencv）
sudo apt install git git clone https://github.com/jayrambhia/Install-OpenCV cd Install-OpenCV/Ubuntu/ chmod +x * ./opencv_latest.sh make -j8 && make pycaffe
3.第28行CUDA_DIR := /usr/local/cuda（安裝cuda的目錄）
4.第49行BLAS := atlas
5.第67，68行
PYTHON_INCLUDE := /usr/include/python2.7 \ /usr/lib/python2.7/dist-packages/numpy/core/include
6.第93，94行
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu/hdf5/serial
7.第102行USE_NCCL := 1（支援多gpu）
git clone https://github.com/NVIDIA/nccl.git cd nccl sudo make install sudo gedit ~/.bashrc export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/home/XXX/nccl/build/lib/" #XXX為使用者名稱 source .bashrc 使用多個gpu 我們在使用caffe的工具訓練網路時，在語句的最後加上以下內容即可 -gpu all #執行所有的gpu -gpu 1,2 #執行1號2號gpu
3.開始編譯
sudo make all sudo make test sudo make pycaffe sudo make runtest
4.測試
python import caffe