前言

DensePose是基於caffe2和Detectron的.現在caffe2已經被併入pytorch,所以要想原始碼安裝caffe2,只能先克隆pytorch,同時要注意的是,Detectron只適用於GPU版本的caffe2.

官方說原始碼安裝caffe2以及detectron需要cuda8.0+cudnn6.0,經過親自試驗,cuda9.0+cudnn7.1也可以.

注意:經過多次嘗試,發現只有使用python2.7版本才能編譯成功gpu版本的caffe2!!!python3編譯不通過!!!!

基礎環境:ubuntu16.04+cuda9.0+cudnn7.1+python2.7

安裝依賴:

sudo apt-get install -y --no-install-recommends \
      libgflags-dev \
      cmake

clone+build

(編譯時間很長,如果之前基礎環境沒問題,這裡應該就沒問題,之前我就是用的python3,結果利用cuda9.0時候只能編譯成功cpu版本的caffe2,而利用cuda8.0的時候直接編譯不通過,所以再強調一遍,一定要python2.7!!)

git clone https://github.com/pytorch/pytorch.git && cd pytorch
git submodule update --init --recursive
python setup.py install
 

檢查caffe2是否安裝成功

python -c 'from caffe2.python import core' 2>/dev/null && echo "Success" || echo "Failure"
Success

檢查是否是gpu版本的caffe2

>>> import caffe2
>>> from caffe2.python import workspace
>>> print(workspace.NumCudaDevices())
1

至此,caffe2編譯安裝成功

安裝cocoapi

# COCOAPI=/path/to/clone/cocoapi
git clone https://github.com/cocodataset/cocoapi.git $COCOAPI
cd $COCOAPI/PythonAPI
# Install into global site-packages
make install

安裝Densepose

clone

# DENSEPOSE=/path/to/clone/densepose
git clone https://github.com/facebookresearch/densepose $DENSEPOSE

安裝python依賴

pip install -r $DENSEPOSE/requirements.txt

編譯

cd densepose
make

Check that Detectron tests pass

cd densepose
python detectron/tests/test_spatial_narrow_as_op.py

(Python test_spatial_narrow_as_op.py出錯)

此時可能會報錯說沒有發現Detectron ops lib,此時我們需要將之前編譯的build的路徑新增到環境中

sudo gedit .bashrc
在最後加上:export PYTHONPATH=/home/wlw/pytorch/build:$PYTHONPATH

或者開啟:densepose/detectron/utils/env.py
在import sys下一行加上:sys.path.insert(0, '/home/wlw/pytorch/build')

重新執行,OK!

Build the custom operators library

(這一步是問題最多的一步)(make ops failing (Could not find a package "Caffe2")

cd densepose
make ops

當執行上述命令後,可能會報以下錯誤(不能發現caffe2):

By not providing "FindCaffe2.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "Caffe2", but
  CMake did not find one.

  Could not find a package configuration file provided by "Caffe2" with any
  of the following names:

   Caffe2Config.cmake
   caffe2-config.cmake

  Add the installation prefix of "Caffe2" to CMAKE_PREFIX_PATH or set
  "Caffe2_DIR" to a directory containing one of the above files.  If "Caffe2"
  provides a separate development package or SDK, be sure it has been
  installed.

這是因為caffe2的路徑沒找對,沒有找到正確的Caffe2Config.cmake,此時需要在終端輸入:

export Caffe2_DIR=/home/wlw/pytorch/torch/lib/tmp_install/share/cmake/Caffe2

這裡有個小技巧:可能你輸入上述命令後還會有其他比如:recipe for target 'ops' failed的問題,那還是因為沒有找到正確的Caffe2Config.cmake,此時你可以在你的計算機中找到所有Caffe2Config.cmake的檔案,將他們的路徑一個一個放到Caffe2_DIR之後,然後sudo make clean,重新make,再make ops,不出意外,總有一個路徑正確,然後make ops成功.

注意:上述的make 和make ops 前面最好不要加sudo,反正我加了sudo之後,就總是出錯.

Check that the custom operator tests pass

cd densepose
python detectron/tests/test_zero_even_op.py

這一步應該是會出現最後一個問題:OSError: /home/wlw/densepose/build/libcaffe2_detectron_custom_ops_gpu.so: undefined symbol: _ZN6google8protobuf8internal9ArenaImpl28AllocateAlignedAndAddCleanupEmPFvPvE

這是因為protobuf的多版本衝突問題,所以需要在densepose/目錄下找到CMakeLists.txt,進行set(PROTOBUF_LIB "/home/wlw/pytorch/build/lib/libprotobuf.a")設定,完整檔案如下:

cmake_minimum_required(VERSION 2.8.12 FATAL_ERROR)

# Find the Caffe2 package.
# Caffe2 exports the required targets, so find_package should work for
# the standard Caffe2 installation. If you encounter problems with finding
# the Caffe2 package, make sure you have run `make install` when installing
# Caffe2 (`make install` populates your share/cmake/Caffe2).
find_package(Caffe2 REQUIRED)

if (${CAFFE2_VERSION} VERSION_LESS 0.8.2)
  # Pre-0.8.2 caffe2 does not have proper interface libraries set up, so we
  # will rely on the old path.
  message(WARNING
      "You are using an older version of Caffe2 (version " ${CAFFE2_VERSION}
      "). Please consider moving to a newer version.")
  include(cmake/legacy/legacymake.cmake)
  return()
endif()

# Add compiler flags.
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -std=c11")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11 -O2 -fPIC -Wno-narrowing")

# Print configuration summary.
include(cmake/Summary.cmake)
detectron_print_config_summary()

# Collect custom ops sources.
file(GLOB CUSTOM_OPS_CPU_SRCS ${CMAKE_CURRENT_SOURCE_DIR}/detectron/ops/*.cc)
file(GLOB CUSTOM_OPS_GPU_SRCS ${CMAKE_CURRENT_SOURCE_DIR}/detectron/ops/*.cu)

# Install custom CPU ops lib.
add_library(
     caffe2_detectron_custom_ops SHARED
     ${CUSTOM_OPS_CPU_SRCS})

#target_link_libraries(caffe2_detectron_custom_ops caffe2_library)
# static protobuf library

add_library(libprotobuf STATIC IMPORTED)
set(PROTOBUF_LIB "/home/wlw/pytorch/build/lib/libprotobuf.a")
set_property(TARGET libprotobuf PROPERTY IMPORTED_LOCATION "${PROTOBUF_LIB}")
target_link_libraries(caffe2_detectron_custom_ops caffe2_library libprotobuf)
install(TARGETS caffe2_detectron_custom_ops DESTINATION lib)

# Install custom GPU ops lib, if gpu is present.
if (CAFFE2_USE_CUDA OR CAFFE2_FOUND_CUDA)
  # Additional -I prefix is required for CMake versions before commit (< 3.7):
  # https://github.com/Kitware/CMake/commit/7ded655f7ba82ea72a82d0555449f2df5ef38594
  list(APPEND CUDA_INCLUDE_DIRS -I${CAFFE2_INCLUDE_DIRS})
  CUDA_ADD_LIBRARY(
      caffe2_detectron_custom_ops_gpu SHARED
      ${CUSTOM_OPS_CPU_SRCS}
      ${CUSTOM_OPS_GPU_SRCS})

  #target_link_libraries(caffe2_detectron_custom_ops_gpu caffe2_gpu_library)
  target_link_libraries(caffe2_detectron_custom_ops_gpu caffe2_gpu_library libprotobuf)
  install(TARGETS caffe2_detectron_custom_ops_gpu DESTINATION lib)
endif()

修改完成後,重新sudo make clean+make+make ops,然後重新執行:python detectron/tests/test_zero_even_op.py,成功後的介面如下:

至此,所有環境搭建成功!