前幾天Google的IO大會上釋出的ML Kit，ML Kit為端上部署深度學習模型提供了一套完整的解決方案，本地執行、雲端都支援。裡面本地部署用到的就是Tensorflow lite。

Tensorflow Lite是在Google去年IO大會上發表的，目前Tensorflow Lite也還在不斷的完善迭代中。

Tensorflow Lite在Android和iOS上部署官網有比較詳細的介紹以及對應的Demo。而對於ARM板子上的部署及測試，官網及網上的資料則相對較少。本文主要描述如何把Tensorflow Lite編譯到ARM板子上，並執行相應的Demo。

0.準備工作：在Ubuntu上準備ARM的交叉編譯環境

可以通過 apt-get install 直接在ubuntu上安裝交叉編譯環境

sudo apt-get install g++-arm-linux-gnueabihf
sudo apt-get install -y gcc-arm-linux-gnueabihf

1.下載Tensorflow原始碼

git clone https://github.com/tensorflow/tensorflow

2.下載Tensorflow相關依賴包

先cd到Tensorflow工程的根目錄，然後執行下面的指令碼

./tensorflow/contrib/lite/download_dependencies.sh
 

3.編譯Tensorflow Lite

注意./tensorflow/contrib/lite/build_rpi_lib.sh中的目標編譯平臺是ARMV7，這裡最好不要改，即使你的目標平臺是ARMv8。改為ARMv8能也能編譯通過，但是貌似沒有把優化編譯進去，執行起來會非常慢，親測。

# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
 

#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================

set -e

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
cd "$SCRIPT_DIR/../../.."

#change CC_PREFIX if u need
CC_PREFIX=arm-linux-gnueabihf- make -j 3 -f tensorflow/contrib/lite/Makefile TARGET=RPI TARGET_ARCH=armv7

在根目錄下執行該指令碼，如下：

./tensorflow/contrib/lite/build_rpi_lib.sh

編譯結束，會在tensorflow/contrib/lite/gen/lib/rpi_armv7目錄下產生libtensorflow-lite.a靜態庫。

4.編譯 label_image Demo

第三步的 build_rpi_lib.sh 指令碼實際是呼叫的 ./tensorflow/contrib/lite/Makefile對Tensorflow Lite 原始碼進行編譯，但是該 Makefile 並不能編譯 tensorflow/contrib/lite/examples/label_image 目錄下的Demo，所以需要修改 Makefile 把 label_image 的原始碼配置到Makefile中，修改方式可以參考 Makefile 裡對 MINIMAL Demo 的配置。如果你不想自己改，下面是已經修改好的。

# Find where we're running from, so we can store generated files here.
ifeq ($(origin MAKEFILE_DIR), undefined)
    MAKEFILE_DIR := $(shell dirname $(realpath $(lastword $(MAKEFILE_LIST))))
endif

# Try to figure out the host system
HOST_OS :=
ifeq ($(OS),Windows_NT)
    HOST_OS = WINDOWS
else
    UNAME_S := $(shell uname -s)
    ifeq ($(UNAME_S),Linux)
            HOST_OS := LINUX
    endif
    ifeq ($(UNAME_S),Darwin)
        HOST_OS := OSX
    endif
endif

ARCH := $(shell if [[ $(shell uname -m) =~ i[345678]86 ]]; then echo x86_32; else echo $(shell uname -m); fi)

# Where compiled objects are stored.
OBJDIR := $(MAKEFILE_DIR)/gen/obj/
BINDIR := $(MAKEFILE_DIR)/gen/bin/
LIBDIR := $(MAKEFILE_DIR)/gen/lib/
GENDIR := $(MAKEFILE_DIR)/gen/obj/

# Settings for the host compiler.
CXX := $(CC_PREFIX)gcc
CXXFLAGS := --std=c++11 -O3 -DNDEBUG
CC := $(CC_PREFIX)gcc
CFLAGS := -O3 -DNDEBUG
LDOPTS :=
LDOPTS += -L/usr/local/lib
ARFLAGS := -r

INCLUDES := \
-I. \
-I$(MAKEFILE_DIR)/../../../ \
-I$(MAKEFILE_DIR)/downloads/ \
-I$(MAKEFILE_DIR)/downloads/eigen \
-I$(MAKEFILE_DIR)/downloads/gemmlowp \
-I$(MAKEFILE_DIR)/downloads/neon_2_sse \
-I$(MAKEFILE_DIR)/downloads/farmhash/src \
-I$(MAKEFILE_DIR)/downloads/flatbuffers/include \
-I$(GENDIR)
# This is at the end so any globally-installed frameworks like protobuf don't
# override local versions in the source tree.
INCLUDES += -I/usr/local/include

LIBS := \
-lstdc++ \
-lpthread \
-lm \
-lz

# If we're on Linux, also link in the dl library.
ifeq ($(HOST_OS),LINUX)
    LIBS += -ldl
endif

include $(MAKEFILE_DIR)/ios_makefile.inc
include $(MAKEFILE_DIR)/rpi_makefile.inc

# This library is the main target for this makefile. It will contain a minimal
# runtime that can be linked in to other programs.
LIB_NAME := libtensorflow-lite.a
LIB_PATH := $(LIBDIR)$(LIB_NAME)

# A small example program that shows how to link against the library.
MINIMAL_PATH := $(BINDIR)minimal
LABEL_IMAGE_PATH :=$(BINDIR)label_image

MINIMAL_SRCS := \
tensorflow/contrib/lite/examples/minimal/minimal.cc
MINIMAL_OBJS := $(addprefix $(OBJDIR), \
$(patsubst %.cc,%.o,$(patsubst %.c,%.o,$(MINIMAL_SRCS))))


LABEL_IMAGE_SRCS := \
tensorflow/contrib/lite/examples/label_image/label_image.cc \
tensorflow/contrib/lite/examples/label_image/bitmap_helpers.cc 
LABEL_IMAGE_OBJS := $(addprefix $(OBJDIR), \
$(patsubst %.cc,%.o,$(patsubst %.c,%.o,$(LABEL_IMAGE_SRCS))))

# What sources we want to compile, must be kept in sync with the main Bazel
# build files.

CORE_CC_ALL_SRCS := \
$(wildcard tensorflow/contrib/lite/*.cc) \
$(wildcard tensorflow/contrib/lite/kernels/*.cc) \
$(wildcard tensorflow/contrib/lite/kernels/internal/*.cc) \
$(wildcard tensorflow/contrib/lite/kernels/internal/optimized/*.cc) \
$(wildcard tensorflow/contrib/lite/kernels/internal/reference/*.cc) \
$(wildcard tensorflow/contrib/lite/*.c) \
$(wildcard tensorflow/contrib/lite/kernels/*.c) \
$(wildcard tensorflow/contrib/lite/kernels/internal/*.c) \
$(wildcard tensorflow/contrib/lite/kernels/internal/optimized/*.c) \
$(wildcard tensorflow/contrib/lite/kernels/internal/reference/*.c) \
$(wildcard tensorflow/contrib/lite/downloads/farmhash/src/farmhash.cc) \
$(wildcard tensorflow/contrib/lite/downloads/fft2d/fftsg.c)
# Remove any duplicates.
CORE_CC_ALL_SRCS := $(sort $(CORE_CC_ALL_SRCS))
CORE_CC_EXCLUDE_SRCS := \
$(wildcard tensorflow/contrib/lite/*test.cc) \
$(wildcard tensorflow/contrib/lite/*/*test.cc) \
$(wildcard tensorflow/contrib/lite/*/*/*test.cc) \
$(wildcard tensorflow/contrib/lite/*/*/*/*test.cc) \
$(wildcard tensorflow/contrib/lite/kernels/test_util.cc) \
$(MINIMAL_SRCS) \
$(LABEL_IMAGE_SRCS)
# Filter out all the excluded files.
TF_LITE_CC_SRCS := $(filter-out $(CORE_CC_EXCLUDE_SRCS), $(CORE_CC_ALL_SRCS))
# File names of the intermediate files target compilation generates.
TF_LITE_CC_OBJS := $(addprefix $(OBJDIR), \
$(patsubst %.cc,%.o,$(patsubst %.c,%.o,$(TF_LITE_CC_SRCS))))
LIB_OBJS := $(TF_LITE_CC_OBJS)

# For normal manually-created TensorFlow C++ source files.
$(OBJDIR)%.o: %.cc
    @mkdir -p $(dir [email protected])
    $(CXX) $(CXXFLAGS) $(INCLUDES) -c $< -o [email protected]

# For normal manually-created TensorFlow C++ source files.
$(OBJDIR)%.o: %.c
    @mkdir -p $(dir [email protected])
    $(CC) $(CCFLAGS) $(INCLUDES) -c $< -o [email protected]

# The target that's compiled if there's no command-line arguments.
all: $(LIB_PATH)  $(MINIMAL_PATH) $(LABEL_IMAGE_PATH)

# Gathers together all the objects we've compiled into a single '.a' archive.
$(LIB_PATH): $(LIB_OBJS)
    @mkdir -p $(dir [email protected])
    $(AR) $(ARFLAGS) $(LIB_PATH) $(LIB_OBJS)

$(MINIMAL_PATH): $(MINIMAL_OBJS) $(LIB_PATH)
    @mkdir -p $(dir [email protected])
    $(CXX) $(CXXFLAGS) $(INCLUDES) \
    -o $(MINIMAL_PATH) $(MINIMAL_OBJS) \
    $(LIBFLAGS) $(LIB_PATH) $(LDFLAGS) $(LIBS)


$(LABEL_IMAGE_PATH): $(LABEL_IMAGE_OBJS) $(LIB_PATH)
    @mkdir -p $(dir [email protected])
    $(CXX) $(CXXFLAGS) $(INCLUDES) \
    -o $(LABEL_IMAGE_PATH) $(LABEL_IMAGE_OBJS) \
    $(LIBFLAGS) $(LIB_PATH) $(LDFLAGS) $(LIBS)


# Gets rid of all generated files.
clean:
    rm -rf $(MAKEFILE_DIR)/gen

# Gets rid of target files only, leaving the host alone. Also leaves the lib
# directory untouched deliberately, so we can persist multiple architectures
# across builds for iOS and Android.
cleantarget:
    rm -rf $(OBJDIR)
    rm -rf $(BINDIR)

$(DEPDIR)/%.d: ;
.PRECIOUS: $(DEPDIR)/%.d

-include $(patsubst %,$(DEPDIR)/%.d,$(basename $(TF_CC_SRCS)))

修改完成後再次執行 ./tensorflow/contrib/lite/build_rpi_lib.sh ，此時在 ./tensorflow/contrib/lite/gen/bin/rpi_armv8 目錄下會產生編譯好的label_image二進位制檔案。

5.拷貝程式到板子上

準備測試圖片 tensorflow/contrib/lite/examples/label_image/testdata/grace_hopper.bmp ，當然用其他的圖片測試也行。此外，還需要從https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lite/g3doc/models.md 地址下載你想要測試的tf lite模型。還需要準備ImageNet的標籤檔案。最後需要傳到板子上的是如下幾個檔案。

grace_hopper.bmp
label_image
labels.txt
mobilenet_v1_1.0_224_quant.tflite/mobilenet_v1_1.0_224.tflite

6.在ARM板子上執行Tensorflow Lite

到此準備工作全部完成了，最後可以在板子上測試Tensorflow Lite的效能了，使用姿勢如下：

 ./label_image -v 1 -m ./mobilenet_v1_1.0_224.tflite  -i ./grace_hopper.bmp -l ./labels.txt

執行效果如下：

average time: 855.91 ms 
0.860174: 653 military uniform
0.0481021: 907 Windsor tie
0.00786706: 466 bulletproof vest
0.00644932: 514 cornet
0.00608028: 543 drumstick

接下來可以再測試下量化後的MobileNet效果：

./label_image -v 4 -m ./mobilenet_v1_1.0_224_quant.tflite  -i ./grace_hopper.bmp -l ./labels.txt

效果如下：

average time: 185.988 ms 
0.427451: 653 military uniform
0.305882: 907 Windsor tie
0.0431373: 668 mortarboard
0.0313726: 458 bow tie
0.0235294: 543 drumstick

上述Demo執行時預設執行緒數為10，若想測試單執行緒效果，只需再新增-t 1引數即可。單執行緒在板子上的效能效果如下，從結果看TF Lite的多執行緒反而沒起到加速的效果（2 3 4執行緒數量我都試過，這個一個跟TF Lite的執行緒優化有關，二是可能跟ARM板子上的大小核相關，單執行緒情況下預設使用的是大核）

./label_image -v 1 -m ./mobilenet_v1_1.0_224.tflite  -i ./grace_hopper.bmp -l ./labels.txt -t 1 

average time: 297.647 ms 
0.860174: 653 military uniform
0.0481024: 907 Windsor tie
0.00786706: 466 bulletproof vest
0.00644934: 514 cornet
0.00608031: 543 drumstick

./label_image -v 4 -m ./mobilenet_v1_1.0_224_quant.tflite  -i ./grace_hopper.bmp -l ./labels.txt -t 1  

average time: 145.173 ms 
0.427451: 653 military uniform
0.305882: 907 Windsor tie
0.0431373: 668 mortarboard
0.0313726: 458 bow tie
0.0235294: 543 drumstick

從效能上看，如果不考慮識別的準確率的話，TF Lite量化後的效能收益還是很不錯的。在同等平臺下，NCNN跑Mobilenet的耗時比TF Lite的非量化版本略快。