VGG_face Caffe 微調(finetuing)詳細教程(一)
引言
VGG Face Descriptor 是牛津大學VGG小組的工作,現在已經開源訓練好的網路結構和模型引數,本文將基於此模型在caffe上使用其他的人臉資料進行模型微調,並驗證微調結果。
一.額外的人臉資料集和預微調模型
要對訓練好的模型進行微調,首先我們得自己準備額外的人臉資料。這裡我選用IMM人臉資料庫,該人臉資料庫包括了240張人臉圖片,共40個人(7女33男),每人6張人臉圖片,每張人臉圖片被標記了58個特徵點。
下載連線:http://www.imm.dtu.dk/~aam/
可能上面的連線下載速度會過慢,我上傳到了部落格上面供大家下載:
https://download.csdn.net/download/huxiny/10751345
下圖是IMM資料庫的人臉圖:
二.資料處理
IMM資料庫裡的圖片是不能直接用的,這裡我們需要做一下資料處理,將圖片資料轉化為lmdb資料。
將圖片資料轉化為lmdb資料我們需要一個txt檔案,這個txt檔案內容是怎麼樣的? 如下圖所示:
檔案內容格式很簡單,檔名 (空格) 檔案所屬類。
我們這裡擁有40個小夥子,當然就有40個類,不過類的編號要從0開始(這是caffe的規定),也就是0~39。
這個檔案的生成很簡單,先別急,我們等會在關心這個檔案,在此之前我們需要將這240張圖片分配到3個資料夾下面,這3個資料夾分別是train 、test和val。
train資料夾用於存放對模型微調所需要的人臉資料;
test資料夾用於存放微調過程中測試所用的人臉資料;
val資料夾用於存放模型微調結束後用於驗證模型的人臉資料。
我們直接使用下面命令移動圖面就好了:
mv *-6*.jpg test
mv *-5*.jpg val
40個人,每個人有6張照片,我們把所有人的第6張照片放到test資料夾中,同理將第5張照片放到val資料夾中,剩下的放到train資料夾中用於訓練。
接下來就該生成train.txt和test.txt檔案了(不用生成val.txt),這裡我用的是python指令碼生成的,很簡單,但是我還是把程式碼貼一下:
import os def file_name(file_dir): for root, dirs, files in os.walk(file_dir): return files def writeToFile(fileName, lines): with open(fileName, 'w') as f: # 若檔案不存在則會建立一個 f.writelines(lines) lines = [] fileNames = file_name('yourPath/train') for i in range(0, len(fileNames)): num = str(int(fileNames[i].split('-')[0]) - 1) line = fileNames[i] + ' ' + num + '\n' lines.append(line) writeToFile('yourPath/train.txt', lines)
改一下路徑就可以用了。
接下來到我們的這小一步的最終目的,將train資料夾和test資料夾內的人臉資料轉化為lmdb資料。
這裡我們使用的是caffe提供給我們的一個convert_imageset工具,該工具在caffe_root/build/tools/ 目錄下。
這裡寫一個指令碼來完成lmdb的轉化:(如果你不嫌麻煩的話你可以直接呼叫命令來實現)
#!/usr/bin/env sh
MY=/home/pzs/husin/caffePython/husin_download/VGG_face
echo "Create train lmdb..."
rm -rf $MY/img_train_lmdb
/home/pzs/caffe/caffe/build/tools/convert_imageset --shuffle --resize_height=224 --resize_width=224 $MY/train/ $MY/train.txt $MY/img_train_lmdb
echo "Create test lmdb..."
rm -rf $MY/img_test_lmdb
/home/pzs/caffe/caffe/build/tools/convert_imageset --shuffle --resize_height=224 --resize_width=224 $MY/test/ $MY/test.txt $MY/img_test_lmdb
echo "All Done"
同樣的,改一下路徑就可以用了。
這裡說明一下,vgg_face輸入的圖片大小格式為224×224,所以使用resize_height和resize_width引數將圖片大小轉換為224×224。
執行指令碼後,得到兩個資料夾:
每個資料夾內都有兩個檔案:
不過我們不用關心這兩個資料夾裡的內容,我們只需要得到這兩個資料夾就夠了,這裡只是提一下,有興趣的同學可以深入瞭解。
三.生成均值檔案
caffe樣本均值檔案的作用這裡就不說了,百度一下就都明白了,這裡只說明怎麼生成均值檔案。
caffe程式提供了一個計算均值的檔案compute_image_mean.cpp,我們直接使用就可以了:
# sudo build/tools/compute_image_mean yourPath/img_train_lmdb yourPath/mean.binaryproto
當然你可以可寫成指令碼,方便下次使用。
compute_image_mean帶兩個引數,第一個引數是lmdb訓練資料位置,第二個引數設定均值檔案的名字及儲存路徑。
執行成功後,會在 yourPath/ 下面生成一個mean.binaryproto的均值檔案。
四.caffe訓練相關配置檔案修改
接下來我們要建立一個train_test.prototxt檔案,該檔案內容是神經網路結構的描述,你可以自己寫,也可以複製下面的程式碼,注意要改動的地方都標出來了。
name: "VGG_FACE_16_Net"
layer {
name: "data"
type: "Data"
top: "data"
top: "label"
data_param {
source: "$/train_lmdb" # 這裡修改,train資料的lmdb檔案地址
backend:LMDB
batch_size: 100 # 這裡修改,根據自己情況修改,我改為了4
}
transform_param {
mean_file: "$/mean.binaryproto" # 這裡修改,均值檔案地址
mirror: true
}
include: { phase: TRAIN }
}
layer {
name: "data"
type: "Data"
top: "data"
top: "label"
data_param {
source: "$/test_lmdb" # 這裡修改,test資料的lmdb檔案地址
backend:LMDB
batch_size: 25 # 這裡修改,根據自己情況修改,我改為了4
}
transform_param {
mean_file: "$/mean.binaryproto" # 這裡修改,均值檔案地址
mirror: true
}
include: {
phase: TEST
}
}
layer {
name: "conv1_1"
type: "Convolution"
bottom: "data"
top: "conv1_1"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 64
kernel_size: 3
pad: 1
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 0
}
}
}
layer {
name: "relu1_1"
type: "ReLU"
bottom: "conv1_1"
top: "conv1_1"
}
layer {
name: "conv1_2"
type: "Convolution"
bottom: "conv1_1"
top: "conv1_2"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 64
kernel_size: 3
pad: 1
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 0
}
}
}
layer {
name: "relu1_2"
type: "ReLU"
bottom: "conv1_2"
top: "conv1_2"
}
layer {
name: "pool1"
type: "Pooling"
bottom: "conv1_2"
top: "pool1"
pooling_param {
pool: MAX
kernel_size: 2
stride: 2
}
}
layer {
name: "conv2_1"
type: "Convolution"
bottom: "pool1"
top: "conv2_1"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 128
kernel_size: 3
pad: 1
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 0
}
}
}
layer {
name: "relu2_1"
type: "ReLU"
bottom: "conv2_1"
top: "conv2_1"
}
layer {
name: "conv2_2"
type: "Convolution"
bottom: "conv2_1"
top: "conv2_2"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 128
kernel_size: 3
pad: 1
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 0
}
}
}
layer {
name: "relu2_2"
type: "ReLU"
bottom: "conv2_2"
top: "conv2_2"
}
layer {
name: "pool2"
type: "Pooling"
bottom: "conv2_2"
top: "pool2"
pooling_param {
pool: MAX
kernel_size: 2
stride: 2
}
}
layer {
name: "conv3_1"
type: "Convolution"
bottom: "pool2"
top: "conv3_1"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 256
kernel_size: 3
pad: 1
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 0
}
}
}
layer {
name: "relu3_1"
type: "ReLU"
bottom: "conv3_1"
top: "conv3_1"
}
layer {
name: "conv3_2"
type: "Convolution"
bottom: "conv3_1"
top: "conv3_2"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 256
kernel_size: 3
pad: 1
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 0
}
}
}
layer {
name: "relu3_2"
type: "ReLU"
bottom: "conv3_2"
top: "conv3_2"
}
layer {
name: "conv3_3"
type: "Convolution"
bottom: "conv3_2"
top: "conv3_3"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 256
kernel_size: 3
pad: 1
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 0
}
}
}
layer {
name: "relu3_3"
type: "ReLU"
bottom: "conv3_3"
top: "conv3_3"
}
layer {
name: "pool3"
type: "Pooling"
bottom: "conv3_3"
top: "pool3"
pooling_param {
pool: MAX
kernel_size: 2
stride: 2
}
}
layer {
name: "conv4_1"
type: "Convolution"
bottom: "pool3"
top: "conv4_1"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 512
kernel_size: 3
pad: 1
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 0
}
}
}
layer {
name: "relu4_1"
type: "ReLU"
bottom: "conv4_1"
top: "conv4_1"
}
layer {
name: "conv4_2"
type: "Convolution"
bottom: "conv4_1"
top: "conv4_2"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 512
kernel_size: 3
pad: 1
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 0
}
}
}
layer {
name: "relu4_2"
type: "ReLU"
bottom: "conv4_2"
top: "conv4_2"
}
layer {
name: "conv4_3"
type: "Convolution"
bottom: "conv4_2"
top: "conv4_3"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 512
kernel_size: 3
pad: 1
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 0
}
}
}
layer {
name: "relu4_3"
type: "ReLU"
bottom: "conv4_3"
top: "conv4_3"
}
layer {
name: "pool4"
type: "Pooling"
bottom: "conv4_3"
top: "pool4"
pooling_param {
pool: MAX
kernel_size: 2
stride: 2
}
}
layer {
name: "conv5_1"
type: "Convolution"
bottom: "pool4"
top: "conv5_1"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 512
kernel_size: 3
pad: 1
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 0
}
}
}
layer {
name: "relu5_1"
type: "ReLU"
bottom: "conv5_1"
top: "conv5_1"
}
layer {
name: "conv5_2"
type: "Convolution"
bottom: "conv5_1"
top: "conv5_2"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 512
kernel_size: 3
pad: 1
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 0
}
}
}
layer {
name: "relu5_2"
type: "ReLU"
bottom: "conv5_2"
top: "conv5_2"
}
layer {
name: "conv5_3"
type: "Convolution"
bottom: "conv5_2"
top: "conv5_3"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 512
kernel_size: 3
pad: 1
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 0
}
}
}
layer {
name: "relu5_3"
type: "ReLU"
bottom: "conv5_3"
top: "conv5_3"
}
layer {
name: "pool5"
type: "Pooling"
bottom: "conv5_3"
top: "pool5"
pooling_param {
pool: MAX
kernel_size: 2
stride: 2
}
}
layer {
name: "fc6"
type: "InnerProduct"
bottom: "pool5"
top: "fc6"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
inner_product_param {
num_output: 4096
weight_filler {
type: "gaussian"
std: 0.005
}
bias_filler {
type: "constant"
value: 1
}
}
}
layer {
name: "relu6"
type: "ReLU"
bottom: "fc6"
top: "fc6"
}
layer {
name: "drop6"
type: "Dropout"
bottom: "fc6"
top: "fc6"
dropout_param {
dropout_ratio: 0.5
}
}
layer {
name: "fc7"
type: "InnerProduct"
bottom: "fc6"
top: "fc7"
# Note that lr_mult can be set to 0 to disable any fine-tuning of this, and any other, layer
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
inner_product_param {
num_output: 4096
weight_filler {
type: "gaussian"
std: 0.005
}
bias_filler {
type: "constant"
value: 1
}
}
}
layer {
name: "relu7"
type: "ReLU"
bottom: "fc7"
top: "fc7"
}
layer {
name: "drop7"
type: "Dropout"
bottom: "fc7"
top: "fc7"
dropout_param {
dropout_ratio: 0.5
}
}
layer {
name: "fc8_flickr"
type: "InnerProduct"
bottom: "fc7"
top: "fc8_flickr"
# lr_mult is set to higher than for other layers, because this layer is starting from random while the others are already trained
propagate_down: false
inner_product_param {
num_output: 356 #這裡修改,改為對應的分類數,有40人,應改為40
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 0
}
}
}
layer {
name: "accuracy"
type: "Accuracy"
bottom: "fc8_flickr"
bottom: "label"
top: "accuracy"
include {
phase: TEST
}
}
layer {
name: "loss"
type: "SoftmaxWithLoss"
bottom: "fc8_flickr"
bottom: "label"
top: "loss"
}
接著我們還要建立一個solver.prototxt檔案,該檔案內容是整個神經網路的一些超引數:
net: "/home/pzs/husin/caffePython/husin_download/VGG_face/train_test.prototxt" # 改
test_iter: 40
test_interval: 10
# lr for fine-tuning should be lower than when starting from scratch
base_lr: 0.001
lr_policy: "step"
gamma: 0.1
# stepsize should also be lower, as we're closer to being done
stepsize: 20000
display: 20
max_iter: 1000
momentum: 0.9
weight_decay: 0.0005
snapshot: 20
snapshot_prefix: "/home/pzs/husin/caffePython/husin_download/VGG_face/snapshot" # 改
# uncomment the following to default to CPU mode solving
solver_mode: CPU
這個檔案沒什麼要改的了(但路徑還是要該的),直接複製就好了,若瞭解引數的含義可以自行按照自己的需求優化調整。
五.訓練
在訓練之前,我們先建立一個snapshot資料夾,該資料夾儲存訓練過程中的caffemodel檔案,solver.prototxt檔案中的snapshot_prefix項指定路徑,snapshot:20表示迭代20次儲存一次caffemodel。
開始訓練:
caffe_root/build/tools/caffe train -solver yourPath/solver.prototxt -weights yourPath/VGG_FACE.caffemodel
執行上面命令就可以開始訓練了,使用的是caffe自帶的工具來執行的,solver引數後面跟著solver檔案地址,weights引數後面跟著需要進行微調的caffemodel。
訓練結果:
這裡我用單核CPU迭代了1000次,花費了近8個小時,所以有GPU就用GPU吧。
精度竟然達到了100%,你敢信?
接下來我還會寫一篇驗證模型的文章,介紹如何使用訓練好的caffe模型。
結束語
該文章是本人對caffe學習的階段總結,有錯誤之處還請大家指出,以共同學習。
And let us not be weary in well-doing, for in due season, we shall reap, if we faint not