FCN訓練自己資料集（person-segmentation）、SIFT-FLOW、SBD和VOC實驗總結
阿新 • • 發佈：2019-01-02
最近花了將近一週的時間，基於提供的原始碼，通過參考網上的部落格，跑通了FCN在三個資料集上的訓練以及測試。在這裡寫下總結，即是記錄，又希望能夠對其他剛剛接觸FCN的人有所幫助。
FCN的原始碼地址：https://github.com/shelhamer/fcn.berkeleyvision.org
訓練自己的資料集：
1.首先準備自己的資料集：
我的資料集是對人體輪廓進行分割。
最開始的標籤是這樣的：
但是這個標籤是有問題的，最後會說明。
2.對原始碼進行修改
將工程所要用到的所有py檔案放在一個目錄下，否則會報錯。
首先是solve.py檔案
#!usr/bin/python # -*- coding: utf-8 -*- import caffe import surgery, score import numpy as np import os import sys sys.path.append('/home/panyun/sourcecode/caffe/python') #引入sys庫並且增加需要的caffe的python路徑 try: import setproctitle setproctitle.setproctitle(os.path.basename(os.getcwd())) except: pass # init # caffe.set_device(int(sys.argv[0])) # 這裡設定使用GPU編號 先註釋掉 caffe.set_device(int(0)) caffe.set_mode_gpu() solver = caffe.SGDSolver('/home/zhaoys/myf/fcn/people-fcn8s/solver.prototxt') # 設定solver.prototxt weights = '/home/zhaoys/myf/fcn/people-fcn8s/caffemodel/fcn8s-heavy-pascal.caffemodel' # 設定下載好的用來finetune的模型 deploy_proto = '/home/zhaoys/myf/fcn/people-fcn8s/deploy.prototxt' # solver.net.copy_from(weights) fcn8snet = caffe.Net(deploy_proto, weights, caffe.TRAIN) surgery.transplant(solver.net, fcn8snet) del fcn8snet # surgeries interp_layers = [k for k in solver.net.params.keys() if 'up' in k] #修改層名稱的時候不要把反捲積層的name中的"up"去掉 surgery.interp(solver.net, interp_layers) # scoring val = np.loadtxt('/home/zhaoys/myf/dataset/people_segmentation/test.txt', dtype=str) # 傳入已經寫好的訓練txt檔案 for _ in range(25): # 設定epoch數和每個epoch中的step數 solver.step(4000) score.seg_tests(solver, False, val, layer='score')
首先，在程式碼裡面對路徑進行了修改。其次，在fcn8s的模型上面進行了fine-tune，由於網路模型和我自己資料集輸入的圖片大小不同，這裡採用了移植的方式。最後，設定epoch數和每個epoch中進行迭代的次數。
train_net: "/home/zhaoys/myf/fcn/people-fcn8s/train.prototxt" test_net: "/home/zhaoys/myf/fcn/people-fcn8s/val.prototxt" test_iter: 736 # make test net, but don't invoke it from the solver itself test_interval: 999999999 display: 40 average_loss: 20 lr_policy: "fixed" # lr for unnormalized softmax base_lr: 1e-12 # high momentum momentum: 0.99 # no gradient accumulation iter_size: 1 max_iter: 100000 weight_decay: 0.0005 snapshot: 4000 snapshot_prefix: "/home/zhaoys/myf/fcn/people-fcn8s/snapshot/train" test_initialization: false debug_info: false
solve.prototxt檔案設定。
layer {
name: "data"
type: "Python"
top: "data"
top: "label"
python_param {
module: "voc_layers"
layer: "SBDDSegDataLayer"
param_str: "{\'sbdd_dir\': \'/home/zhaoys/myf/dataset/people_segmentation\', \'seed\': 1337, \'split\': \'train\', \'mean\': (184.17504, 193.82224, 204.57951)}"
}
}
layer {
name: "conv1_1"
type: "Convolution"
bottom: "data"
top: "conv1_1"
param {
lr_mult: 1.0
decay_mult: 1.0
}
param {
lr_mult: 2.0
decay_mult: 0.0
}
convolution_param {
num_output: 64
pad: 100
kernel_size: 3
stride: 1
}
}
layer {
name: "relu1_1"
type: "ReLU"
bottom: "conv1_1"
top: "conv1_1"
}
layer {
name: "conv1_2"
type: "Convolution"
bottom: "conv1_1"
top: "conv1_2"
param {
lr_mult: 1.0
decay_mult: 1.0
}
param {
lr_mult: 2.0
decay_mult: 0.0
}
convolution_param {
num_output: 64
pad: 1
kernel_size: 3
stride: 1
}
}
layer {
name: "relu1_2"
type: "ReLU"
bottom: "conv1_2"
top: "conv1_2"
}
layer {
name: "pool1"
type: "Pooling"
bottom: "conv1_2"
top: "pool1"
pooling_param {
pool: MAX
kernel_size: 2
stride: 2
}
}
layer {
name: "conv2_1"
type: "Convolution"
bottom: "pool1"
top: "conv2_1"
param {
lr_mult: 1.0
decay_mult: 1.0
}
param {
lr_mult: 2.0
decay_mult: 0.0
}
convolution_param {
num_output: 128
pad: 1
kernel_size: 3
stride: 1
}
}
layer {
name: "relu2_1"
type: "ReLU"
bottom: "conv2_1"
top: "conv2_1"
}
layer {
name: "conv2_2"
type: "Convolution"
bottom: "conv2_1"
top: "conv2_2"
param {
lr_mult: 1.0
decay_mult: 1.0
}
param {
lr_mult: 2.0
decay_mult: 0.0
}
convolution_param {
num_output: 128
pad: 1
kernel_size: 3
stride: 1
}
}
layer {
name: "relu2_2"
type: "ReLU"
bottom: "conv2_2"
top: "conv2_2"
}
layer {
name: "pool2"
type: "Pooling"
bottom: "conv2_2"
top: "pool2"
pooling_param {
pool: MAX
kernel_size: 2
stride: 2
}
}
layer {
name: "conv3_1"
type: "Convolution"
bottom: "pool2"
top: "conv3_1"
param {
lr_mult: 1.0
decay_mult: 1.0
}
param {
lr_mult: 2.0
decay_mult: 0.0
}
convolution_param {
num_output: 256
pad: 1
kernel_size: 3
stride: 1
}
}
layer {
name: "relu3_1"
type: "ReLU"
bottom: "conv3_1"
top: "conv3_1"
}
layer {
name: "conv3_2"
type: "Convolution"
bottom: "conv3_1"
top: "conv3_2"
param {
lr_mult: 1.0
decay_mult: 1.0
}
param {
lr_mult: 2.0
decay_mult: 0.0
}
convolution_param {
num_output: 256
pad: 1
kernel_size: 3
stride: 1
}
}
layer {
name: "relu3_2"
type: "ReLU"
bottom: "conv3_2"
top: "conv3_2"
}
layer {
name: "conv3_3"
type: "Convolution"
bottom: "conv3_2"
top: "conv3_3"
param {
lr_mult: 1.0
decay_mult: 1.0
}
param {
lr_mult: 2.0
decay_mult: 0.0
}
convolution_param {
num_output: 256
pad: 1
kernel_size: 3
stride: 1
}
}
layer {
name: "relu3_3"
type: "ReLU"
bottom: "conv3_3"
top: "conv3_3"
}
layer {
name: "pool3"
type: "Pooling"
bottom: "conv3_3"
top: "pool3"
pooling_param {
pool: MAX
kernel_size: 2
stride: 2
}
}
layer {
name: "conv4_1"
type: "Convolution"
bottom: "pool3"
top: "conv4_1"
param {
lr_mult: 1.0
decay_mult: 1.0
}
param {
lr_mult: 2.0
decay_mult: 0.0
}
convolution_param {
num_output: 512
pad: 1
kernel_size: 3
stride: 1
}
}
layer {
name: "relu4_1"
type: "ReLU"
bottom: "conv4_1"
top: "conv4_1"
}
layer {
name: "conv4_2"
type: "Convolution"
bottom: "conv4_1"
top: "conv4_2"
param {
lr_mult: 1.0
decay_mult: 1.0
}
param {
lr_mult: 2.0
decay_mult: 0.0
}
convolution_param {
num_output: 512
pad: 1
kernel_size: 3
stride: 1
}
}
layer {
name: "relu4_2"
type: "ReLU"
bottom: "conv4_2"
top: "conv4_2"
}
layer {
name: "conv4_3"
type: "Convolution"
bottom: "conv4_2"
top: "conv4_3"
param {
lr_mult: 1.0
decay_mult: 1.0
}
param {
lr_mult: 2.0
decay_mult: 0.0
}
convolution_param {
num_output: 512
pad: 1
kernel_size: 3
stride: 1
}
}
layer {
name: "relu4_3"
type: "ReLU"
bottom: "conv4_3"
top: "conv4_3"
}
layer {
name: "pool4"
type: "Pooling"
bottom: "conv4_3"
top: "pool4"
pooling_param {
pool: MAX
kernel_size: 2
stride: 2
}
}
layer {
name: "conv5_1"
type: "Convolution"
bottom: "pool4"
top: "conv5_1"
param {
lr_mult: 1.0
decay_mult: 1.0
}
param {
lr_mult: 2.0
decay_mult: 0.0
}
convolution_param {
num_output: 512
pad: 1
kernel_size: 3
stride: 1
}
}
layer {
name: "relu5_1"
type: "ReLU"
bottom: "conv5_1"
top: "conv5_1"
}
layer {
name: "conv5_2"
type: "Convolution"
bottom: "conv5_1"
top: "conv5_2"
param {
lr_mult: 1.0
decay_mult: 1.0
}
param {
lr_mult: 2.0
decay_mult: 0.0
}
convolution_param {
num_output: 512
pad: 1
kernel_size: 3
stride: 1
}
}
layer {
name: "relu5_2"
type: "ReLU"
bottom: "conv5_2"
top: "conv5_2"
}
layer {
name: "conv5_3"
type: "Convolution"
bottom: "conv5_2"
top: "conv5_3"
param {
lr_mult: 1.0
decay_mult: 1.0
}
param {
lr_mult: 2.0
decay_mult: 0.0
}
convolution_param {
num_output: 512
pad: 1
kernel_size: 3
stride: 1
}
}
layer {
name: "relu5_3"
type: "ReLU"
bottom: "conv5_3"
top: "conv5_3"
}
layer {
name: "pool5"
type: "Pooling"
bottom: "conv5_3"
top: "pool5"
pooling_param {
pool: MAX
kernel_size: 2
stride: 2
}
}
layer {
name: "fc6"
type: "Convolution"
bottom: "pool5"
top: "fc6"
param {
lr_mult: 1.0
decay_mult: 1.0
}
param {
lr_mult: 2.0
decay_mult: 0.0
}
convolution_param {
num_output: 4096
pad: 0
kernel_size: 7
stride: 1
}
}
layer {
name: "relu6"
type: "ReLU"
bottom: "fc6"
top: "fc6"
}
layer {
name: "drop6"
type: "Dropout"
bottom: "fc6"
top: "fc6"
dropout_param {
dropout_ratio: 0.5
}
}
layer {
name: "fc7"
type: "Convolution"
bottom: "fc6"
top: "fc7"
param {
lr_mult: 1.0
decay_mult: 1.0
}
param {
lr_mult: 2.0
decay_mult: 0.0
}
convolution_param {
num_output: 4096
pad: 0
kernel_size: 1
stride: 1
}
}
layer {
name: "relu7"
type: "ReLU"
bottom: "fc7"
top: "fc7"
}
layer {
name: "drop7"
type: "Dropout"
bottom: "fc7"
top: "fc7"
dropout_param {
dropout_ratio: 0.5
}
}
layer {
name: "score_fr_n"
type: "Convolution"
bottom: "fc7"
top: "score_fr"
param {
lr_mult: 1.0
decay_mult: 1.0
}
param {
lr_mult: 2.0
decay_mult: 0.0
}
convolution_param {
num_output: 2
pad: 0
kernel_size: 1
}
}
layer {
name: "upscore2_n"
type: "Deconvolution"
bottom: "score_fr"
top: "upscore2"
param {
lr_mult: 0.0
}
convolution_param {
num_output: 2
bias_term: false
kernel_size: 4
stride: 2
}
}
layer {
name: "score_pool4_n"
type: "Convolution"
bottom: "pool4"
top: "score_pool4"
param {
lr_mult: 1.0
decay_mult: 1.0
}
param {
lr_mult: 2.0
decay_mult: 0.0
}
convolution_param {
num_output: 2
pad: 0
kernel_size: 1
}
}
layer {
name: "score_pool4c"
type: "Crop"
bottom: "score_pool4"
bottom: "upscore2"
top: "score_pool4c"
crop_param {
axis: 2
offset: 5
}
}
layer {
name: "fuse_pool4"
type: "Eltwise"
bottom: "upscore2"
bottom: "score_pool4c"
top: "fuse_pool4"
eltwise_param {
operation: SUM
}
}
layer {
name: "upscore_pool4_n"
type: "Deconvolution"
bottom: "fuse_pool4"
top: "upscore_pool4"
param {
lr_mult: 0.0
}
convolution_param {
num_output: 2
bias_term: false
kernel_size: 4
stride: 2
}
}
layer {
name: "score_pool3_n"
type: "Convolution"
bottom: "pool3"
top: "score_pool3"
param {
lr_mult: 1.0
decay_mult: 1.0
}
param {
lr_mult: 2.0
decay_mult: 0.0
}
convolution_param {
num_output: 2
pad: 0
kernel_size: 1
}
}
layer {
name: "score_pool3c"
type: "Crop"
bottom: "score_pool3"
bottom: "upscore_pool4"
top: "score_pool3c"
crop_param {
axis: 2
offset: 9
}
}
layer {
name: "fuse_pool3"
type: "Eltwise"
bottom: "upscore_pool4"
bottom: "score_pool3c"
top: "fuse_pool3"
eltwise_param {
operation: SUM
}
}
layer {
name: "upscore8_n"
type: "Deconvolution"
bottom: "fuse_pool3"
top: "upscore8"
param {
lr_mult: 0.0
}
convolution_param {
num_output: 2
bias_term: false
kernel_size: 16
stride: 8
}
}
layer {
name: "score"
type: "Crop"
bottom: "upscore8"
bottom: "data"
top: "score"
crop_param {
axis: 2
offset: 31
}
}
layer {
name: "loss"
type: "SoftmaxWithLoss"
bottom: "score"
bottom: "label"
top: "loss"
loss_param {
normalize: false
}
}
train.prototxt檔案設定。
裡面進行的改動主要有：1，輸入資料層的改動。2.out_num由原來的21改為2，由於筆者進行的是二分類。3.將2中對應層的name屬性最後加"_n"，不然會和solve.py中移植過來的引數衝突。4.將loss層中的parameter中的忽略label值刪掉。因為原來使用的voc資料集中的mask中有白邊。並且筆者最後的標籤中的畫素值也並沒有255。只有0，1。
要說明的一點是：nomalize是進行正則化，我設定為了預設的false，這樣的話訓練出來的loss值非常的大，4，5位數是常有的事，不過筆者認為這樣是正常的。因為測試出來的指標如mIOU比較不錯，還是以指標為準。後來我將其改為true，發現loss為很小的值，但是指標卻不太好。
test.prototxt中設定和這個類似。
import caffe
import numpy as np
from PIL import Image
import random
import cv2
class VOCSegDataLayer(caffe.Layer):
"""
Load (input image, label image) pairs from PASCAL VOC
one-at-a-time while reshaping the net to preserve dimensions.
Use this to feed data to a fully convolutional network.
"""
def setup(self, bottom, top):
"""
Setup data layer according to parameters:
- voc_dir: path to PASCAL VOC year dir
- split: train / val / test
- mean: tuple of mean values to subtract
- randomize: load in random order (default: True)
- seed: seed for randomization (default: None / current time)
for PASCAL VOC semantic segmentation.
example
params = dict(voc_dir="/path/to/PASCAL/VOC2011",
mean=(104.00698793, 116.66876762, 122.67891434),
split="val")
"""
# config
params = eval(self.param_str)
self.voc_dir = params['voc_dir']
self.split = params['split']
self.mean = np.array(params['mean'])
self.random = params.get('randomize', True)
self.seed = params.get('seed', None)
# two tops: data and label
if len(top) != 2:
raise Exception("Need to define two tops: data and label.")
# data layers have no bottoms
if len(bottom) != 0:
raise Exception("Do not define a bottom.")
# load indices for images and labels
split_f = '{}/{}.txt'.format(self.voc_dir,
self.split)
self.indices = open(split_f, 'r').read().splitlines()
self.idx = 0
# make eval deterministic
if 'train' not in self.split:
self.random = False
# randomization: seed and pick
if self.random:
random.seed(self.seed)
self.idx = random.randint(0, len(self.indices)-1)
def reshape(self, bottom, top):
# load image + label image pair
self.data = self.load_image(self.indices[self.idx])
self.label = self.load_label(self.indices[self.idx])
# reshape tops to fit (leading 1 is for batch dimension)
top[0].reshape(1, *self.data.shape)
top[1].reshape(1, *self.label.shape)
def forward(self, bottom, top):
# assign output
top[0].data[...] = self.data
top[1].data[...] = self.label
# pick next input
if self.random:
self.idx = random.randint(0, len(self.indices)-1)
else:
self.idx += 1
if self.idx == len(self.indices):
self.idx = 0
def backward(self, top, propagate_down, bottom):
pass
def load_image(self, idx):
"""
Load input image and preprocess for Caffe:
- cast to float
- switch channels RGB -> BGR
- subtract mean
- transpose to channel x height x width order
"""
im = Image.open('{}/images/{}.jpg'.format(self.voc_dir, idx))
im = im.resize((500, 500))
in_ = np.array(im, dtype=np.float32)
in_ = in_[:,:,::-1]
in_ -= self.mean
in_ = in_.transpose((2,0,1))
return in_
def load_label(self, idx):
"""
Load label image as 1 x height x width integer array of label indices.
The leading singleton dimension is required by the loss.
"""
im = Image.open('{}/masks/{}-profile.png'.format(self.voc_dir, idx))
im = im.resize((500, 500))
label = np.array(im, dtype=np.uint8)
label = label[np.newaxis, ...]
return label
class SBDDSegDataLayer(caffe.Layer):
"""
Load (input image, label image) pairs from the SBDD extended labeling
of PASCAL VOC for semantic segmentation
one-at-a-time while reshaping the net to preserve dimensions.
Use this to feed data to a fully convolutional network.
"""
def setup(self, bottom, top):
"""
Setup data layer according to parameters:
- sbdd_dir: path to SBDD `dataset` dir
- split: train / seg11valid
- mean: tuple of mean values to subtract
- randomize: load in random order (default: True)
- seed: seed for randomization (default: None / current time)
for SBDD semantic segmentation.
N.B.segv11alid is the set of segval11 that does not intersect with SBDD.
Find it here: https://gist.github.com/shelhamer/edb330760338892d511e.
example
params = dict(sbdd_dir="/path/to/SBDD/dataset",
mean=(104.00698793, 116.66876762, 122.67891434),
split="valid")
"""
# config
params = eval(self.param_str)
self.sbdd_dir = params['sbdd_dir']
self.split = params['split']
self.mean = np.array(params['mean'])
self.random = params.get('randomize', True)
self.seed = params.get('seed', None)
# two tops: data and label
if len(top) != 2:
raise Exception("Need to define two tops: data and label.")
# data layers have no bottoms
if len(bottom) != 0:
raise Exception("Do not define a bottom.")
# load indices for images and labels
split_f = '{}/{}.txt'.format(self.sbdd_dir,
self.split)
self.indices = open(split_f, 'r').read().splitlines()
self.idx = 0
# make eval deterministic
if 'train' not in self.split:
self.random = False
# randomization: seed and pick
if self.random:
random.seed(self.seed)
self.idx = random.randint(0, len(self.indices)-1)
def reshape(self, bottom, top):
# load image + label image pair
self.data = self.load_image(self.indices[self.idx])
self.label = self.load_label(self.indices[self.idx])
# reshape tops to fit (leading 1 is for batch dimension)
top[0].reshape(1, *self.data.shape)
top[1].reshape(1, *self.label.shape)
def forward(self, bottom, top):
# assign output
top[0].data[...] = self.data
top[1].data[...] = self.label
# pick next input
if self.random:
self.idx = random.randint(0, len(self.indices)-1)
else:
self.idx += 1
if self.idx == len(self.indices):
self.idx = 0
def backward(self, top, propagate_down, bottom):
pass
def load_image(self, idx):
"""
Load input image and preprocess for Caffe:
- cast to float
- switch channels RGB -> BGR
- subtract mean
- transpose to channel x height x width order
"""
im = Image.open('{}/images/{}.jpg'.format(self.sbdd_dir, idx))
im = im.resize((500, 500))
in_ = np.array(im, dtype=np.float32)
in_ = in_[:,:,::-1]
in_ -= self.mean
in_ = in_.transpose((2,0,1))
return in_
def load_label(self, idx):
"""
Load label image as 1 x height x width integer array of label indices.
The leading singleton dimension is required by the loss.
"""
im = Image.open('{}/masks/{}-profile.png'.format(self.sbdd_dir, idx))
im = im.resize((500, 500))
label = np.array(im, dtype=np.uint8)
label = label[np.newaxis, ...]
return label
這裡對voc_layers.py資料層進行了修改。
需要說明的是：最好使用原始碼提供的python型別的資料層，不要將資料集改為LMDB格式進行訓練，因為這樣會漏掉python函式中一些比較必要的功能函式，導致最後訓練不出來。拿到自己的資料集，將其套用進fcn8s例子中的訓練、測試層即可。
這裡筆者的訓練集是jpg格式的，訓練集是png格式的。
對輸入的影象進行了resize，是因為，輸入的影象的大小超出了caffe最終blob的限制。
至此已經基本完成了檔案的配置。
cd到檔案目錄下進行訓練，輸入python solve.py進行訓練。
出現了下面的報錯。
產生這個問題的原因是標籤的值和我們輸出的類別不匹配，即不為2。
我們的資料集明明是黑白的，為什麼不為2呢？
於是就對資料集進行了考察。
from PIL import Image
import numpy as np
import matplotlib.pyplot as plt
im = Image.open('/home/zhaoys/myf/dataset/people_segmentation/masks/00001-profile.png')
img = np.array(im)
print img
print img.size
print img.dtype
print img.shape
print img[540, 360]
print np.unique(im)
print im
列印圖片中所有不同畫素值的種類輸出的結果為：
[ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35
36 37 38 39 217 218 219 220 221 222 224 225 226 227 228 229 230 231
232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249
250 251 252 253 254 255]為什麼有這麼多種的畫素值？不管了先進行二值化處理，變為2類吧。執行下面的指令碼檔案。
import os
import cv2
path = "/home/zhaoys/myf/dataset/people_segmentation/bad_masks"
os.chdir(path)
imgList = os.listdir(path)
for img in imgList:
img_name = img
print str(img_name)
img = cv2.imread(img)
img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
ret, img = cv2.threshold(img, 127, 255, cv2.THRESH_BINARY_INV)
ret, img = cv2.threshold(img, 127, 1, cv2.THRESH_BINARY)
cv2.imwrite("/home/zhaoys/myf/dataset/people_segmentation/masks/" + img_name[0:-3] + "png", img)
print ("done!")
這樣就可以了。所有正常的標籤圖片都儲存在了masks裡面。
這裡需要說明的是，千萬不能儲存為比較坑的.jpg檔案格式，因為這種格式的圖片檔案是有失真壓縮的，就算進行二值化，還是會有其他的畫素值。這裡儲存為了.png格式。
再次執行上上面的指令碼檔案，輸出結果為[0,1]，此時圖片中只有兩種畫素型別，即0和1，這樣就滿足了二分類標籤的要求。
另外讀取圖片的時候要通過灰度圖讀取進來，這樣最後生成的圖片是8位的，不然就是24位的，這樣就跟網路中的引數不匹配了。
最後筆者產生的資料集中的標籤圖片是這樣的。
沒錯，看起來全部是黑的。因為灰度是0和1。
圖片的位寬是8位的。
執行下面的指令碼檔案轉化為索引圖。
from random import randint
from PIL import Image
import os
import cv2
path = "/home/zhaoys/myf/dataset/people_segmentation/masks"
os.chdir(path)
imgList = os.listdir(path)
for pic in imgList:
img = Image.open(pic)
img.putpalette([0,0,0,255,0,0])
img.save(pic)
print ("done!")得到的圖片是這樣的。
3.進行最終的訓練
訓練是基於上面全黑的masks訓練的，不過通過索引圖訓練應該也是可以的。
測試的指標如下：
到這裡已經訓練成功了。
測試的infer檔案。
import numpy as np
from PIL import Image
import caffe
import vis
import cv2
# the demo image is "2007_000129" from PASCAL VOC
# load image, switch to BGR, subtract mean, and make dims C x H x W for Caffe
im = Image.open('/home/zhaoys/myf/fcn/people-fcn8s/demo/image.jpg')
in_ = np.array(im, dtype=np.float32)
in_ = in_[:,:,::-1]
in_ -= np.array((184.17504, 193.82224, 204.57951))
in_ = in_.transpose((2,0,1))
# load net
net = caffe.Net('/home/zhaoys/myf/fcn/people-fcn8s/infer.prototxt', '/home/zhaoys/myf/fcn/people-fcn8s/snapshot/train_iter_4000.caffemodel', caffe.TEST)
# shape for input (data blob is N x C x H x W), set data
net.blobs['data'].reshape(1, *in_.shape)
net.blobs['data'].data[...] = in_
# run net and take argmax for prediction
net.forward()
'''
out = net.blobs['score'].data[0].argmax(axis=0)
image_0 = net.blobs['data'].data[0,0]
image_1 = net.blobs['conv3_1'].data[0,0]
image_2 = net.blobs['conv5_3'].data[0,1]
image_3 = net.blobs['fuse_pool4'].data[0,1]
cv2.imshow('1', out)
cv2.imshow('2', image_0)
cv2.imshow('3', image_1)
cv2.imshow('4', image_2)
cv2.imshow('5', image_3)
cv2.waitKey(0)
img = np.array(out)
print img
print img.size
print img.dtype
print img.shape
print img[540, 360]
print np.unique(img)
'''
# visualize segmentation in PASCAL VOC colors
voc_palette = vis.make_palette(2)
out_im = Image.fromarray(vis.color_seg(out, voc_palette))
out_im.save('/home/zhaoys/myf/fcn/people-fcn8s/demo/output.png')
masked_im = Image.fromarray(vis.vis_seg(im, out, voc_palette))
masked_im.save('/home/zhaoys/myf/fcn/people-fcn8s/demo/visualization.jpg')
測試的infer.prototxt網路結構檔案。
layer {
name: "input"
type: "Input"
top: "data"
input_param {
shape { dim: 1 dim: 3 dim: 500 dim: 500 }
}
}
layer {
name: "conv1_1"
type: "Convolution"
bottom: "data"
top: "conv1_1"
param {
lr_mult: 1.0
decay_mult: 1.0
}
param {
lr_mult: 2.0
decay_mult: 0.0
}
convolution_param {
num_output: 64
pad: 100
kernel_size: 3
stride: 1
}
}
layer {
name: "relu1_1"
type: "ReLU"
bottom: "conv1_1"
top: "conv1_1"
}
layer {
name: "conv1_2"
type: "Convolution"
bottom: "conv1_1"
top: "conv1_2"
param {
lr_mult: 1.0
decay_mult: 1.0
}
param {
lr_mult: 2.0
decay_mult: 0.0
}
convolution_param {
num_output: 64
pad: 1
kernel_size: 3
stride: 1
}
}
layer {
name: "relu1_2"
type: "ReLU"
bottom: "conv1_2"
top: "conv1_2"
}
layer {
name: "pool1"
type: "Pooling"
bottom: "conv1_2"
top: "pool1"
pooling_param {
pool: MAX
kernel_size: 2
stride: 2
}
}
layer {
name: "conv2_1"
type: "Convolution"
bottom: "pool1"
top: "conv2_1"
param {
lr_mult: 1.0
decay_mult: 1.0
}
param {
lr_mult: 2.0
decay_mult: 0.0
}
convolution_param {
num_output: 128
pad: 1
kernel_size: 3
stride: 1
}
}
layer {
name: "relu2_1"
type: "ReLU"
bottom: "conv2_1"
top: "conv2_1"
}
layer {
name: "conv2_2"
type: "Convolution"
bottom: "conv2_1"
top: "conv2_2"
param {
lr_mult: 1.0
decay_mult: 1.0
}
param {
lr_mult: 2.0
decay_mult: 0.0
}
convolution_param {
num_output: 128
pad: 1
kernel_size: 3
stride: 1
}
}
layer {
name: "relu2_2"
type: "ReLU"
bottom: "conv2_2"
top: "conv2_2"
}
layer {
name: "pool2"
type: "Pooling"
bottom: "conv2_2"
top: "pool2"
pooling_param {
pool: MAX
kernel_size: 2
stride: 2
}
}
layer {
name: "conv3_1"
type: "Convolution"
bottom: "pool2"
top: "conv3_1"
param {
lr_mult: 1.0
decay_mult: 1.0
}
param {
lr_mult: 2.0
decay_mult: 0.0
}
convolution_param {
num_output: 256
pad: 1
kernel_size: 3
stride: 1
}
}
layer {
name: "relu3_1"
type: "ReLU"
bottom: "conv3_1"
top: "conv3_1"
}
layer {
name: "conv3_2"
type: "Convolution"
bottom: "conv3_1"
top: "conv3_2"
param {
lr_mult: 1.0
decay_mult: 1.0
}
param {
lr_mult: 2.0
decay_mult: 0.0
}
convolution_param {
num_output: 256
pad: 1
kernel_size: 3
stride: 1
}
}
layer {
name: "relu3_2"
type: "ReLU"
bottom: "conv3_2"
top: "conv3_2"
}
layer {
name: "conv3_3"
type: "Convolution"
bottom: "conv3_2"
top: "conv3_3"
param {
lr_mult: 1.0
decay_mult: 1.0
}
param {
lr_mult: 2.0
decay_mult: 0.0
}
convolution_param {
num_output: 256
pad: 1
kernel_size: 3
stride: 1
}
}
layer {
name: "relu3_3"
type: "ReLU"
bottom: "conv3_3"
top: "conv3_3"
}
layer {
name: "pool3"
type: "Pooling"
bottom: "conv3_3"
top: "pool3"
pooling_param {
pool: MAX
kernel_size: 2
stride: 2
}
}
layer {
name: "conv4_1"
type: "Convolution"
bottom: "pool3"
top: "conv4_1"
param {
lr_mult: 1.0
decay_mult: 1.0
}
param {
lr_mult: 2.0
decay_mult: 0.0
}
convolution_param {
num_output: 512
pad: 1
kernel_size: 3
stride: 1
}
}
layer {
name: "relu4_1"
type: "ReLU"
bottom: "conv4_1"
top: "conv4_1"
}
layer {
name: "conv4_2"
type: "Convolution"
bottom: "conv4_1"
top: "conv4_2"
param {
lr_mult: 1.0
decay_mult: 1.0
}
param {
lr_mult: 2.0
decay_mult: 0.0
}
convolution_param {
num_output: 512
pad: 1
kernel_size: 3
stride: 1
}
}
layer {
name: "relu4_2"
type: "ReLU"
bottom: "conv4_2"
top: "conv4_2"
}
layer {
name: "conv4_3"
type: "Convolution"
bottom: "conv4_2"
top: "conv4_3"
param {
lr_mult: 1.0
decay_mult: 1.0
}
param {
lr_mult: 2.0
decay_mult: 0.0
}
convolution_param {
num_output: 512
pad: 1
kernel_size: 3
stride: 1
}
}
layer {
name: "relu4_3"
type: "ReLU"
bottom: "conv4_3"
top: "conv4_3"
}
layer {
name: "pool4"
type: "Pooling"
bottom: "conv4_3"
top: "pool4"
pooling_param {
pool: MAX
kernel_size: 2
stride: 2
}
}
layer {
name: "conv5_1"
type: "Convolution"
bottom: "pool4"
top: "conv5_1"
param {
lr_mult: 1.0
decay_mult: 1.0
}
param {
lr_mult: 2.0
decay_mult: 0.0
}
convolution_param {
num_output: 512
pad: 1
kernel_size: 3
stride: 1
}
}
layer {
name: "relu5_1"
type: "ReLU"
bottom: "conv5_1"
top: "conv5_1"
}
layer {
name: "conv5_2"
type: "Convolution"
bottom: "conv5_1"
top: "conv5_2"
param {
lr_mult: 1.0
decay_mult: 1.0
}
param {
lr_mult: 2.0
decay_mult: 0.0
}
convolution_param {
num_output: 512
pad: 1
kernel_size: 3
stride: 1
}
}
layer {
name: "relu5_2"
type: "ReLU"
bottom: "conv5_2"
top: "conv5_2"
}
layer {
name: "conv5_3"
type: "Convolution"
bottom: "conv5_2"
top: "conv5_3"
param {
lr_mult: 1.0
decay_mult: 1.0
}
param {
lr_mult: 2.0
decay_mult: 0.0
}
convolution_param {
num_output: 512
pad: 1
kernel_size: 3
stride: 1
}
}
layer {
name: "relu5_3"
type: "ReLU"
bottom: "conv5_3"
top: "conv5_3"
}
layer {
name: "pool5"
type: "Pooling"
bottom: "conv5_3"
top: "pool5"
pooling_param {
pool: MAX
kernel_size: 2
stride: 2
}
}
layer {
name: "fc6"
type: "Convolution"
bottom: "pool5"
top: "fc6"
param {
lr_mult: 1.0
decay_mult: 1.0
}
param {
lr_mult: 2.0
decay_mult: 0.0
}
convolution_param {
num_output: 4096
pad: 0
kernel_size: 7
stride: 1
}
}
layer {
name: "relu6"
type: "ReLU"
bottom: "fc6"
top: "fc6"
}
layer {
name: "drop6"
type: "Dropout"
bottom: "fc6"
top: "fc6"
dropout_param {
dropout_ratio: 0.5
}
}
layer {
name: "fc7"
type: "Convolution"
bottom: "fc6"
top: "fc7"
param {
lr_mult: 1.0
decay_mult: 1.0
}
param {
lr_mult: 2.0
decay_mult: 0.0
}
convolution_param {
num_output: 4096
pad: 0
kernel_size: 1
stride: 1
}
}
layer {
name: "relu7"
type: "ReLU"
bottom: "fc7"
top: "fc7"
}
layer {
name: "drop7"
type: "Dropout"
bottom: "fc7"
top: "fc7"
dropout_param {
dropout_ratio: 0.5
}
}
layer {
name: "score_fr_n"
type: "Convolution"
bottom: "fc7"
top: "score_fr"
param {
lr_mult: 1.0
decay_mult: 1.0
}
param {
lr_mult: 2.0
decay_mult: 0.0
}
convolution_param {
num_output: 2
pad: 0
kernel_size: 1
}
}
layer {
name: "upscore2_n"
type: "Deconvolution"
bottom: "score_fr"
top: "upscore2"
param {
lr_mult: 0.0
}
convolution_param {
num_output: 2
bias_term: false
kernel_size: 4
stride: 2
}
}
layer {
name: "score_pool4_n"
type: "Convolution"
bottom: "pool4"
top: "score_pool4"
param {
lr_mult: 1.0
decay_mult: 1.0
}
param {
lr_mult: 2.0
decay_mult: 0.0
}
convolution_param {
num_output: 2
pad: 0
kernel_size: 1
}
}
layer {
name: "score_pool4c"
type: "Crop"
bottom: "score_pool4"
bottom: "upscore2"
top: "score_pool4c"
crop_param {
axis: 2
offset: 5
}
}
layer {
name: "fuse_pool4"
type: "Eltwise"
bottom: "upscore2"
bottom: "score_pool4c"
top: "fuse_pool4"
eltwise_param {
operation: SUM
}
}
layer {
name: "upscore_pool4_n"
type: "Deconvolution"
bottom: "fuse_pool4"
top: "upscore_pool4"
param {
lr_mult: 0.0
}
convolution_param {
num_output: 2
bias_term: false
kernel_size: 4
stride: 2
}
}
layer {
name: "score_pool3_n"
type: "Convolution"
bottom: "pool3"
top: "score_pool3"
param {
lr_mult: 1.0
decay_mult: 1.0
}
param {
lr_mult: 2.0
decay_mult: 0.0
}
convolution_param {
num_output: 2
pad: 0
kernel_size: 1
}
}
layer {
name: "score_pool3c"
type: "Crop"
bottom: "score_pool3"
bottom: "upscore_pool4"
top: "score_pool3c"
crop_param {
axis: 2
offset: 9
}
}
layer {
name: "fuse_pool3"
type: "Eltwise"
bottom: "upscore_pool4"
bottom: "score_pool3c"
top: "fuse_pool3"
eltwise_param {
operation: SUM
}
}
layer {
name: "upscore8_n"
type: "Deconvolution"
bottom: "fuse_pool3"
top: "upscore8"
param {
lr_mult: 0.0
}
convolution_param {
num_output: 2
bias_term: false
kernel_size: 16
stride: 8
}
}
layer {
name: "score"
type: "Crop"
bottom: "upscore8"
bottom: "data"
top: "score"
crop_param {
axis: 2
offset: 31
}
}輸出的圖片。
4.在另外兩個資料集上
跟這個類似。如果把這個資料集跑通的話另外兩個根據官方的原始碼進行修改可以很容易的跑起來了。
參考過的部落格：
基於caffe的fcn網路的訓練 - xxiaozr的部落格 - CSDN部落格
FCN網路的訓練——以SIFT-Flow 資料集為例 - 一生不可自決 - 部落格園
fcn+caffe+製作自己的資料集 - 9分鐘帶帽丶 - 部落格園
FCN製作自己的資料集、訓練和測試 caffe - jacke121的專欄 - CSDN部落格用FCN訓練自己的資料集+caffe - Rlin_by的部落格 - CSDN部落格
用FCN訓練自己的資料集+caffe - Rlin_by的部落格 - CSDN部落格
CAFFE -FCN訓練配置過程 - JIN JI 2013.12.24 - CSDN
部落格FCN訓練自己的資料集及測試 - Darlewo - CSDN部落格
python 讀取、儲存、二值化、灰度化圖片+opencv處理圖片的方法 - JohnieLi的部落格 - CSDN部落格
FCN製作自己的資料集，訓練，測試 - Cheese的部落格 - CSDN部落格