SDM For Face Alignment 流程介紹及Matlab程式碼實現之預處理篇
SDM全稱為 Supervised Descent Method,是一種機器學習的方法,可以被用來做Face Alignment.
下面我們將通過matlab程式碼來梳理整個實現的過程。
預處理階段
Input: ../data/lfpw/trainset (811張圖片)
Output: mean_shape 811張圖片的特徵點的平均值
我們從網上download下訓練資料集,包括image和ground-truth points, 我們希望可以得到所有圖片的平均特徵點,但是由於每張圖片的尺寸各異,圖片裡的人臉也是各不相同,因此,只是簡簡單單將ground-truth points平均一下是沒有意義的,所以必須把他們統一到一個尺寸下。
我們可以提取人臉,將其放縮到400*400的尺寸下。然後通過取變換後的特徵點的平均值來作為平均特徵點。那麼如何進行呢?方法如下:
先正則化第一張圖片
1.取第一張圖片ground-truth points的包圍盒(即包含特徵點的最小矩形)。
2.將包圍盒的左上角向座標系左上角平移包圍盒一半的寬和高,作為新的包圍盒的左上角,寬和高分別取原來的2倍。這樣裁剪出的人臉就基本上是人的正臉了,同時相應的變換特徵點的位置。
3.放縮上面新得到的圖片到400*400,同時相應的變換特徵點的位置。
這樣第一張圖片的400*400的正臉以及相應的特徵點就取得了。
如下圖:
再正則其他圖片
我們通過普氏分析將其他圖片的特徵點與第一張正則化的特徵點對齊,獲得統一尺寸下的特徵點,這樣就可求解平均值了。
bounding_box.m程式碼,用來裁剪正臉:
function [cropmin,cropmax,offset,minshape,marginW,marginH] = ...
bounding_box ( shape,img )
%cropmin,cropmax分別是由特徵點的包圍盒延拓的正臉的左上角和右下角
% if(offset==[0 0]表示正臉未躍出圖片
% else 人臉需要做平移,平移後的左上角的座標點為(1,1),平移的長度為offset
% minshape:特徵點包圍盒的左上角
% marginW:特徵點包圍盒的寬的一半
% marginH:特徵點包圍盒的高的一半
%shape:特徵點,以水平的為x,以豎直的為y,同matlab的影象處理工具箱的相反
minshape = min(shape);%min_x,min_y
maxshape = max(shape);%max_x,max_y
%% calculating bounding box % %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
width = maxshape(1) - minshape(1);
height = maxshape(2) - minshape(2);
marginW = width/2;
marginH = height/2;
cropmin = round(minshape - [marginW marginH]);
cropmax = round(maxshape + [marginW marginH]);
SIZE= size(img);%由於是彩色圖,所以SIZE是三維,因此不能寫成[m,n]
offset = [0 0];%前面的盒子求出了正臉的大小包圍盒,但是如果一張照片中的頭像偏向左邊和上邊,將導致求出的正臉包圍盒超過原點,越出影象,因此需要將正臉平移,平移的尺寸為offset()+1,平移後的正臉左上角座標為(1,1)
if(cropmin(1)<=0)
offset(1) = -cropmin(1);
cropmin(1) = 1;
end
if(cropmin(2)<=0)
offset(2) = -cropmin(2);
cropmin(2) = 1;
end
% %如下為補充項,防止裁剪的圖片過大超過原圖片的邊界
if(cropmax(1)>=SIZE(2))
cropmax(1) = SIZE(2);
end
if(cropmax(2)>=SIZE(1))
cropmax(2) = SIZE(1);
end
end
normalize_first_shape.m程式碼:處理第一張圖片
function [shape] = normalize_first_shape( Data, options )
shape = Data.shape;
image = Data.img;%補充項
image=imread(image);
%% calculating bounding box %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
[cropmin,cropmax,offset,minshape,marginW,marginH] = bounding_box ( shape,image );
%%輸出offset不為0的圖片的位置
%{
if offset~=[0 0]
disp('我們要找的頭像偏左或偏上的圖片已找到,地址為:');
disp(Data.img);
pause;
end
%}
%% calculate scale factor %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
W_H = cropmax - cropmin;
wh1 = W_H(1);
wh2 = W_H(2);
CanvasSize = options.canvasSize;%標準的正臉大小
scf = CanvasSize(1)/wh1;
if(scf*wh2 > CanvasSize(2))
scf = CanvasSize(2)/wh2;
end
%% croping image (for debug only) %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
debug =0;
if debug
img = imread(Data.img);
cropImage = img(cropmin(2):cropmax(2), cropmin(1):cropmax(1));
scaleImage = imresize(cropImage, scf);
end
%% scale shape and image %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
shape = shape - repmat((minshape - [marginW marginH] + offset) ...
, size(shape, 1), 1);
shape = shape*scf;
if debug
% Displaying image and feature points.
figure(1);
imshow(image);
figure(3);
imshow(scaleImage);
hold on;
plot(shape(:, 1), shape(:, 2), 'g*');
pause;
end
end
normalize_rest_shape.m:依據正則化的第一張圖片特徵點來正則化其他圖片的特徵點。
function [shape,img] = normalize_rest_shape ( ref, data, options )
cvw = options.canvasSize(1);
cvh = options.canvasSize(2);
base = ref.shape;
shape = data.shape;
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Use procrustes analysis to align shape.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
[d, z, tform] = procrustes(base, shape, 'Reflection',false);
%% normaling shape %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
debug =0;
if debug
Trans = -1/tform.b*tform.c*tform.T';
Trans = Trans(1, :);
transM = [1/tform.b*tform.T Trans'];
cvXY = [1 cvw 1 cvw;
1 1 cvh cvh];
img = im2double(rgb2gray(imread(data.img)));
normImg = quad2Box(img, cvXY, transM);
figure(2);
imshow(normImg);
hold on;
plot(z(:, 1), z(:, 2), 'r.');
pause;
end
shape = z;
end
然後求解平均特徵點即可。