作者（ [email protected] ）資訊。

（如有需要可以郵件聯絡我）

上學期神經網最後的project有關CNN的部分我們學習使用了MatConvNet。

從名字我們就知道，Matlab Convolution Neural Network。所以就不做過多介紹了。

正文：

MatConvNet 本身是用Matlab去編譯執行C++檔案，所以我們需要搭建相關連線。

這裡我用到的是Matlab2016a和VS2015兩個平臺去實現的。這裡我極力推薦VS2015版本和Matlab2013以後的版本。

因為我當時嘗試了VS2017不可以，VS2010可以但是必須要按照網上的相關步驟去一步一步的按照順序的安裝SDK，相當的麻煩。所以我強烈推薦大家用VS2015.因為只需要在安裝的時候選上開發者工具包並一起安裝就可以很簡單的進行二者的搭建。

配置方法：

1. 新增MatConvNet內的相關資料夾至Matlab路徑中。

2. 輸入：mex -setup cpp 等待系統配置。

3. 配置成功之後，輸入：vl_compilenn進行搭建相關檔案。

如下圖：

第二步，我們就可以利用MatConvNet自帶的資料集進行訓練了，不過我們當時並沒有使用自帶的cnn_mnist_init.m檔案來做訓練，我的請看下面程式碼。

下面是我的CNN的程式設計：

function cnn_mnist_NNclass(varargin)

warning off

% CNN_MNIST  Demonstrated MatConNet on MNIST

% run( fullfile(fileparts(mfilename('fullpath')), '../matlab/vl_setupnn.m') ) ;
run('C:\Users\matconvnet-1.0-beta23\matconvnet-1.0-beta23\matlab/vl_setupnn.m') ;

opts.dataDir = 'data/mnist' ;
opts.expDir = 'data/mnist-baseline' ;
opts.imdbPath = fullfile(opts.expDir, 'imdb.mat');
opts.train.batchSize = 100 ;
opts.train.numEpochs = 100 ;
opts.train.continue = true ;
% opts.train.useGpu = [] ;
opts.train.gpus = [];
opts.train.learningRate = 0.001 ;
opts.train.expDir = opts.expDir ;
opts = vl_argparse(opts, varargin);
opts.train.subsetSize = 1e4;    % statsogk

% --------------------------------------------------------------------
%                                                         Prepare data
% --------------------------------------------------------------------

if exist(opts.imdbPath)
  imdb = load(opts.imdbPath) ;
else
  imdb = getMnistImdb(opts) ;
  mkdir(opts.expDir) ;
 
  save(opts.imdbPath, '-struct', 'imdb') ;
end

% Use a subset of the images for faster training. 
if opts.train.subsetSize > 0
    imdb = getSubset(imdb,opts);
end

% Define a network similar to LeNet
f=1/100 ;
net.layers = {} ;
net.layers{end+1} = struct('type', 'conv', ...
                           'filters', f*randn(5,5,1,20, 'single'), ...
                           'biases', zeros(1, 20, 'single'), ...
                           'stride', 1, ...
                           'pad', 0) ;
% net.layers{end+1} = struct('type', 'relu') ;
net.layers{end+1} = struct('type', 'pool', ...
                           'method', 'max', ...
                           'pool', [2 2], ...
                           'stride', 2, ...
                           'pad', 0) ;
net.layers{end+1} = struct('type', 'conv', ...
                           'filters', f*randn(5,5,20,50, 'single'),...
                           'biases', zeros(1,50,'single'), ...
                           'stride', 1, ...
                           'pad', 0) ;
% net.layers{end+1} = struct('type', 'relu') ;
net.layers{end+1} = struct('type', 'pool', ...
                           'method', 'max', ...
                           'pool', [2 2], ...
                           'stride', 2, ...
                           'pad', 0) ;
net.layers{end+1} = struct('type', 'conv', ...
                           'filters', f*randn(4,4,50,500, 'single'),...
                           'biases', zeros(1,500,'single'), ...
                           'stride', 1, ...
                           'pad', 0) ;
net.layers{end+1} = struct('type', 'relu') ;

net.layers{end+1} = struct('type', 'conv', ...
                           'filters', f*randn(1,1,500,10, 'single'),...
                           'biases', zeros(1,10,'single'), ...
                           'stride', 1, ...
                           'pad', 0) ;
net.layers{end+1} = struct('type', 'softmaxloss') ;

% --------------------------------------------------------------------
%                                                                Train
% --------------------------------------------------------------------

% Take the mean out and make GPU if needed
imdb.images.data = bsxfun(@minus, imdb.images.data, mean(imdb.images.data,4)) ;
% if opts.train.useGpu
if opts.train.gpus
    imdb.images.data = gpuArray(imdb.images.data) ;
end

[ net, info ] = cnn_train(net, imdb, @getBatch, opts.train, 'val', find(imdb.images.set == 3)) ;

% --------------------------------------------------------------------
function [im, labels] = getBatch(imdb, batch)
% --------------------------------------------------------------------
im = imdb.images.data(:,:,:,batch) ;
labels = imdb.images.labels(1,batch) ;

% --------------------------------------------------------------------
function imdb = getMnistImdb(opts)
% --------------------------------------------------------------------
files = {'train-images-idx3-ubyte', ...
         'train-labels-idx1-ubyte', ...
         't10k-images-idx3-ubyte', ...
         't10k-labels-idx1-ubyte'} ;

mkdir(opts.dataDir) ;
for i=1:4
  if ~exist(fullfile(opts.dataDir, files{i}), 'file')
    url = sprintf('http://yann.lecun.com/exdb/mnist/%s.gz',files{i}) ;
    fprintf('downloading %s\n', url) ;
    gunzip(url, opts.dataDir) ;
  end
end

f=fopen(fullfile(opts.dataDir, 'train-images-idx3-ubyte'),'r') ;
x1=fread(f,inf,'uint8');
fclose(f) ;
x1=permute(reshape(x1(17:end),28,28,60e3),[2 1 3]) ;

f=fopen(fullfile(opts.dataDir, 't10k-images-idx3-ubyte'),'r') ;
x2=fread(f,inf,'uint8');
fclose(f) ;
x2=permute(reshape(x2(17:end),28,28,10e3),[2 1 3]) ;

f=fopen(fullfile(opts.dataDir, 'train-labels-idx1-ubyte'),'r') ;
y1=fread(f,inf,'uint8');
fclose(f) ;
y1=double(y1(9:end)')+1 ;

f=fopen(fullfile(opts.dataDir, 't10k-labels-idx1-ubyte'),'r') ;
y2=fread(f,inf,'uint8');
fclose(f) ;
y2=double(y2(9:end)')+1 ;

imdb.images.data = single(reshape(cat(3, x1, x2),28,28,1,[])) ;
imdb.images.labels = cat(2, y1, y2) ;
imdb.images.set = [ones(1,numel(y1)) 3*ones(1,numel(y2))] ;
imdb.meta.sets = {'train', 'val', 'test'} ;
imdb.meta.classes = arrayfun(@(x)sprintf('%d',x),0:9,'uniformoutput',false) ;

% ------------------------------------------------------------------------------
function imdb = getSubset(imdb,opts)
% ------------------------------------------------------------------------------
assert(opts.train.subsetSize <= nnz(imdb.images.set == 1),...
        'Subset size is bigger than the total train set size')
inds = find(imdb.images.set == 1);   % indices  must be from the train set
inds = randsample(inds, length(inds)-opts.train.subsetSize );
imdb.images.labels(inds) = [];
imdb.images.set(inds) = [];
imdb.images.data(:,:,:,inds) = [];
 

1. 原始CNN（LeNets）的訓練結果：

2.減去兩層（layer）之後的訓練結果：

3.增加兩層之後的訓練結果：

4.在原始網路中新增Dropout layer之後的訓練結果：

實驗結果討論：（在我的project裡面我已經寫了相關的討論（英文），大家可以百度翻譯一下看看，我這裡就不再寫了，直接複製貼上了。）

1. Thediscussion about the Top1 err and Top5 err.

First,you make a prediction using the CNN and obtain the predicted class multinomialdistribution (∑pclass=1∑pclass=1).

Now, inthe case of top-1 score, you check if the top class (the one having the highestprobability) is the same as the target label.

In thecase of top-5 score, you check if the target label is one of your top 5predictions (the 5 ones with the highest probabilities).

In bothcases, the top score is computed as the times a predicted label matched thetarget label, divided by the number of data-points evaluated.

Finally,when 5-CNNs are used, you first average their predictions and follow the sameprocedure for calculating the top-1 and top-5 scores.

2. Thediscussion about the differences between reduced and increased 

Generallyif we use more layers, it becomes more complex and more prone to overfitting.To avoid that kind of problem when increasing the layers, I add relu layers.And the result shows that it performs than the original one. But if I addlayers consisting of additional conv-pool , then it would cause overfitting anderror will go up.

Alsowhen decreasing two layers , I got less error than the original one because itreduces the complexity. More complexity means more extract details from thefeature. Through erasing some layers, network becomes nice circumstance andperforms better than the original one.

A dropout layerrandomly sets a layer's input elements to zero with a given probability.

This corresponds to temporarily droppinga randomly chosen unit and all of its connections from the network duringtraining. So, for each new input element, the software randomly selects asubset of neurons, hence forms a different layer architecture. Thesearchitectures use common weights, but because the learning does not depend onspecific neurons and connections, the dropout layer might help preventoverfitting

4. DROPOUT FOR REGULARIZATION

Hinton,Srivastava, Krizhevsky, Sutskever, & Salakhutdinov (2012) introduced thedropout regularization algorithm. Although dropout works in a different waythan L1 and L2, it accomplishes the same goal—the prevention of overfitting.However, the algorithm goes about the task by actually removing neurons and connections—atleast temporarily. Unlike L1 and L2, no weight penalty is added. Dropout doesnot directly seek to train small weights. Most neural network frameworksimplement dropout as a separate layer. Dropout layers function as a regular,densely connected neural network layer. The only difference is that the dropoutlayers will periodically drop some of their neurons during training. You canuse dropout layers on regular feedforward neural networks. Figure 6 showsdropout in action.

作者（ [email protected] ）資訊。