該程式的撰寫方式幾乎與第2章“SGD與批處理比較”中的SGDvsBatch.m檔案的撰寫方式相同。

The architecture of this file is almostidentical to that of the SGDvsBatch.m file in the “Comparison of the SGD andthe Batch” section in Chapter 2.

clear all

X = [ 0 0 1;

   01 1;

   10 1;

   11 1;

   ];

D = [ 0 0 1 1];

E1 = zeros(1000, 1);

E2 = zeros(1000, 1);

W11 = 2*rand(4, 3) - 1; % Cross entropy

W12 = 2*rand(1, 4) - 1; %

W21 = W11; % Sum of squared error

W22 = W12; %

for epoch = 1:1000

   [W11W12] = BackpropCE(W11, W12, X, D);

   [W21W22] = BackpropXOR(W21, W22, X, D);

   es1= 0;

   es2= 0;

   N = 4;

   fork = 1:N

          x= X(k, :)';

          d = D(k);

          v1 = W11*x;

          y1 = Sigmoid(v1);

          v = W12*y1;

          y = Sigmoid(v);

          es1 = es1 + (d - y)^2;

          v1 = W21*x;

          y1 = Sigmoid(v1);

          v = W22*y1;

          y = Sigmoid(v);

          es2= es2 + (d - y)^2;

   end

   E1(epoch)= es1 / N;

   E2(epoch)= es2 / N;
 

end

plot(E1, ‘r’)

hold on

plot(E2, ‘b:’)

xlabel(‘Epoch’)

ylabel(‘Average of Training error’)

legend(‘Cross Entropy’, ‘Sum of SquaredError’)

該程式呼叫BackpropCE和BackpropXOR函式，並分別訓練神經網路1000次。

This program calls the BackpropCE and theBackpropXOR functions and trains the neural networks 1,000 times each.

對於每個神經網路，計算每個時代輸出誤差(es1和es2)的平方和，並計算其平均值(E1和E2)。

The squared sum of the output error (es1and es2) is calculated at every epoch for each neural network, and theiraverage (E1 and E2) is calculated.

W11、W12、W21和W22分別為各自神經網路的權重矩陣。

W11, W12, W21, and W22 are the weightmatrices of respective neural networks.

完成1000次訓練後，在圖中的時代軸上比較平均誤差。

Once the 1,000 trainings have beencompleted, the mean errors are compared over the epoch on the graph.

如圖3-14所示，交叉熵驅動的訓練能夠以更快的速度減少訓練誤差。

As Figure 3-14 shows, the crossentropy-driven training reduces the training error at a much faster rate.

圖3-14 交叉熵驅動的訓練能夠以更快的速度減少訓練誤差Crossentropy-driven training reduces training error at a much faster rate

換句話說，交叉熵驅動的學習規則產生更快的學習過程。

In other words, the cross entropy-drivenlearning rule yields a faster learning process.

這是大多數深度學習的代價函式採用交叉熵函式的原因。

This is the reason that most cost functionsfor Deep Learning employ the cross entropy function.

現在已經完成了反向傳播演算法的學習內容。

This completes the contents for theback-propagation algorithm.

——本文譯自Phil Kim所著的《Matlab Deep Learning》

更多精彩文章請關注微訊號：