ReLU函式（ReLU Function）

本節通過例項介紹ReLU函式。

This section introduces the ReLU functionvia the example.

DeepReLU函式利用反向傳播演算法對給定的深度神經網路進行訓練。

The function DeepReLU trains the given deepneural network using the back-propagation algorithm.

通過採用網路權重和訓練資料，返回訓練後的權重。

It takes the weights of the network andtraining data and returns the trained weights.

[W1,W2, W3, W4] = DeepReLU(W1, W2, W3, W4, X, D)

其中W1、W2、W3和W4分別是輸入層-隱藏層1、隱藏層1-隱藏層2、隱藏層2-隱藏層3、隱藏層3-輸出層之間的權值。

where W1, W2, W3, and W4 are weightmatrices of input-hidden1, hidden1-hidden2, hidden2-hidden3, and hidden3-outputlayers, respectively.

X和D分別是訓練資料的輸入和正確輸出矩陣。

X and D are input and correct outputmatrices of the training data.

以下顯示了DeepReLU.m檔案的程式碼清單，該程式碼實現了DeepReLU函式功能。

The following listing shows the DeepReLU.m file,which implements the DeepReLU function.

function [W1, W2, W3, W4] = DeepReLU(W1,W2, W3, W4, X, D)
alpha = 0.01;
N = 5;
for k = 1:N
x = reshape(X(:, :, k), 25,1);
v1= W1*x;
y1= ReLU(v1);

          v2= W2*y1;

          y2 = ReLU(v2);

          v3 = W3*y2;

          y3 = ReLU(v3);

          v = W4*y3;

          y = Softmax(v);

          d = D(k, :)';

          e = d - y;

          delta = e;

          e3 = W4'*delta;

          delta3 = (v3 > 0).*e3;

          e2 = W3'*delta3;

          delta2 = (v2 > 0).*e2;

          e1 = W2'*delta2;

          delta1 = (v1 > 0).*e1;

          dW4 = alpha*delta*y3';

          W4= W4 + dW4;

          dW3= alpha*delta3*y2';

          W3= W3 + dW3;

          dW2= alpha*delta2*y1';

          W2= W2 + dW2;

          dW1= alpha*delta1*x';

          W1= W1 + dW1;

   end
 

end

該程式碼匯入訓練資料，使用增量規則計算權重更新（dW1、dW2、dW3和dW4），並調整神經網路的權重。

This code imports the training data,calculates the weight updates (dW1, dW2, dW3, and dW4) using the delta rule,and adjusts the weight of the neural network.

至此，該過程與以前的訓練程式碼相同。

So far, the process is identical to theprevious training codes.

唯一的差別是隱藏節點採用ReLU函式，而不是sigmoid函式。

It only differs in that the hidden nodesemploy the function ReLU, in place of sigmoid.

當然，使用不同的啟用函式會使得相應的導數發生變化。

Of course, the use of a differentactivation function yields a change in its derivative as well.

現在，讓我們來看看DeepReLU呼叫的ReLU函式。

Now, let’s look into the function ReLU thatthe function DeepReLU calls.

ReLU.m檔案中的ReLU函式程式碼清單如下。

The listing of the function ReLU shown hereis implemented in the ReLU.m file.

由於這裡只是對ReLU進行定義，略去了進一步的討論。

As this is just a definition, furtherdiscussion is omitted.

function y =ReLU(x)

   y = max(0, x);

end

後向傳播程式碼採用後向傳播演算法調整權值。

Consider the back-propagation algorithmportion, which adjusts the weights using the back-propagation algorithm.

以下程式碼為DeepReLU.m檔案中的增量計算部分。

The following listing shows the extract ofthe delta calculation from the DeepReLU.m file.

——本文譯自Phil Kim所著的《Matlab Deep Learning》

更多精彩文章請關注微訊號：