手動實現一個簡單的兩層神經網路，主要目的是體驗神經網路訓練的三個步驟：
1.通過w、x求出loss函式，這一步稱為前向傳播；

2.用第1步求出的loss函式根據鏈式法則（求導）求解出各個w對loss函式的貢獻值，即為反向傳播；

3.根據第2步求出的w對loss函式的貢獻值來調節w，使得loss函式最小；

程式碼如下：

import numpy as np
def nonlin(x, deriv=False):
    if(deriv == True):
        return x*(1-x)
    return 1/(1+np.exp(-x))

X = np.array([[0,0,1],
             [0,1,1],
             [1,0,1],
             [1,1,1]])
print(X.shape)

y = np.array([[0],
             [1],
             [1],
             [0]])
print(y.shape)

np.random.seed(1)
w0 = 2 * np.random.random((3, 4))-1
w1 = 2 * np.random.random((4, 1))-1
print(w0)
print(w0.shape)
print(w1)
print(w1.shape)

for j in range(60000):
    
    l0 = X
    l1 = nonlin(np.dot(l0,w0))
    l2 = nonlin(np.dot(l1,w1))
    
    l2_error = y - l2
    if (j%10000 == 0):
        print("Error:"+str(np.mean(np.abs(l2_error))))
    l2_delta = l2_error*nonlin(l2, deriv=True)
    l1_error = l2_delta.dot(w1.T)
    
    l1_delta = l1_error * nonlin(l1, deriv=True)
    
    w1 += l1.T.dot(l2_delta)
    w0 += l0.T.dot(l1_delta)

print(w0)
print(w1)
 

(4, 3)
(4, 1)
[[-0.16595599  0.44064899 -0.99977125 -0.39533485]
 [-0.70648822 -0.81532281 -0.62747958 -0.30887855]
 [-0.20646505  0.07763347 -0.16161097  0.370439  ]]
(3, 4)
[[-0.5910955 ]
 [ 0.75623487]
 [-0.94522481]
 [ 0.34093502]]
(4, 1)
Error:0.496410031903
Error:0.00858452565325
Error:0.00578945986251
Error:0.00462917677677
Error:0.00395876528027
Error:0.00351012256786
[[ 4.6013571   4.17197193 -6.30956245 -4.19745118]
 [-2.58413484 -5.81447929 -6.60793435 -3.68396123]
 [ 0.97538679 -2.02685775  2.52949751  5.84371739]]
[[ -6.96765763]
 [  7.14101949]
 [-10.31917382]
 [  7.86128405]]