網絡結構

卷積層->池化層->卷積層->池化層->全連接層->Softmax分類器

卷積層激活函數使用relu

全連接層激活函數使用relu

池化層模式使用SAME，所以stride取2，且池化層和卷積層一樣，通常設置為SAME模式，本模式下stride=2正好實現1/2變換

網絡實現

# Author : Hellcat
# Time   : 2017/12/7

import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_data

mnist = input_data.read_data_sets(‘../../../Mnist_data‘,one_hot=True)
sess = tf.InteractiveSession()

def weight_variable(shape):
    initial = tf.truncated_normal(shape,stddev=0.1)
    return tf.Variable(initial)

def bias_variable(shape):
    # 偏置項使用極小值初始化，防止Relu出現死亡節點（dead neuron）
    initial = tf.constant(0.1, shape=shape)
    return tf.Variable(initial)

def conv2d(x, W):
    return tf.nn.conv2d(x, W, strides=[1,1,1,1], padding=‘SAME‘)

def max_pool_2x2(x):
    # 2x2池化，步長為2
    return tf.nn.max_pool(x, ksize=[1,2,2,1], strides=[1,2,2,1], padding=‘SAME‘)

x = tf.placeholder(tf.float32, [None, 784])
y_ = tf.placeholder(tf.float32, [None, 10])
x_image = tf.reshape(x, [-1, 28, 28, 1])

# 5x5濾波器，1通道，32特征圖
W_conv1 = weight_variable([5,5,1,32])
b_conv1 = bias_variable([32])

h_conv1 = tf.nn.relu((conv2d(x_image, W_conv1) + b_conv1))
h_pool1 = max_pool_2x2(h_conv1)

# 5x5濾波器，32通道，64特征圖
W_conv2 = weight_variable([5,5,32,64])
b_conv2 = bias_variable([64])

h_conv2 = tf.nn.relu(conv2d(h_pool1, W_conv2) + b_conv2)
h_pool2 = max_pool_2x2(h_conv2)

# 28x28，經過2次步長為2的最大池化(SAME)，大小變為28/2/2，即7x7
W_fc1 = weight_variable([7*7*64,1024])
b_fc1 = bias_variable([1024])
h_pool2_flat = tf.reshape(h_pool2, [-1,7*7*64])
h_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat, W_fc1) + b_fc1)

# dropout層
keep_prob = tf.placeholder(tf.float32)
h_fc1_drop = tf.nn.dropout(h_fc1, keep_prob)

W_fc2 = weight_variable([1024, 10])
b_fc2 = bias_variable([10])
y_conv = tf.nn.softmax(tf.matmul(h_fc1_drop, W_fc2) + b_fc2)

# axis=1,按行來計算
cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y_conv),axis=1))
train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)

correct_prediction = tf.equal(tf.argmax(y_conv,axis=1), tf.argmax(y_,axis=1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

tf.global_variables_initializer().run()
for i in range(20000):
    batch = mnist.train.next_batch(50)
    train_step.run(feed_dict={x:batch[0],y_:batch[1],keep_prob:0.5})
    if i % 100 == 0:
        train_accuracy = accuracy.eval(feed_dict={x:batch[0],y_:batch[1],keep_prob:1.0})
        print(‘step {0} traning accuracy {1:.3f}‘.format(i,train_accuracy))

print(‘test accuracy {}‘.format(accuracy.eval(
    feed_dict={x:mnist.test.images,y_:mnist.test.labels,keep_prob:1.0})))

收斂情況還不錯，前1000輪結果如下，

step 0 traning accuracy 0.040
step 100 traning accuracy 0.940
step 200 traning accuracy 0.940
step 300 traning accuracy 0.980
step 400 traning accuracy 0.980
step 500 traning accuracy 0.900
step 600 traning accuracy 0.920
step 700 traning accuracy 0.960
step 800 traning accuracy 1.000
step 900 traning accuracy 0.960
step 1000 traning accuracy 1.000
……

『TensorFlow』讀書筆記_簡單卷積神經網絡