train

• 首先，原始學習率要降低：

LEARNING_RATE_BASE = 0.01

• 第二點，x是一個四維的矩陣：

x = tf.placeholder(tf.float32,[BATCH_SIZE,mnist_inference_6_4_1.IMAGE_SIZE,mnist_inference_6_4_1.IMAGE_SIZE,mnist_inference_6_4_1.NUM_CHANNELS],name='x-input')

mnist_inference_6_4_1.NUM_CHANNELS為圖片的深度。

• xs也要換成四維矩陣：

            xs,ys =
 
 mnist.train.next_batch(BATCH_SIZE)
            reshaped_xs = np.reshape(xs,(BATCH_SIZE,mnist_inference_6_4_1.IMAGE_SIZE,mnist_inference_6_4_1.IMAGE_SIZE,mnist_inference_6_4_1.NUM_CHANNELS))
            _,loss_value,step = sess.run([train_op,loss,global_step],feed_dict={x:reshaped_xs,y_:ys})

inference

layer1

    with tf.variable_scope('layer1-conv1'):
        conv1_weights = tf.get_variable("weight",[CONV1_SIZE,CONV1_SIZE,NUM_CHANNELS,CONV1_DEEP],initializer=tf.truncated_normal_initializer(stddev=0.1))
        conv1_biases = tf.get_variable("bias",[CONV1_DEEP],initializer=tf.constant_initializer(
 
0.0))

        conv1 = tf.nn.conv2d(input_tensor,conv1_weights,strides=[1,1,1,1],padding='SAME')
        relu1 = tf.nn.relu(tf.nn.bias_add(conv1,conv1_biases))

• 首先我們看下strides引數：

strides: A list of ints.
1-D of length 4. The stride of the sliding window for each dimension
of input. Must be in the same order as the dimension specified with format.

strides代表著移動的步長，它的順序必須和input_tensor一樣，及[BATCH_SIZE,mnist_inference_6_4_1.IMAGE_SIZE,mnist_inference_6_4_1.IMAGE_SIZE,mnist_inference_6_4_1.NUM_CHANNELS]。由於BATCH_SIZE和mnist_inference_6_4_1.NUM_CHANNELS上肯定是一步一步的移動的，所以陣列的第一個值和最後一個值肯定為1。

• padding='SAME'，表示填充0，不改變Image的大小。
• 注意tf.nn.bias_add(conv1,conv1_biases)，並不是conv1與conv1_biases直接相加。

layer2

    with tf.name_scope('layer2-pool1'):
        pool1 = tf.nn.max_pool(relu1,ksize=[1,2,2,1],strides=[1,2,2,1],padding='SAME')

max_pool表示是取最大值的池化層。
我們來看下引數ksize：

ksize: A list of ints that has length >= 4. The size of the window for
each dimension of the input tensor.

視窗各個維度多大小。由於池化層視窗只在當前資料中的當前深度做，所以陣列的第一個值和最後一個值肯定為1。

layer5

layer3和layer4前面的類似，我們跳過它們來看layer5：

    pool_shape = pool2.get_shape().as_list()
    nodes = pool_shape[1] * pool_shape[2] * pool_shape[3]
    reshaped = tf.reshape(pool2,[pool_shape[0],nodes])

    with tf.variable_scope('layer5-fc1'):
        fc1_weights = tf.get_variable("weight", [nodes,FC_SIZE],
                                        initializer=tf.truncated_normal_initializer(stddev=0.1))
        if regularizer != None:
            tf.add_to_collection('losses',regularizer(fc1_weights))

        fc1_biases = tf.get_variable("bias", [FC_SIZE], initializer=tf.constant_initializer(0.1))
        fc1 = tf.nn.relu(tf.matmul(reshaped,fc1_weights) + fc1_biases)
        if train:
            fc1 = tf.nn.dropout(fc1,0.5)

• get_shape().as_list()能得到pool2的size。
  $pool\_shape[1] * pool\_shape[2] * pool\_shape[3] = 長×寬×深$，這相當把長方體拉成一條直線。pool_shape[0]為BATCH_SIZE
• dropout為了賦值過擬合，可以似的一定比例的輸出變為0。

其餘部分就是全連線神經網路了，layer6也和layer5類似。

結果：

After 1 training step(s), loss is 6.06818
After 101 training step(s), loss is 2.24668
After 201 training step(s), loss is 1.65929
After 301 training step(s), loss is 1.30799
After 401 training step(s), loss is 1.3613
After 501 training step(s), loss is 0.960646
After 601 training step(s), loss is 0.954722
After 701 training step(s), loss is 0.883449
After 801 training step(s), loss is 0.870421
After 901 training step(s), loss is 0.905906
After 1001 training step(s), loss is 0.932337