對於機器學習和人工智慧研究人員而言，好多人都只是構建好模型後就沒有進一步處理了，停留在一個比較粗糙的模型上面，沒有將其變成一個產品，其實好多創業型人工智慧公司都是設計好模型後，將其轉化成產品，之後再推向市場。每一個深度學習研究者心中或多或少都想成為一名創業者，但不知道超哪個方向發展。那麼，本文將從最簡單的網頁應用開始，一步一步帶領你使用TensorFlow建立一個卷積神經網路（CNN）模型後，使用Flash RESTful API將模型變成一個網頁應用產品。
       本文使用TensorFlow NN模組構建CNN模型，並在CIFAR-10資料集上進行訓練和測試。為了使模型可以遠端訪問，使用Python建立Flask web應用來接收上傳的影象，並使用HTTP返回其分類標籤。

1.安裝Python、TensorFlow、PyCharm和Flask API

       孔子云：工欲善其事，比先利其器。程式設計師亦如此，在進行開發前，需要準備好開發環境並基本掌握開發工具。Python是第一個需要安裝的工具，因為整個環境都依賴於它。如果你已經配置好了開發環境，那麼可以跳過第一步。

1.1 安裝Anaconda/Python

       雖然可以安裝傳統的方法安裝Python，但是建議使用類似於Anaconda這樣完整的包，因為裡面已經安裝了一些好的庫可供你直接呼叫。本文中使用的是Anaconda3版本，對於Windows系統，可以從該網站下載並安裝。
       為了確保Anaconda3是否安裝成功，在CMD命令列中輸入（where Python），如果結果類似於下圖，則表明安裝成功。

1.2 安裝TensorFlow

       在上一步Anaconda3安裝完畢後，接下來是安裝TensorFlow（TF）。本文使用的是Windows系統下CPU版本的TF，安裝指導可以見此連結。
       TF的安裝步驟如下：

1）使用下面程式碼建立conda環境：

C:> conda create -n tensorflow pip python=3.5

       這為TF安裝建立了一個空的檔案以保持虛擬環境（virtual environment, venv），vevn的位置在Anaconda3安裝的目錄檔案下：（Anaconda3envstensorflow）。

2）使用下行命令啟用venv

C:> activate tensorflow

       上行命令告訴我們venv和所需安裝的所有庫，輸入這行命令後，命令列將變(tensorflow)C:>，接下來是安裝TensorFlow包。

3）在啟用venv後，Window下CPU版本的TensorFlow可以直接使用pip安裝：

程式碼
       為了測試TF是否安裝成功，可以匯入TensorFlow，若結果與下圖一樣，則表明安裝成功。但是在匯入TF之前，請確保venv被啟用。

1.3安裝PyCharm Python IDE

       相較於在CMD命令列中輸入程式碼，本文更傾向於使用Python IDE。本文選擇PyCharm，Windows版本的下載連結在此。此外，下載安裝完畢後，需要設定Python編譯器，操作如下圖所示，選擇之前安裝的Python.exe作為IDE的編譯器。

1.4 安裝Flask

       最後的一個工具是安裝Flask RESTful API，安裝命令如下：

C:> pip install Flask-API

       在全部安裝完畢後，接下來要開始新建工程了。

2.下載並預處理CIFAR-10資料集

       CIFAR-10資料集可以在此下載，該資料集包含60,000張影象，並將其劃分為訓練集和測試集。其中訓練集有5個資料夾，分別命名為data_batch_1、data_batch_2...,data_batch_5，每個資料夾中包含10,000張，每張都是32x32x3的RGB影象。測試集只有一個資料夾，命名為batches.meta，包含10,000張影象。訓練集和測試集中包含的影象類別為飛機（airplane）、手機（automobile）、鳥（bird）、貓（cat）、鹿（deer）、狗（dog）、青蛙（frog）、馬（horse）、船（ship）以及truck（卡車）。
       由於資料集中的每個檔案都是二進位制檔案，因此應該對其進行解碼以檢索實際的影象資料。基於此，建立unpickle_patch函式來執行如下操作：

def unpickle_patch(file):

    """
    Decoding the binary file.
    :param file:File to decode it data.
    :return:Dictionary of the file holding details including input data and output labels.
    """
    patch_bin_file = open(file, 'rb')#Reading the binary file.
    patch_dict = pickle.load(patch_bin_file, encoding='bytes')#Loading the details of the binary file into a dictionary.
    return patch_dict#Returning the dictionary.

       該方法接收二進位制檔案並返回一個包含有關此檔案詳細資訊的字典。該字典除了標籤之外還包含檔案中所有的10,000個樣本的資料。
       為了解碼整個訓練集，建立get_dataset_images函式。該函式接收資料集路徑並僅對訓練資料起作用。因此，它會過濾一些檔案並只返回以data_batch_開頭的檔案。測試資料在模型訓練好後再進行處理。
       對於每一個訓練資料夾，使用unpickle_patch函式解碼，該函式輸出一個字典。之後使用get_dataset_images函式獲取影象資料以及其對應的類別標籤。影象資料是從“data”鍵中檢索，類別標籤從“labels”鍵中檢索。
       由於資料圖形是以一維向量的形式儲存，此外TensorFlow接收的是三維形式，因此應對其進行變換處理。基於此，get_dataset_images函式接收的引數為影象的檔案路徑、行/列數以及影象的通道數。

def get_dataset_images(dataset_path, im_dim=32, num_channels=3):

    """
    This function accepts the dataset path, reads the data, and returns it after being reshaped to match the requierments of the CNN.
    :param dataset_path:Path of the CIFAR10 dataset binary files.
    :param im_dim:Number of rows and columns in each image. The image is expected to be rectangular.
    :param num_channels:Number of color channels in the image.
    :return:Returns the input data after being reshaped and output labels.
    """
    num_files = 5#Number of training binary files in the CIFAR10 dataset.
    images_per_file = 10000#Number of samples withing each binary file.
    files_names = os.listdir(patches_dir)#Listing the binary files in the dataset path.
    """
    Creating an empty array to hold the entire training data after being reshaped.
    The dataset has 5 binary files holding the data. Each binary file has 10,000 samples. Total number of samples in the dataset is 5*10,000=50,000.
    Each sample has a total of 3,072 pixels. These pixels are reshaped to form a RGB image of shape 32x32x3.
    Finally, the entire dataset has 50,000 samples and each sample of shape 32x32x3 (50,000x32x32x3).
    """
    dataset_array = numpy.zeros(shape=(num_files * images_per_file, im_dim, im_dim, num_channels))
    #Creating an empty array to hold the labels of each input sample. Its size is 50,000 to hold the label of each sample in the dataset.
    dataset_labels = numpy.zeros(shape=(num_files * images_per_file), dtype=numpy.uint8)
    index = 0#Index variable to count number of training binary files being processed.
    for file_name in files_names:
        """
        Because the CIFAR10 directory does not only contain the desired training files and has some  other files, it is required to filter the required files.
        Training files start by 'data_batch_' which is used to test whether the file is for training or not.
        """
        if file_name[0:len(file_name) - 1] == "data_batch_":
            print("Working on : ", file_name)
            """
            Appending the path of the binary files to the name of the current file.
            Then the complete path of the binary file is used to decoded the file and return the actual pixels values.
            """
            data_dict = unpickle_patch(dataset_path+file_name)
            """
            Returning the data using its key 'data' in the dictionary.
            Character b is used before the key to tell it is binary string.
            """
            images_data = data_dict[b"data"]
            #Reshaping all samples in the current binary file to be of 32x32x3 shape.
            images_data_reshaped = numpy.reshape(images_data, newshape=(len(images_data), im_dim, im_dim, num_channels))
            #Appending the data of the current file after being reshaped.
            dataset_array[index * images_per_file:(index + 1) * images_per_file, :, :, :] = images_data_reshaped
            #Appening the labels of the current file.
            dataset_labels[index * images_per_file:(index + 1) * images_per_file] = data_dict[b"labels"]
            index = index + 1#Incrementing the counter of the processed training files by 1 to accept new file.
    return dataset_array, dataset_labels#Returning the training input data and output labels.

       處理好訓練集後，下一步構建CNN模型並進行訓練。

3.使用TensorFlow構建CNN模型

       使用creat_CNN函式建立CNN模型，該函式建立卷積層（conv）、ReLU啟用函式、最大池化（max pooling）、dropout以及全連線層（full connection,FC），最後一層全連線層輸出結果。每一層的輸出都是下一層的輸入，這就要求相鄰兩層之間的特徵圖尺寸大小要一致。此外，對於每個conv、ReLU以及最大池化層等，都有一些超引數需要設定，比如卷積或池化時候設定的步長等。

def create_CNN(input_data, num_classes, keep_prop):

    """
    Builds the CNN architecture by stacking conv, relu, pool, dropout, and fully connected layers.
    :param input_data:patch data to be processed.
    :param num_classes:Number of classes in the dataset. It helps determining the number of outputs in the last fully connected layer.
    :param keep_prop:probability of dropping neurons in the dropout layer.
    :return: last fully connected layer.
    """
    #Preparing the first convolution layer.
    filters1, conv_layer1 = create_conv_layer(input_data=input_data, filter_size=5, num_filters=4)
    """
    Applying ReLU activation function over the conv layer output. 
    It returns a new array of the same shape as the input array.
    """
    relu_layer1 = tensorflow.nn.relu(conv_layer1)
    print("Size of relu1 result : ", relu_layer1.shape)
    """
    Max pooling is applied to the ReLU layer result to achieve translation invariance.
    It returns a new array of a different shape from the the input array relative to the strides and kernel size used.
    """
    max_pooling_layer1 = tensorflow.nn.max_pool(value=relu_layer1,
                                                ksize=[1, 2, 2, 1],
                                                strides=[1, 1, 1, 1],
                                                padding="VALID")
    print("Size of maxpool1 result : ", max_pooling_layer1.shape)

    #Similar to the previous conv-relu-pool layers, new layers are just stacked to complete the CNN architecture.
    #Conv layer with 3 filters and each filter is of sisze of 5x5.
    filters2, conv_layer2 = create_conv_layer(input_data=max_pooling_layer1, filter_size=7, num_filters=3)
    relu_layer2 = tensorflow.nn.relu(conv_layer2)
    print("Size of relu2 result : ", relu_layer2.shape)
    max_pooling_layer2 = tensorflow.nn.max_pool(value=relu_layer2,
                                                ksize=[1, 2, 2, 1],
                                                strides=[1, 1, 1, 1],
                                                padding="VALID")
    print("Size of maxpool2 result : ", max_pooling_layer2.shape)

    #Conv layer with 2 filters and a filter sisze of 5x5.
    filters3, conv_layer3 = create_conv_layer(input_data=max_pooling_layer2, filter_size=5, num_filters=2)
    relu_layer3 = tensorflow.nn.relu(conv_layer3)
    print("Size of relu3 result : ", relu_layer3.shape)
    max_pooling_layer3 = tensorflow.nn.max_pool(value=relu_layer3,
                                                ksize=[1, 2, 2, 1],
                                                strides=[1, 1, 1, 1],
                                                padding="VALID")
    print("Size of maxpool3 result : ", max_pooling_layer3.shape)

    #Adding dropout layer before the fully connected layers to avoid overfitting.
    flattened_layer = dropout_flatten_layer(previous_layer=max_pooling_layer3, keep_prop=keep_prop)

    #First fully connected (FC) layer. It accepts the result of the dropout layer after being flattened (1D).
    fc_resultl = fc_layer(flattened_layer=flattened_layer, num_inputs=flattened_layer.get_shape()[1:].num_elements(),
                          num_outputs=200)
    #Second fully connected layer accepting the output of the previous fully connected layer. Number of outputs is equal to the number of dataset classes.
    fc_result2 = fc_layer(flattened_layer=fc_resultl, num_inputs=fc_resultl.get_shape()[1:].num_elements(),
                          num_outputs=num_classes)
    print("Fully connected layer results : ", fc_result2)
    return fc_result2#Returning the result of the last FC layer.

       由於卷積層將輸入資料與設定的卷積核進行卷積運算，因此create_CNN函式將輸入資料作為輸入引數，這些資料是由get_dataset_images函式返回的資料。create_conv_layer函式接收輸入資料、過濾器大小和過濾器數量，並返回輸入資料與過濾器集合進行卷積的結果。這組濾波器的大小根據輸入影象的深度而設定。 create_conv_layer的定義如下：

def create_conv_layer(input_data, filter_size, num_filters):

    """
    Builds the CNN convolution (conv) layer.
    :param input_data:patch data to be processed.
    :param filter_size:#Number of rows and columns of each filter. It is expected to have a rectangular filter.
    :param num_filters:Number of filters.
    :return:The last fully connected layer of the network.
    """
    """
    Preparing the filters of the conv layer by specifiying its shape. 
    Number of channels in both input image and each filter must match.
    Because number of channels is specified in the shape of the input image as the last value, index of -1 works fine.
    """
    filters = tensorflow.Variable(tensorflow.truncated_normal(shape=(filter_size, filter_size, tensorflow.cast(input_data.shape[-1], dtype=tensorflow.int32), num_filters),
                                                              stddev=0.05))
    print("Size of conv filters bank : ", filters.shape)

    """
    Building the convolution layer by specifying the input data, filters, strides along each of the 4 dimensions, and the padding.
    Padding value of 'VALID' means the some borders of the input image will be lost in the result based on the filter size.
    """
    conv_layer = tensorflow.nn.conv2d(input=input_data,
                                      filter=filters,
                                      strides=[1, 1, 1, 1],
                                      padding="VALID")
    print("Size of conv result : ", conv_layer.shape)
    return filters, conv_layer#Returing the filters and the convolution layer result.

       對於dropout層，接收一個保持神經元的概率引數，它表明會有多少神經元在dropout層被丟棄。 dropout層是使用dropout_flatten_layer函式實現，如下所示：

ef dropout_flatten_layer(previous_layer, keep_prop):

    """
    Applying the dropout layer.
    :param previous_layer: Result of the previous layer to the dropout layer.
    :param keep_prop: Probability of keeping neurons.
    :return: flattened array.
    """
    dropout = tensorflow.nn.dropout(x=previous_layer, keep_prob=keep_prop)
    num_features = dropout.get_shape()[1:].num_elements()
    layer = tensorflow.reshape(previous_layer, shape=(-1, num_features))#Flattening the results.
    return layer

       由於最後一個FC層的輸出神經元數應等於資料集類別數量，因此資料集類的數量將用作create_CNN函式的另一個輸入引數。全連線層是使用fc_layer函式建立，該函式接收dropout層輸出結果，輸出結果中的特徵數量以及來自此FC層的輸出神經元的數量。根據輸入和輸出的數量，建立一個權重張量，然後乘以flattened_layer得到FC層的返回結果。

def fc_layer(flattened_layer, num_inputs, num_outputs):

    """
    uilds a fully connected (FC) layer.
    :param flattened_layer: Previous layer after being flattened.
    :param num_inputs: Number of inputs in the previous layer.
    :param num_outputs: Number of outputs to be returned in such FC layer.
    :return:
    """
    #Preparing the set of weights for the FC layer. It depends on the number of inputs and number of outputs.
    fc_weights = tensorflow.Variable(tensorflow.truncated_normal(shape=(num_inputs, num_outputs),
                                                              stddev=0.05))
    #Matrix multiplication between the flattened array and the set of weights.
    fc_resultl = tensorflow.matmul(flattened_layer, fc_weights)
    return fc_resultl#Output of the FC layer (result of matrix multiplication).

       使用TensorBoard可以視覺化網路模型結構，如下圖所示：

4.訓練CNN模型

       在構建好CNN模型之後，下一步就是使用之前處理的訓練資料進行模型訓練，程式碼如下所示。程式碼首先準備訓練資料的路徑，然後呼叫之前的討論過的函式，訓練的CNN使用梯度下降演算法，優化方式是儘可能最小化代價函式。

#Nnumber of classes in the dataset. Used to specify number of outputs in the last fully connected layer.
num_datatset_classes = 10
#Number of rows & columns in each input image. The image is expected to be rectangular Used to reshape the images and specify the input tensor shape.
im_dim = 32
#Number of channels in rach input image. Used to reshape the images and specify the input tensor shape.
num_channels = 3

#Directory at which the training binary files of the CIFAR10 dataset are saved.
patches_dir = "C:\\Users\\Dell\\Downloads\\Compressed\\cifar-10-python\\cifar-10-batches-py\\"
#Reading the CIFAR10 training binary files and returning the input data and output labels. Output labels are used to test the CNN prediction accuracy.
dataset_array, dataset_labels = get_dataset_images(dataset_path=patches_dir, im_dim=im_dim, num_channels=num_channels)
print("Size of data : ", dataset_array.shape)

"""
Input tensor to hold the data read above. It is the entry point of the computational graph.
The given name of 'data_tensor' is useful for retreiving it when restoring the trained model graph for testing.
"""
data_tensor = tensorflow.placeholder(tensorflow.float32, shape=[None, im_dim, im_dim, num_channels], name='data_tensor')

"""
Tensor to hold the outputs label. 
The name "label_tensor" is used for accessing the tensor when tesing the saved trained model after being restored.
"""
label_tensor = tensorflow.placeholder(tensorflow.float32, shape=[None], name='label_tensor')

#The probability of dropping neurons in the dropout layer. It is given a name for accessing it later.
keep_prop = tensorflow.Variable(initial_value=0.5, name="keep_prop")

#Building the CNN architecure and returning the last layer which is the fully connected layer.
fc_result2 = create_CNN(input_data=data_tensor, num_classes=num_datatset_classes, keep_prop=keep_prop)

"""
Predicitions probabilities of the CNN for each training sample.
Each sample has a probability for each of the 10 classes in the dataset.
Such tensor is given a name for accessing it later.
"""
softmax_propabilities = tensorflow.nn.softmax(fc_result2, name="softmax_probs")

"""
Predicitions labels of the CNN for each training sample.
The input sample is classified as the class of the highest probability.
axis=1 indicates that maximum of values in the second axis is to be returned. This returns that maximum class probability fo each sample.
"""
softmax_predictions = tensorflow.argmax(softmax_propabilities, axis=1)

#Cross entropy of the CNN based on its calculated probabilities.
cross_entropy = tensorflow.nn.softmax_cross_entropy_with_logits(logits=tensorflow.reduce_max(input_tensor=softmax_propabilities, reduction_indices=[1]),
                                                                labels=label_tensor)
#Summarizing the cross entropy into a single value (cost) to be minimized by the learning algorithm.
cost = tensorflow.reduce_mean(cross_entropy)
#Minimizng the network cost using the Gradient Descent optimizer with a learning rate is 0.01.
error = tensorflow.train.GradientDescentOptimizer(learning_rate=.01).minimize(cost)

#Creating a new TensorFlow Session to process the computational graph.
sess = tensorflow.Session()
#Wiriting summary of the graph to visualize it using TensorBoard.
tensorflow.summary.FileWriter(logdir="./log/", graph=sess.graph)
#Initializing the variables of the graph.
sess.run(tensorflow.global_variables_initializer())

"""
Because it may be impossible to feed the complete data to the CNN on normal machines, it is recommended to split the data into a number of patches.
A percent of traning samples is used to create each path. Samples for each path can be randomly selected.
"""
num_patches = 5#Number of patches
for patch_num in numpy.arange(num_patches):
    print("Patch : ", str(patch_num))
    percent = 80 #percent of samples to be included in each path.
    #Getting the input-output data of the current path.
    shuffled_data, shuffled_labels = get_patch(data=dataset_array, labels=dataset_labels, percent=percent)
    #Data required for cnn operation. 1)Input Images, 2)Output Labels, and 3)Dropout probability
    cnn_feed_dict = {data_tensor: shuffled_data,
                     label_tensor: shuffled_labels,
                     keep_prop: 0.5}
    """
    Training the CNN based on the current patch. 
    CNN error is used as input in the run to minimize it.
    SoftMax predictions are returned to compute the classification accuracy.
    """
    softmax_predictions_, _ = sess.run([softmax_predictions, error], feed_dict=cnn_feed_dict)
    #Calculating number of correctly classified samples.
    correct = numpy.array(numpy.where(softmax_predictions_ == shuffled_labels))
    correct = correct.size
    print("Correct predictions/", str(percent * 50000/100), ' : ', correct)

       與其將整個資料集一下子送入網路中去，不如將資料分為一批資料塊（patch），將資料塊分批次送入網路之中形成一個迴圈。每個資料塊都包含訓練資料的子集，這些資料塊使用get_patch函式返回，該函式接收的引數為輸入資料、標籤以及返回的百分數，函式根據百分數劃分子集。

def get_patch(data, labels, percent=70):

    """
    Returning patch to train the CNN.
    :param data: Complete input data after being encoded and reshaped.
    :param labels: Labels of the entire dataset.
    :param percent: Percent of samples to get returned in each patch.
    :return: Subset of the data (patch) to train the CNN model.
    """
    #Using the percent of samples per patch to return the actual number of samples to get returned.
    num_elements = numpy.uint32(percent*data.shape[0]/100)
    shuffled_labels = labels#Temporary variable to hold the data after being shuffled.
    numpy.random.shuffle(shuffled_labels)#Randomly reordering the labels.
    """
    The previously specified percent of the data is returned starting from the beginning until meeting the required number of samples. 
    The labels indices are also used to return their corresponding input images samples.
    """
    return data[shuffled_labels[:num_elements], :, :, :], shuffled_labels[:num_elements]

5.儲存訓練好的CNN模型

       在訓練好CNN模型之後，需要儲存訓練好的引數以便測試時使用，儲存路徑由你指定，程式碼如下：

#Saving the model after being trained.
saver = tensorflow.train.Saver()
save_model_path = "C:\\model\\"
save_path = saver.save(sess=sess, save_path=save_model_path+"model.ckpt")
print("Model saved in : ", save_path)

6.準備測試資料並載入訓練好的CNN模型

       在測試之前，需要準備測試資料並恢復以前的訓練模型。測試資料準備與訓練資料準備的情況類似，不同的是隻有一個二進位制檔案需要解碼，根據修改後的get_dataset_images函式對測試檔案進行解碼，該函式完全按照訓練資料所做的那樣呼叫unpickle_patch函式：

def get_dataset_images(test_path_path, im_dim=32, num_channels=3):

    """
    Similar to the one used in training except that there is just a single testing binary file for testing the CIFAR10 trained models.
    """
    print("Working on testing patch")
    data_dict = unpickle_patch(test_path_path)
    images_data = data_dict[b"data"]
    dataset_array = numpy.reshape(images_data, newshape=(len(images_data), im_dim, im_dim, num_channels))
    return dataset_array, data_dict[b"labels"]

7.測試CNN模型

       準備好測試資料並恢復訓練好的模型後，可以按照以下程式碼開始測試模型。值得一提的是，目標是僅返回輸入樣本的網路預測結果，這也是TF會話執行只返回預測的原因。此外，與訓練CNN時會話將盡可能降低代價不同的是，在測試中並不想將成本降到最低，而是關注於預測精度。另一個有趣的地方是，dropout層的概率設定為1，即不丟棄任何節點。

#Dataset path containing the testing binary file to be decoded.
patches_dir = "C:\\Users\\Dell\\Downloads\\Compressed\\cifar-10-python\\cifar-10-batches-py\\"
dataset_array, dataset_labels = get_dataset_images(test_path_path=patches_dir + "test_batch", im_dim=32, num_channels=3)
print("Size of data : ", dataset_array.shape)

sess = tensorflow.Session()

#Restoring the previously saved trained model.
saved_model_path = 'C:\\Users\\Dell\\Desktop\\model\\'
saver = tensorflow.train.import_meta_graph(saved_model_path+'model.ckpt.meta')
saver.restore(sess=sess, save_path=saved_model_path+'model.ckpt')

#Initalizing the varaibales.
sess.run(tensorflow.global_variables_initializer())

graph = tensorflow.get_default_graph()

"""
Restoring previous created tensors in the training phase based on their given tensor names in the training phase.
Some of such tensors will be assigned the testing input data and their outcomes (data_tensor, label_tensor, and keep_prop).
Others are helpful in assessing the model prediction accuracy (softmax_propabilities and softmax_predictions).
"""
softmax_propabilities = graph.get_tensor_by_name(name="softmax_probs:0")
softmax_predictions = tensorflow.argmax(softmax_propabilities, axis=1)
data_tensor = graph.get_tensor_by_name(name="data_tensor:0")
label_tensor = graph.get_tensor_by_name(name="label_tensor:0")
keep_prop = graph.get_tensor_by_name(name="keep_prop:0")

#keep_prop is equal to 1 because there is no more interest to remove neurons in the testing phase.
feed_dict_testing = {data_tensor: dataset_array,
                     label_tensor: dataset_labels,
                     keep_prop: 1.0}
#Running the session to predict the outcomes of the testing samples.
softmax_propabilities_, softmax_predictions_ = sess.run([softmax_propabilities, softmax_predictions],
                                                      feed_dict=feed_dict_testing)
#Assessing the model accuracy by counting number of correctly classified samples.
correct = numpy.array(numpy.where(softmax_predictions_ == dataset_labels))
correct = correct.size
print("Correct predictions/10,000 : ", correct)

8.構建Flask web應用

       在訓練好CNN模型後，將它加入到HTTP伺服器中，並允許使用者線上使用。使用者將使用HTTP客戶端上傳一張影象，該影象之後會被HTTP伺服器（Flask web應用）接收，該應用將基於訓練好的CNN模型預測該影象的類別，並最終將類別返還給HTTP客戶端。整個過程如下圖所示：

import flask
#Creating a new Flask Web application. It accepts the package name.
app = flask.Flask("CIFAR10_Flask_Web_App")

"""
To activate the Web server to receive requests, the application must run.
A good practice is to check whether the file is whether the file called from an external Python file or not.
If not, then it will run.
"""
if __name__ == "__main__":
    """
    In this example, the app will run based on the following properties:
    host: localhost
    port: 7777
    debug: flag set to True to return debugging information.
    """
    app.run(host="localhost", port=7777, debug=True)

       目前，伺服器沒有提供任何功能。伺服器應該做的第一件事是允許使用者上傳影象，當用戶訪問該應用程式的根URL時，該應用程式不會執行任何操作。應用程式可以將使用者重定向到使用者可以上傳影象的HTML頁面。為此，該應用程式有一個redirect_upload的函式，可將使用者重定向到用於上傳影象的頁面，讓使用者在訪問應用程式根目錄後執行此函式能的是使用以下行建立的路由：

app.add_url_rule(rule="/", endpoint="homepage", view_func=redirect_upload)

       上行程式碼表示：如果使用者訪問應用程式的根目錄（標記為“/”），則將呼叫檢視函式（redirect_upload）。除了渲染upload_image.html的HTML頁面之外，這個函式什麼也不做，此頁面位於伺服器的特殊模板目錄下。模板目錄內的頁面通過呼叫render_template函式來呈現。

def redirect_upload():

    """
    A viewer function that redirects the Web application from the root to a HTML page for uploading an image to get classified.
    The HTML page is located under the /templates directory of the application.
    :return: HTML page used for uploading an image. It is 'upload_image.html' in this exmaple.
    """
    return flask.render_template(template_name_or_list="upload_image.html")
"""
Creating a route between the homepage URL (http://localhost:7777) to a viewer function that is called after getting to such URL. 
Endpoint 'homepage' is used to make the route reusable without hard-coding it later.
"""
app.add_url_rule(rule="/", endpoint="homepage", view_func=redirect_upload)

       HTML頁面的螢幕顯示如下圖所示：

       以下程式碼是上圖頁面的HTML程式碼，實現的功能也很簡單，允許使用者上傳一張影象，當提交此類表單時，POST HTTP訊息將被返回給URL：
http://localhost:7777/upload/

<!DOCTYPE html>
<html lang="en">
<head>
    <link rel="stylesheet" type="text/css" href="{{url_for(endpoint='static', filename='project_styles.css')}}">
    <meta charset="UTF-8">
    <title>Upload Image</title>
</head>
<body>
<form enctype="multipart/form-data" method="post" action="http://localhost:7777/upload/">
    <center>
    <h3>Select CIFAR10 image to predict its label.</h3>
    <input type="file" name="image_file" accept="image/*"><br>
    <input type="submit" value="Upload">
    </center>
</form>
</body>
</html>

       在從HTML表單返回到伺服器之後，將呼叫與action表單屬性中指定URL關聯的檢視函式upload_image，該函式獲取使用者選擇的影象並將其儲存到伺服器。

def upload_image():

    """
    Viewer function that is called in response to getting to the 'http://localhost:7777/upload' URL.
    It uploads the selected image to the server.
    :return: redirects the application to a new page for predicting the class of the image.
    """
    #Global variable to hold the name of the image file for reuse later in prediction by the 'CNN_predict' viewer functions.
    global secure_filename
    if flask.request.method == "POST":#Checking of the HTTP method initiating the request is POST.
        img_file = flask.request.files["image_file"]#Getting the file name to get uploaded.
        secure_filename = werkzeug.secure_filename(img_file.filename)#Getting a secure file name. It is a good practice to use it.
        img_path = os.path.join(app.root_path, secure_filename)#Preparing the full path under which the image will get saved.
        img_file.save(img_path)#Saving the image in the specified path.
        print("Image uploaded successfully.")
        """
        After uploading the image file successfully, next is to predict the class label of it.
        The application will fetch the URL that is tied to the HTML page responsible for prediction and redirects the browser to it.
        The URL is fetched using the endpoint 'predict'.
        """
        return flask.redirect(flask.url_for(endpoint="predict"))
    return "Image upload failed."
"""
Creating a route between the URL (http://localhost:7777/upload) to a viewer function that is called after navigating to such URL. 
Endpoint 'upload' is used to make the route reusable without hard-coding it later.
The set of HTTP method the viewer function is to respond to is added using the 'methods' argument.
In this case, the function will just respond to requests of method of type POST.
"""
app.add_url_rule(rule="/upload/", endpoint="upload", view_func=upload_image, methods=["POST"])

       將影象成功上傳到伺服器後，已準備好讀取影象並使用之前訓練過的CNN模型預測其類別標籤。基於此，upload_image函式將應用程式重定向到負責預測影象類標籤的檢視器函式。這個檢視器功能是通過它的端點（endpoint）來達到的，如下行所指定的：

return flask.redirect(flask.url_for(endpoint="predict"))

       負責預測影象類別標籤的函式CNN_predict定義如下：

def CNN_predict():

    """
    Reads the uploaded image file and predicts its label using the saved pre-trained CNN model.
    :return: Either an error if the image is not for CIFAR10 dataset or redirects the browser to a new page to show the prediction result if no error occurred.
    """
    """
    Setting the previously created 'secure_filename' to global.
    This is because to be able invoke a global variable created in another function, it must be defined global in the caller function.
    """
    global secure_filename
    #Reading the image file from the path it was saved in previously.
    img = scipy.misc.imread(os.path.join(app.root_path, secure_filename))

    """
    Checking whether the image dimensions match the CIFAR10 specifications.
    CIFAR10 images are RGB (i.e. they have 3 dimensions). It number of dimenions was not equal to 3, then a message will be returned.
    """
    if(img.ndim) == 3:
        """
        Checking if the number of rows and columns of the read image matched CIFAR10 (32 rows and 32 columns).
        """
        if img.shape[0] == img.shape[1] and img.shape[0] == 32:
            """
            Checking whether the last dimension of the image has just 3 channels (Red, Green, and Blue).
            """
            if img.shape[-1] == 3:
                """
                Passing all conditions above, the image is proved to be of CIFAR10.
                This is why it is passed to the predictor.
                """
                predicted_class = CIFAR10_CNN_Predict_Image.main(img)
                """
                After predicting the class label of the input image, the prediction label is rendered on an HTML page.
                The HTML page is fetched from the /templates directory. The HTML page accepts an input which is the predicted class.
                """
                return flask.render_template(template_name_or_list="prediction_result.html", predicted_class=predicted_class)
            else:
                # If the image dimensions do not match the CIFAR10 specifications, then an HTML page is rendered to show the problem.
                return flask.render_template(template_name_or_list="error.html", img_shape=img.shape)
        else:
            # If the image dimensions do not match the CIFAR10 specifications, then an HTML page is rendered to show the problem.
            return flask.render_template(template_name_or_list="error.html", img_shape=img.shape)
    return "An error occurred."#Returned if there is a different error other than wrong image dimensions.
"""
Creating a route between the URL (http://localhost:7777/predict) to a viewer function that is called after navigating to such URL. 
Endpoint 'predict' is used to make the route reusable without hard-coding it later.
"""
app.add_url_rule(rule="/predict/", endpoint="predict", view_func=CNN_predict)

       負責預測影象類別標籤的主函式定義如下，它載入訓練好的模型並執行會話，返回影象的預測類別，預測的類別將返回到Flask Web應用程式。