Tensorflow,CNN和MNIST資料 識別手寫的數字(入門,完整程式碼,問題解析)
MNIST解析:
1. 匯入所需模組:
#讀圖
from PIL import Image
#顯示
import matplotlib.pyplot as plt
#TensorFlow
import tensorflow as tf
#MNIST資料
from tensorflow.examples.tutorials.mnist import input_data
2.下載並讀入MNIST資料
mnist = input_data.read_data_sets('MNIST_data', one_hot=True)
3.定義網路結構:
def conv2d(x,w): return tf.nn.conv2d(x,w,strides=[1,1,1,1],padding='SAME') #自定義池化函式 def max_pool_2x2(x): return tf.nn.max_pool(x,ksize=[1,2,2,1],strides=[1,2,2,1],padding='SAME')
#設定佔位符,尺寸為樣本輸入和輸出的尺寸 x=tf.placeholder(tf.float32,[None,784]) y_=tf.placeholder(tf.float32,[None,10]) x_img=tf.reshape(x,[-1,28,28,1]) #設定第一個卷積層和池化層 w_conv1=tf.Variable(tf.truncated_normal([3,3,1,16],stddev=0.1)) print(w_conv1.name) b_conv1=tf.Variable(tf.constant(0.1,shape=[16])) h_conv1=tf.nn.relu(conv2d(x_img,w_conv1)+b_conv1) h_pool1=max_pool_2x2(h_conv1) #設定第二個卷積層和池化層 w_conv2=tf.Variable(tf.truncated_normal([3,3,16,25],stddev=0.1)) b_conv2=tf.Variable(tf.constant(0.1,shape=[25])) h_conv2=tf.nn.relu(conv2d(h_pool1,w_conv2)+b_conv2) h_pool2=max_pool_2x2(h_conv2) #設定第一個全連線層 w_fc1=tf.Variable(tf.truncated_normal([7*7*25,516],stddev=0.1)) b_fc1=tf.Variable(tf.constant(0.1,shape=[516])) h_pool2_flat=tf.reshape(h_pool2,[-1,7*7*25]) h_fc1=tf.nn.relu(tf.matmul(h_pool2_flat,w_fc1)+b_fc1) #dropout(隨機權重失活) keep_prob=tf.placeholder(tf.float32) h_fc1_drop=tf.nn.dropout(h_fc1,keep_prob) #設定第二個全連線層 w_fc2=tf.Variable(tf.truncated_normal([516,10],stddev=0.1)) b_fc2=tf.Variable(tf.constant(0.1,shape=[10])) y_out=tf.nn.softmax(tf.matmul(h_fc1_drop,w_fc2)+b_fc2) #建立loss function,為交叉熵 loss=tf.reduce_mean(-tf.reduce_sum(y_*tf.log(y_out),reduction_indices=[1])) #配置Adam優化器,學習速率為1e-4 train_step=tf.train.AdamOptimizer(1e-4).minimize(loss) #建立正確率計算表示式 correct_prediction=tf.equal(tf.argmax(y_out,1),tf.argmax(y_,1)) accuracy=tf.reduce_mean(tf.cast(correct_prediction,tf.float32))
4.設定引數 isTrain ,為True表示訓練,為False使用自己輸入的影象測試
isTrain = True
5.開始訓練
#開始資料訓練 sess=tf.InteractiveSession() saver = tf.train.Saver() # defaults to saving all variables,只儲存最後一代的模型 tf.global_variables_initializer().run() tf.reset_default_graph() #訓練 if isTrain: for i in range(800): batch=mnist.train.next_batch(20) if i%100==0: train_accuracy=accuracy.eval(feed_dict={x:batch[0],y_:batch[1],keep_prob:1}) print ("step %d,train_accuracy= %g"%(i,train_accuracy)) train_step.run(feed_dict={x:batch[0],y_:batch[1],keep_prob:0.5}) saver.save(sess, 'F:/MNIST/model.ckpt') #saver.save(sess, 'F:/MNIST/model.ckpt',global_step=i+1) #儲存模型引數,注意把這裡改為自己的路徑,第三個引數將訓練的次數作為字尾加入到模型名字中。 #訓練之後,使用測試集進行測試,輸出最終結果 print ("test_accuracy= %g"%accuracy.eval(feed_dict={x:mnist.test.images[:100],y_:mnist.test.labels[:100],keep_prob:1})) #輸入自己的資料 else: #saver.restore(sess, "F:/MNIST/model.ckpt")#這裡使用了之前儲存的模型引數 result=imageprepare() ckpt = tf.train.get_checkpoint_state('F:/MNIST/') if ckpt and ckpt.model_checkpoint_path: saver.restore(sess, ckpt.model_checkpoint_path) print ("Model restored.") else: pass prediction=tf.argmax(y_out,1) predint=prediction.eval(feed_dict={x: [result],keep_prob: 1.0}, session=sess) print('recognize result:') print(predint[0])
6.使用自己的資料測試
def imageprepare():
"""
This function returns the pixel values.
The imput is a png file location.
"""
file_name='F:/MNIST/11.jpg'#匯入自己的圖片地址
#in terminal 'mogrify -format png *.jpg' convert jpg to png
im = Image.open(file_name).convert('L')
#im.save("/home/mzm/MNIST_recognize/sample.png")
plt.imshow(im)
plt.show()
tv = list(im.getdata()) #get pixel values
#normalize pixels to 0 and 1. 0 is pure white, 1 is pure black.
tva = [ (255-x)*1.0/255.0 for x in tv]
#print(tva)
return tva
將此段程式碼加在定義卷積和池化函式之後,將isTrain改為False,還要講自己想測試的那張圖放入imageprepare()函式中file_name的路徑下,也可以自己定義。即可執行。
我最開始是參考連結①中的程式碼,發現出現瞭如下的幾個錯誤:
1.記憶體不夠:
於是我更改了網路的大小和訓練時候的batchsize,如果你們電腦記憶體比較多就按照①中的引數來。不過看他部落格下的反應貌似效果也沒有特別好。
測試的結果一是和網路結構有關,也和網路引數有關,同時也和你自己的本身圖片有關。注意MNIST都是黑白色構成(這裡的黑色是一個0-1的浮點數,黑色越深表示數值越靠近1),如果你的測試圖片和訓練集相差很遠,那也別求準確率很高了。
2.載入模型的時候會出現Key Variable_*** not found in checkpoint的解決思路
我的錯誤:
NotFoundError: Restoring from checkpoint failed. This is most likely due to a Variable name or other graph key that is missing from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:
Key Variable_10 not found in checkpoint
[[{{node save_1/RestoreV2}} = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save_1/Const_0_0, save_1/RestoreV2/tensor_names, save_1/RestoreV2/shape_and_slices)]]
1、首先是自己定義的引數變數是否和儲存的引數變數的型別是一致的
2、在檔案下面是否有一個叫做chockpoint的東西存在
3、最後如果執行多次出現NotFoundError (see above for traceback): Key Variable_4 not found in checkpoint [[Node: save_2/RestoreV2 = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT,這種情況,請使用tf.reset_default_graph(),還有就是這句話要放在前面一點。
4、還有就是儲存好了模型,如果使用的是spyder請先關閉,之後在嘗試讀入模型如果以上都無法解決你的問題,請刪除模型之後,重新寫入,再載入試試
最開始選擇了方法4但是也是偶爾成功,還是不懂為什麼。
最後在師兄的幫助下弄懂了原因,找到了根治頑疾的方法:
我們在定義網路結構的時候定義變數選擇的是tf.Variable,“使用tf.Variable()
時,如果檢測到命名衝突,系統會自己處理。使用tf.get_variable()
時,系統不會處理衝突,而會報錯”(本句話來自https://blog.csdn.net/u012436149/article/details/53696970)。當然tf.get_variable()必須要第一個引數name,而tf.Variable()也可以不需要加引數name.這個name是tensorflow graph中的命名,如果對此有興趣移至本人的另一篇文章
TensorFlow中的name 和python程式碼中的變數名
我們載入模型前需定義的和模型一樣的網路結構,後加載的網路結構中的tf.Variable
檢測到命名衝突,會給相同的變數重新命名,導致找不到載入項,所以restore失敗。
舉個例子:
解決方法:
① 重啟IDE咯
② 在訓練前重置預設模型,即加上這句話:
tf.reset_default_graph()
③ 使用get_variable代替Variable(程式碼中的八處都需要改,注意變數的name不能一樣否則會報錯)
# w_conv1=tf.Variable(tf.truncated_normal([3,3,1,16],stddev=0.1))
w_conv1=tf.get_variable(name="v1",initializer=tf.truncated_normal([3,3,1,16],stddev=0.1))
如果你用的也是Spyder的IDE ,要在整體程式碼前加入tf.reset_default_graph(),否則你跑過一遍之後就會提示你的變數已經在預設的graph中出現,因此你需要在每次執行前清空預設graph的所有變數,以免get_variable報錯。
④ Failed to rename: F:/MNIST/model.ckpt.index.tempstate6656860424833232902 to: F:/MNIST/model.ckpt.index : Access is denied.
; Input/output error
[[{{node save/SaveV2}} = SaveV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/SaveV2/tensor_names, save/SaveV2/shape_and_slices, b1/_35, b1/Adam/_37, b1/Adam_1/_39, b2/_41, b2/Adam/_43, b2/Adam_1/_45, b3/_47, b3/Adam/_49, b3/Adam_1/_51, b4/_53, b4/Adam/_55, b4/Adam_1/_57, beta1_power/_59, beta2_power/_61, v1/_63, v1/Adam/_65, v1/Adam_1/_67, v2/_69, v2/Adam/_71, v2/Adam_1/_73, v3/_75, v3/Adam/_77, v3/Adam_1/_79, v4/_81, v4/Adam/_83, v4/Adam_1/_85)]]
刪掉上次執行生成的檔案,重新執行
3.整體程式碼
eg1:使用get_variable
# -*- coding: utf-8 -*-
"""
Created on Wed Nov 14 14:46:13 2018
@author: zhang
"""
# -*- coding: utf-8 -*-
"""
Created on Tue Nov 13 13:43:03 2018
@author: zhang
"""
from PIL import Image
import tensorflow as tf
import matplotlib.pyplot as plt
#import cv2
from tensorflow.examples.tutorials.mnist import input_data
#讀取資料
mnist = input_data.read_data_sets('MNIST_data', one_hot=True)
tf.reset_default_graph()
#構建cnn網路結構
#自定義卷積函式(後面卷積時就不用寫太多)
def conv2d(x,w):
return tf.nn.conv2d(x,w,strides=[1,1,1,1],padding='SAME')
#自定義池化函式
def max_pool_2x2(x):
return tf.nn.max_pool(x,ksize=[1,2,2,1],strides=[1,2,2,1],padding='SAME')
def imageprepare():
"""
This function returns the pixel values.
The imput is a png file location.
"""
file_name='F:/MNIST/11.jpg'#匯入自己的圖片地址
#in terminal 'mogrify -format png *.jpg' convert jpg to png
im = Image.open(file_name).convert('L')
#im.save("/home/mzm/MNIST_recognize/sample.png")
plt.imshow(im)
plt.show()
tv = list(im.getdata()) #get pixel values
#normalize pixels to 0 and 1. 0 is pure white, 1 is pure black.
tva = [ (255-x)*1.0/255.0 for x in tv]
#print(tva)
return tva
"""
This function returns the predicted integer.
The imput is the pixel values from the imageprepare() function.
"""
# Define the model (same as when creating the model file)
#設定佔位符,尺寸為樣本輸入和輸出的尺寸
x=tf.placeholder(tf.float32,[None,784])
y_=tf.placeholder(tf.float32,[None,10])
x_img=tf.reshape(x,[-1,28,28,1])
#tf.reset_default_graph()
#設定第一個卷積層和池化層
#with tf.variable_scope('model') as scope:
#w_conv1=tf.Variable(tf.truncated_normal([3,3,1,16],stddev=0.1))
w_conv1=tf.get_variable(name="v1",initializer=tf.truncated_normal([3,3,1,16],stddev=0.1))
print(w_conv1.name)
#b_conv1=tf.Variable(tf.constant(0.1,shape=[16]))
b_conv1=tf.get_variable(name="b1",initializer=tf.constant(0.1,shape=[16]))
h_conv1=tf.nn.relu(conv2d(x_img,w_conv1)+b_conv1)
h_pool1=max_pool_2x2(h_conv1)
#設定第二個卷積層和池化層
#w_conv2=tf.Variable(tf.truncated_normal([3,3,16,25],stddev=0.1))
#b_conv2=tf.Variable(tf.constant(0.1,shape=[25]))
w_conv2=tf.get_variable(name="v2",initializer=tf.truncated_normal([3,3,16,25],stddev=0.1))
b_conv2=tf.get_variable(name="b2",initializer=tf.constant(0.1,shape=[25]))
h_conv2=tf.nn.relu(conv2d(h_pool1,w_conv2)+b_conv2)
h_pool2=max_pool_2x2(h_conv2)
#設定第一個全連線層
#w_fc1=tf.Variable(tf.truncated_normal([7*7*25,516],stddev=0.1))
#b_fc1=tf.Variable(tf.constant(0.1,shape=[516]))
w_fc1=tf.get_variable(name="v3",initializer=tf.truncated_normal([7*7*25,516],stddev=0.1))
b_fc1=tf.get_variable(name="b3",initializer=tf.constant(0.1,shape=[516]))
h_pool2_flat=tf.reshape(h_pool2,[-1,7*7*25])
h_fc1=tf.nn.relu(tf.matmul(h_pool2_flat,w_fc1)+b_fc1)
#dropout(隨機權重失活)
keep_prob=tf.placeholder(tf.float32)
h_fc1_drop=tf.nn.dropout(h_fc1,keep_prob)
#設定第二個全連線層
#w_fc2=tf.Variable(tf.truncated_normal([516,10],stddev=0.1))
#b_fc2=tf.Variable(tf.constant(0.1,shape=[10]))
w_fc2=tf.get_variable(name="v4",initializer=tf.truncated_normal([516,10],stddev=0.1))
b_fc2=tf.get_variable(name="b4",initializer=tf.constant(0.1,shape=[10]))
y_out=tf.nn.softmax(tf.matmul(h_fc1_drop,w_fc2)+b_fc2)
#建立loss function,為交叉熵
loss=tf.reduce_mean(-tf.reduce_sum(y_*tf.log(y_out),reduction_indices=[1]))
#配置Adam優化器,學習速率為1e-4
train_step=tf.train.AdamOptimizer(1e-4).minimize(loss)
#建立正確率計算表示式
correct_prediction=tf.equal(tf.argmax(y_out,1),tf.argmax(y_,1))
accuracy=tf.reduce_mean(tf.cast(correct_prediction,tf.float32))
#開始喂資料,訓練
isTrain = False
#開始資料訓練
sess=tf.InteractiveSession()
saver = tf.train.Saver() # defaults to saving all variables,只儲存最後一代的模型
tf.global_variables_initializer().run()
if isTrain:
for i in range(800):
batch=mnist.train.next_batch(20)
if i%100==0:
train_accuracy=accuracy.eval(feed_dict={x:batch[0],y_:batch[1],keep_prob:1})
print ("step %d,train_accuracy= %g"%(i,train_accuracy))
train_step.run(feed_dict={x:batch[0],y_:batch[1],keep_prob:0.5})
saver.save(sess, 'F:/MNIST/model.ckpt')
#saver.save(sess, 'F:/MNIST/model.ckpt',global_step=i+1) #儲存模型引數,注意把這裡改為自己的路徑,第三個引數將訓練的次數作為字尾加入到模型名字中。
#訓練之後,使用測試集進行測試,輸出最終結果
print ("test_accuracy= %g"%accuracy.eval(feed_dict={x:mnist.test.images[:100],y_:mnist.test.labels[:100],keep_prob:1}))
else:
#saver.restore(sess, "F:/MNIST/model.ckpt")#這裡使用了之前儲存的模型引數
result=imageprepare()
ckpt = tf.train.get_checkpoint_state('F:/MNIST/')
if ckpt and ckpt.model_checkpoint_path:
saver.restore(sess, ckpt.model_checkpoint_path)
print ("Model restored.")
else:
pass
prediction=tf.argmax(y_out,1)
predint=prediction.eval(feed_dict={x: [result],keep_prob: 1.0}, session=sess)
print('recognize result:')
print(predint[0])
eg2:使用variable:
# -*- coding: utf-8 -*-
"""
Created on Wed Nov 14 14:46:13 2018
@author: zhang
"""
# -*- coding: utf-8 -*-
"""
Created on Tue Nov 13 13:43:03 2018
@author: zhang
"""
from PIL import Image
import tensorflow as tf
import matplotlib.pyplot as plt
#import cv2
from tensorflow.examples.tutorials.mnist import input_data
#讀取資料
mnist = input_data.read_data_sets('MNIST_data', one_hot=True)
tf.reset_default_graph()
#構建cnn網路結構
#自定義卷積函式(後面卷積時就不用寫太多)
def conv2d(x,w):
return tf.nn.conv2d(x,w,strides=[1,1,1,1],padding='SAME')
#自定義池化函式
def max_pool_2x2(x):
return tf.nn.max_pool(x,ksize=[1,2,2,1],strides=[1,2,2,1],padding='SAME')
def imageprepare():
"""
This function returns the pixel values.
The imput is a png file location.
"""
file_name='F:/MNIST/11.jpg'#匯入自己的圖片地址
#in terminal 'mogrify -format png *.jpg' convert jpg to png
im = Image.open(file_name).convert('L')
#im.save("/home/mzm/MNIST_recognize/sample.png")
plt.imshow(im)
plt.show()
tv = list(im.getdata()) #get pixel values
#normalize pixels to 0 and 1. 0 is pure white, 1 is pure black.
tva = [ (255-x)*1.0/255.0 for x in tv]
#print(tva)
return tva
"""
This function returns the predicted integer.
The imput is the pixel values from the imageprepare() function.
"""
# Define the model (same as when creating the model file)
#設定佔位符,尺寸為樣本輸入和輸出的尺寸
x=tf.placeholder(tf.float32,[None,784])
y_=tf.placeholder(tf.float32,[None,10])
x_img=tf.reshape(x,[-1,28,28,1])
#tf.reset_default_graph()
#設定第一個卷積層和池化層
#with tf.variable_scope('model') as scope:
w_conv1=tf.Variable(tf.truncated_normal([3,3,1,16],stddev=0.1))
#w_conv1=tf.get_variable(name="v1",initializer=tf.truncated_normal([3,3,1,16],stddev=0.1))
print(w_conv1.name)
b_conv1=tf.Variable(tf.constant(0.1,shape=[16]))
#b_conv1=tf.get_variable(name="b1",initializer=tf.constant(0.1,shape=[16]))
h_conv1=tf.nn.relu(conv2d(x_img,w_conv1)+b_conv1)
h_pool1=max_pool_2x2(h_conv1)
#設定第二個卷積層和池化層
w_conv2=tf.Variable(tf.truncated_normal([3,3,16,25],stddev=0.1))
b_conv2=tf.Variable(tf.constant(0.1,shape=[25]))
#w_conv2=tf.get_variable(name="v2",initializer=tf.truncated_normal([3,3,16,25],stddev=0.1))
#b_conv2=tf.get_variable(name="b2",initializer=tf.constant(0.1,shape=[25]))
h_conv2=tf.nn.relu(conv2d(h_pool1,w_conv2)+b_conv2)
h_pool2=max_pool_2x2(h_conv2)
#設定第一個全連線層
w_fc1=tf.Variable(tf.truncated_normal([7*7*25,516],stddev=0.1))
b_fc1=tf.Variable(tf.constant(0.1,shape=[516]))
#w_fc1=tf.get_variable(name="v3",initializer=tf.truncated_normal([7*7*25,516],stddev=0.1))
#b_fc1=tf.get_variable(name="b3",initializer=tf.constant(0.1,shape=[516]))
h_pool2_flat=tf.reshape(h_pool2,[-1,7*7*25])
h_fc1=tf.nn.relu(tf.matmul(h_pool2_flat,w_fc1)+b_fc1)
#dropout(隨機權重失活)
keep_prob=tf.placeholder(tf.float32)
h_fc1_drop=tf.nn.dropout(h_fc1,keep_prob)
#設定第二個全連線層
w_fc2=tf.Variable(tf.truncated_normal([516,10],stddev=0.1))
b_fc2=tf.Variable(tf.constant(0.1,shape=[10]))
#w_fc2=tf.get_variable(name="v4",initializer=tf.truncated_normal([516,10],stddev=0.1))
#b_fc2=tf.get_variable(name="b4",initializer=tf.constant(0.1,shape=[10]))
y_out=tf.nn.softmax(tf.matmul(h_fc1_drop,w_fc2)+b_fc2)
#建立loss function,為交叉熵
loss=tf.reduce_mean(-tf.reduce_sum(y_*tf.log(y_out),reduction_indices=[1]))
#配置Adam優化器,學習速率為1e-4
train_step=tf.train.AdamOptimizer(1e-4).minimize(loss)
#建立正確率計算表示式
correct_prediction=tf.equal(tf.argmax(y_out,1),tf.argmax(y_,1))
accuracy=tf.reduce_mean(tf.cast(correct_prediction,tf.float32))
#開始喂資料,訓練
isTrain = False
#開始資料訓練
sess=tf.InteractiveSession()
saver = tf.train.Saver() # defaults to saving all variables,只儲存最後一代的模型
tf.global_variables_initializer().run()
if isTrain:
for i in range(800):
batch=mnist.train.next_batch(20)
if i%100==0:
train_accuracy=accuracy.eval(feed_dict={x:batch[0],y_:batch[1],keep_prob:1})
print ("step %d,train_accuracy= %g"%(i,train_accuracy))
train_step.run(feed_dict={x:batch[0],y_:batch[1],keep_prob:0.5})
saver.save(sess, 'F:/MNIST/model.ckpt')
#saver.save(sess, 'F:/MNIST/model.ckpt',global_step=i+1) #儲存模型引數,注意把這裡改為自己的路徑,第三個引數將訓練的次數作為字尾加入到模型名字中。
#訓練之後,使用測試集進行測試,輸出最終結果
print ("test_accuracy= %g"%accuracy.eval(feed_dict={x:mnist.test.images[:100],y_:mnist.test.labels[:100],keep_prob:1}))
else:
#saver.restore(sess, "F:/MNIST/model.ckpt")#這裡使用了之前儲存的模型引數
result=imageprepare()
ckpt = tf.train.get_checkpoint_state('F:/MNIST/')
if ckpt and ckpt.model_checkpoint_path:
saver.restore(sess, ckpt.model_checkpoint_path)
print ("Model restored.")
else:
pass
prediction=tf.argmax(y_out,1)
predint=prediction.eval(feed_dict={x: [result],keep_prob: 1.0}, session=sess)
print('recognize result:')
print(predint[0])
結果展示:
我最開始輸入的是這個28*28的影象:
但是總是識別不對
於是我對影象進行了二值化,閾值選擇的是100:
from PIL import Image
# load a color image
im = Image.open('F:/MNIST/11.jpg' )#當前目錄建立picture資料夾
# convert to grey level image
Lim = im.convert('L' )
#Lim.save('pice.jpg' )
# setup a converting table with constant threshold
threshold = 100
table = []
for i in range(256):
if i < threshold:
table.append(0)
else:
table.append(1)
# convert to binary image by the table
bim = Lim.point(table, '1' )
bim.save('F:/MNIST/111.jpg' )
二值化結果如下:
訓練和測試結果如下:
結束~快樂的跑程式碼去吧
本文程式碼參考幾篇文章,可以直接使用。
參考連結: