TensorBoard 簡介

官網上給的定義是：The computations you'll use tensorflow for （like training a massive deep neural network ) can be complex and confusing . To make it easier to understand, debug, and optimize tensorflow programs,we're included a suite of visualization tools called tensorboard.

也就是說，使用tensorflow這個框架搭建的深度神經網路會很複雜，出錯了也很難除錯，為了便於使用者理解你所訓練的網路，也便於除錯程式，他們推出了tensorboard這個工具。

那麼它到底能幹什麼呢？You can use tensorboard to visualize your tensorflow graph, plot quantitative metrics about the execution of your graph, and show additional data like images that pass through it.

它可以視覺化你的圖結構，顯示類似準確圖，代價值等等指標值，顯示引數隨著迭代次數的變化，以及它的分佈圖，顯示你訓練使用的資料，可以顯示很多理解和除錯你的網路所需要的東西。要顯示什麼也是可以在程式碼中進行控制的，可以這樣理解，tensorboard並不是只要你訓練了一個網路就會自動生成對應的board，它需要你在程式碼中加入相應的元件才可以顯示到tensorboard中。

tensorboard是依賴tensorflow的，官網說：tensorboard operates by reading tensorflow events files ,which contain summary data that you can generate when running tensorflow . tensorboard的執行通過讀取tensorflow的事件檔案，檔案中包含了所有的tensorflow中生成的總資料。tensorflow在執行的時候會產生很多很多的資料，它也提供了函式去獲取這些資料並且儲存下來，寫到events file中，然後tensorboard讀取這些檔案，形成視覺化的圖表展示。

比如說，你要訓練一個CNN網路識別MNIST數字，你想要記錄學習因子隨著迭代次數的改變，你可以對節點執行tf.summary.scalar 操作，然後給每個一summary一個標識名稱，也就是 為summary新增一個tag .

當然不止會用到一個summary，如果知道cost的變化，或者引數w和b的變化情況，也可以分別給這些節點新增summary，定義完所有的summary節點後，需要把他們都結合成一個節點 。

use tf.summary.merge_all to combine them into a singel op that generates all the summary data.

在執行summary整合後，會得到一個summary protobuf，將這個哥protobuf通過tf.summary.FileWriter 寫入硬碟儲存下來。（protobuf全稱是protocal buffer，是谷歌開發的一種資料交換的格式）

if you want you could run the merged summary op every sigle steep ,and record a ton of training data . that 's likely to be more data than you need, a better way is to run the op every n steops

在程式執行的時候，可以選擇每迭代一次就執行一次合併過的summary op，然後寫入磁碟，這樣就會有非常多的資料，超過你正常需要的資料，所以可以選擇每幾個迭代進行一次op。

在cmd中啟動tensorboard： tensorboard --logdir = 'path to the log directory' ，這個log資料夾需要自己新建，最好放在本專案資料夾下。

這是官網給出的一個示例，下面的MNIST的例子個人認為要比這個更好理解，因為它就是按照流程從上到下完成的，官網這裡還做了函式封裝，使用了drop out，稍微難懂一點，可以先看下面的例子，明白這個流程是怎麼走的，再看官網的例子就會清晰很多。

#這個函式的功能其實就是把一些常用的對資料處理的操作封裝到一起
#對var進行求平均值，方差，最大最小值，以及直方圖
def variable_summaries(var):
  """Attach a lot of summaries to a Tensor (for TensorBoard visualization)."""
  with tf.name_scope('summaries'):
    mean = tf.reduce_mean(var)
    tf.summary.scalar('mean', mean)

    with tf.name_scope('stddev'):
      stddev = tf.sqrt(tf.reduce_mean(tf.square(var - mean)))

    tf.summary.scalar('stddev', stddev)
    tf.summary.scalar('max', tf.reduce_max(var))
    tf.summary.scalar('min', tf.reduce_min(var))
    tf.summary.histogram('histogram', var)


#定義一個簡單網路，有一個隱藏層 
def nn_layer(input_tensor, input_dim, output_dim, layer_name, act=tf.nn.relu):
  """
  Reusable code for making a simple neural net layer.
  It does a matrix multiply, bias add, and then uses relu to nonlinearize.
  It also sets up name scoping so that the resultant graph is easy to read,
  and adds a number of summary ops.

  """

  # Adding a name scope ensures logical grouping of the layers in the graph.
  with tf.name_scope(layer_name):
    # This Variable will hold the state of the weights for the layer
    with tf.name_scope('weights'):
      weights = weight_variable([input_dim, output_dim])
      variable_summaries(weights)
    with tf.name_scope('biases'):
      biases = bias_variable([output_dim])
      variable_summaries(biases)
    with tf.name_scope('Wx_plus_b'):
      preactivate = tf.matmul(input_tensor, weights) + biases
      tf.summary.histogram('pre_activations', preactivate)

    activations = act(preactivate, name='activation')
    tf.summary.histogram('activations', activations)

    return activations


hidden1 = nn_layer(x, 784, 500, 'layer1')

#使用drop out
with tf.name_scope('dropout'):
  keep_prob = tf.placeholder(tf.float32)
  tf.summary.scalar('dropout_keep_probability', keep_prob)
  dropped = tf.nn.dropout(hidden1, keep_prob)

# Do not apply softmax activation yet, see below.
y = nn_layer(dropped, 500, 10, 'layer2', act=tf.identity)

with tf.name_scope('cross_entropy'):
  # The raw formulation of cross-entropy,
  #
  # tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(tf.softmax(y)),
  #                               reduction_indices=[1]))
  #
  # can be numerically unstable.
  #
  # So here we use tf.losses.sparse_softmax_cross_entropy on the
  # raw logit outputs of the nn_layer above.
  with tf.name_scope('total'):
    cross_entropy = tf.losses.sparse_softmax_cross_entropy(labels=y_, logits=y)
tf.summary.scalar('cross_entropy', cross_entropy)

with tf.name_scope('train'):
  train_step = tf.train.AdamOptimizer(FLAGS.learning_rate).minimize(cross_entropy)

with tf.name_scope('accuracy'):
  with tf.name_scope('correct_prediction'):
    correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))
  with tf.name_scope('accuracy'):
    accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
tf.summary.scalar('accuracy', accuracy)

# Merge all the summaries and write them out to /tmp/mnist_logs (by default)
merged = tf.summary.merge_all()
train_writer = tf.summary.FileWriter(FLAGS.summaries_dir + '/train',
                                      sess.graph)
test_writer = tf.summary.FileWriter(FLAGS.summaries_dir + '/test')
tf.global_variables_initializer().run()

# Train the model, and also write summaries.
# Every 10th step, measure test-set accuracy, and write test summaries
# All other steps, run train_step on training data, & add training summaries

def feed_dict(train):
  """Make a TensorFlow feed_dict: maps data onto Tensor placeholders."""
  if train or FLAGS.fake_data:
    xs, ys = mnist.train.next_batch(100, fake_data=FLAGS.fake_data)
    k = FLAGS.dropout
  else:
    xs, ys = mnist.test.images, mnist.test.labels
    k = 1.0
  return {x: xs, y_: ys, keep_prob: k}

for i in range(FLAGS.max_steps):
  if i % 10 == 0:  # Record summaries and test-set accuracy
    summary, acc = sess.run([merged, accuracy], feed_dict=feed_dict(False))
    test_writer.add_summary(summary, i)
    print('Accuracy at step %s: %s' % (i, acc))
  else:  # Record train set summaries, and train
    summary, _ = sess.run([merged, train_step], feed_dict=feed_dict(True))
    train_writer.add_summary(summary, i)

MNIST示例

#導包 獲取資料
import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_data

mnist = input_data.read_data_sets('MNIST_data',one_hot = True)

#定義批次和一共有多少批次
batch_size = 100
n_batch = mnist.train.num_examples // batch_size


#引數概要 官網案例中的函式
def variable_summaries(var):
    with tf.name_scope('summaries'):
        mean = tf.reduce_mean(var) # 平均值
        tf.summary.scalar('mean',mean) # 記錄這個值 並且給出名字
        with tf.name_scope('stddev'):
            stddev = tf.sqrt(tf.reduce_mean(tf.square(var - mean)))
        tf.summary.scalar('stddev',stddev) #標準差
        tf.summary.scalar('max',tf.reduce_max(var))
        tf.summary.scalar('min',tf.reduce_min(var))
        tf.summary.histogram('histogram',var) # 直方圖

#定義訓練資料X，Y的名稱空間
#名稱空間 可以簡單理解為 給它在tensorboard上的graph中起個名字
with tf.name_scope('input'):
    #定義兩個placeholder
    x = tf.placeholder(tf.float32,[None,784],name = 'x-input')
    y = tf.placeholder(tf.float32,[None,10],name = 'y-input')


#因為沒有隱藏層，其實這一層就是輸出層
with tf.name_scope('layer'):
    with tf.name_scope('weight'):
        W = tf.Variable(tf.zeros([784,10]),name = 'W')
        variable_summaries(W)
    with tf.name_scope('biases'):
        b = tf.Variable(tf.zeros([1,10]),name = 'b')
        b = tf.squeeze(b)
        variable_summaries(b)
    with tf.name_scope('Z'):
        Z = tf.matmul(x,W)+b
    with tf.name_scope('softmax'):
        #官網這裡是沒有呼叫softmax的，包括吳恩達老師留的作業也說不要調
        #tf將softmax和計算cost繫結在一起，在計算cost的時候就會執行這個函式
        #但網上有些tf的教程是呼叫了的，兩種我都試過，結果在其他條件不變的情況下，不調準確率高
        prediction = Z 
  
      
#損失值
with tf.name_scope('cost'):
    cost=tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=y,  logits=prediction))
    tf.summary.scalar('cost',cost)
    
with tf.name_scope('train'):
    train = tf.train.GradientDescentOptimizer(0.2).minimize(cost)

with tf.name_scope('accuracy'):
    with tf.name_scope('correct_prediction'):
        correct_prediction = tf.equal(tf.argmax(y,1),tf.argmax(prediction,1))
    with tf.name_scope('accuracy'):
        accuracy = tf.reduce_mean(tf.cast(correct_prediction,tf.float32))
        tf.summary.scalar('accuracy',accuracy)


#合併所有的summary
merge = tf.summary.merge_all()
    

with tf.Session() as sess:
    #前面定義的變數，就需要先初始化變數
    sess.run(tf.global_variables_initializer())
    
    #將graph這個圖寫到當前路勁的logs資料夾下
    writer = tf.summary.FileWriter('logs/',sess.graph)
    
    for i in range(51): #迭代100次
        for batch in range(n_batch):  
            
            batch_xs,batch_ys = mnist.train.next_batch(batch_size)
            summary,_=sess.run([merge,train],feed_dict={x:batch_xs,y:batch_ys})
        
        writer.add_summary(summary,i)
        train_acc = sess.run(accuracy,feed_dict={x:mnist.train.images,y:mnist.train.labels})
        test_acc = sess.run(accuracy,feed_dict={x:mnist.test.images,y:mnist.test.labels})
        print('after ',i, 'the train accuracy is ',train_acc,',the test accuracy is ',test_acc)  

開啟tensorboard，介紹一下他的介面：

SCALARS中放了很多這樣的表，準確度，代價函式，以及引數值，都是隨著迭代次數的變化而變化

GRAPHS中放著網路的圖結構，可以將某個子模組點開看細節，也可以將子模組脫離整體，或者將脫離整體的模組整合在一起，通過單擊右鍵就可以看到操作提示。

DISTRIBUTION中放了引數w和b的分佈圖：

HISTOGRAM中放了引數的直方圖：