variable scope

tensorflow 為了更好的管理變數,提供了variable scope機制
官方解釋:
Variable scope object to carry defaults to provide to get_variable.

Many of the arguments we need for get_variable in a variable store are most easily handled with a context. This object is used for the defaults.

Attributes:

• name: name of the current scope, used as prefix in get_variable.
• initializer: 傳給get_variable的預設initializer.如果get_variable的時候指定了initializer,那麼將覆蓋這個預設的initializer.
• regularizer: 傳給get_variable的預設regulizer.
• reuse: Boolean or None, setting the reuse in get_variable.
• caching_device: string, callable, or None: the caching device passed to get_variable.
• partitioner: callable or None: the partitioner passed to get_variable.
• custom_getter: default custom getter passed to get_variable.
• name_scope: The name passed to tf.name_scope.
• dtype: default type passed to get_variable (defaults to DT_FLOAT).

regularizer引數的作用是給在本variable_scope下建立的weights加上正則項.這樣我們就可以不同variable_scope下的引數加不同的正則項了.

可以看出,用variable scope管理get_varibale是很方便的

如何確定 get_variable 的 prefixed name

首先, variable scope是可以巢狀的:

with variable_scope.variable_scope("tet1"):
    var3 = tf.get_variable("var3",shape=[2],dtype=tf.float32)
    print var3.name
    with variable_scope.variable_scope("tet2"):
        var4 = tf.get_variable("var4",shape=[2],dtype=tf.float32)
        print var4.name
#輸出為****************
#tet1/var3:0
#tet1/tet2/var4:0
#*********************

get_varibale.name 以建立變數的 scope 作為名字的prefix

def te2():
    with variable_scope.variable_scope("te2"):
        var2 = tf.get_variable("var2",shape=[2], dtype=tf.float32)
        print var2.name
        def te1():
            with variable_scope.variable_scope("te1"):
                var1 = tf.get_variable("var1", shape=[2], dtype=tf.float32)
            return var1
        return te1() #在scope te2 內呼叫的
res = te2()
print res.name
#輸出*********************
#te2/var2:0
#te2/te1/var1:0
#************************

觀察和上個程式的不同

def te2():
    with variable_scope.variable_scope("te2"):
        var2 = tf.get_variable("var2",shape=[2], dtype=tf.float32)
        print var2.name
        def te1():
            with variable_scope.variable_scope("te1"):
                var1 = tf.get_variable("var1", shape=[2], dtype=tf.float32)
            return var1
    return te1()  #在scope te2外面呼叫的
res = te2()
print res.name
#輸出*********************
#te2/var2:0
#te1/var1:0
#************************

還有需要注意一點的是tf.variable_scope("name") 與 tf.variable_scope(scope)的區別，看下面程式碼

程式碼1

import tensorflow as tf
with tf.variable_scope("scope"):
    tf.get_variable("w",shape=[1])#這個變數的name是 scope/w
    with tf.variable_scope("scope"):
        tf.get_variable("w", shape=[1]) #這個變數的name是 scope/scope/w
# 這兩個變數的名字是不一樣的，所以不會產生衝突

程式碼2

import tensorflow as tf
with tf.variable_scope("yin"):
    tf.get_variable("w",shape=[1])
    scope = tf.get_variable_scope()#這個變數的name是 scope/w
    with tf.variable_scope(scope):#這種方式設定的scope，是用的外部的scope
        tf.get_variable("w", shape=[1])#這個變數的name也是 scope/w
# 兩個變數的名字一樣，會報錯

共享變數

共享變數的前提是，變數的名字是一樣的，變數的名字是由變數名和其scope字首一起構成， tf.get_variable_scope().reuse_variables() 是允許共享當前scope下的所有變數。reused variables可以看同一個節點

with tf.variable_scope("level1"):
    tf.get_variable("w",shape=[1])
    scope = tf.get_variable_scope()
    with tf.variable_scope("level2"):
        tf.get_variable("w", shape=[1])

with tf.variable_scope("level1", reuse=True): #即使巢狀的variable_scope也會被reuse
    tf.get_variable("w",shape=[1])
    scope = tf.get_variable_scope()
    with tf.variable_scope("level2"):
        tf.get_variable("w", shape=[1])

其它

tf.get_variable_scope() :獲取當前scope
tf.get_variable_scope().reuse_variables() 共享變數