1. 程式人生 > >99、Tensorflow Serving 實現模型的部署

99、Tensorflow Serving 實現模型的部署

昨晚終於實現了Tensorflow模型的部署 使用TensorFlow Serving

1、使用Docker 獲取Tensorflow Serving的映象,Docker在國內的需要將映象的Repository地址設定為阿里雲的加速地址,這個大家可以自己去CSDN上面找

然後啟動docker

2、使用Tensorflow 的 SaveModelBuilder儲存Tensorflow的計算圖模型,並且設定Signature,

Signature主要用來標識模型的輸入值的名稱和型別

builder = saved_model_builder.SavedModelBuilder(export_path)
        
        
        classification_inputs = utils.build_tensor_info(cnn.input_x)
        classification_dropout_keep_prob = utils.build_tensor_info(cnn.dropout_keep_prob)
        classification_outputs_classes = utils.build_tensor_info(prediction_classes)
        classification_outputs_scores = utils.build_tensor_info(cnn.scores)

   
        classification_signature = signature_def_utils.build_signature_def(
        inputs={signature_constants.CLASSIFY_INPUTS: classification_inputs,
                     signature_constants.CLASSIFY_INPUTS:classification_dropout_keep_prob
                     },
        outputs={
              signature_constants.CLASSIFY_OUTPUT_CLASSES:
              classification_outputs_classes,
              signature_constants.CLASSIFY_OUTPUT_SCORES:
              classification_outputs_scores
         },
         method_name=signature_constants.CLASSIFY_METHOD_NAME)

        tensor_info_x = utils.build_tensor_info(cnn.input_x)
        tensor_info_y = utils.build_tensor_info(cnn.predictions)
        tensor_info_dropout_keep_prob = utils.build_tensor_info(cnn.dropout_keep_prob)

        prediction_signature = signature_def_utils.build_signature_def(
               inputs={'inputX': tensor_info_x,
                            'input_dropout_keep_prob':tensor_info_dropout_keep_prob},
               outputs={'predictClass': tensor_info_y},
        method_name=signature_constants.PREDICT_METHOD_NAME)

        legacy_init_op = tf.group(tf.tables_initializer(), name='legacy_init_op')
  
        #add the sigs to the servable
        builder.add_meta_graph_and_variables(
                sess, [tag_constants.SERVING],
                signature_def_map={
                    'textclassified':
                    prediction_signature,
                    signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY:
                    classification_signature,
         },
         legacy_init_op=legacy_init_op)
         #save it!
        builder.save(True)

儲存之後的計算圖的結構可以從下面這裡看見,下面這裡只給出模型的signature部分,因為signature裡面定義了你到時候call restful介面的引數名稱和型別

signature_def {
    key: "serving_default"
    value {
      inputs {
        key: "inputs"
        value {
          name: "dropout_keep_prob:0"
          dtype: DT_FLOAT
          tensor_shape {
            unknown_rank: true
          }
        }
      }
      outputs {
        key: "classes"
        value {
          name: "index_to_string_Lookup:0"
          dtype: DT_STRING
          tensor_shape {
            dim {
              size: 1
            }
          }
        }
      }
      outputs {
        key: "scores"
        value {
          name: "output/scores:0"
          dtype: DT_FLOAT
          tensor_shape {
            dim {
              size: -1
            }
            dim {
              size: 2
            }
          }
        }
      }
      method_name: "tensorflow/serving/classify"
    }
  }
  signature_def {
    key: "textclassified"
    value {
      inputs {
        key: "inputX"
        value {
          name: "input_x:0"
          dtype: DT_INT32
          tensor_shape {
            dim {
              size: -1
            }
            dim {
              size: 40
            }
          }
        }
      }
      inputs {
        key: "input_dropout_keep_prob"
        value {
          name: "dropout_keep_prob:0"
          dtype: DT_FLOAT
          tensor_shape {
            unknown_rank: true
          }
        }
      }
      outputs {
        key: "predictClass"
        value {
          name: "output/predictions:0"
          dtype: DT_INT64
          tensor_shape {
            dim {
              size: -1
            }
          }
        }
      }
      method_name: "tensorflow/serving/predict"
    }
  }
}

從上面的Signature定義可以看出 到時候call restfull 介面需要傳兩個引數,

int32型別的名稱為inputX引數

float型別名稱為input_drop_out_keep_prob的引數

上面的protocol buffer 中

textclassified表示使用TextCnn卷積神經網路來進行預測,然後預測功能的名稱叫做textclassified

 3、將模型部署到Tensorflow Serving 上面

首先把模型通過工具傳輸到docker上面

模型的結構如下

傳到docker上面,然後在外邊套一個資料夾名字起位模型的名字,叫做

text_classified_model
然後執行下面這條命令執行tensorflow/serving
docker run -p 8500:8500 --mount type=bind,source=/home/docker/model/text_classified_model,target=/mo
dels/text_classified_model -e MODEL_NAME=text_classified_model -t tensorflow/serving
source表示模型在docker上面的路徑
target表示模型在docker中TensorFlow/serving container上面的路徑

 然後輸入如下所示

另一個是gRPC介面埠是8500

gRPC是HTTP/2協議,REST API 是HTTP/1協議

區別是gRPC只有POST/GET兩種請求方式

REST API還有其餘很多種 列如 PUT/DELETE 等

4、客戶端呼叫gPRC介面

需要傳兩個引數,

一個是

inputX

另一個是

input_dropout_keep_prob
'''
Created on 2018年10月17日

@author: 95890
'''

"""Send text to tensorflow serving and gets result
"""


# This is a placeholder for a Google-internal import.

from grpc.beta import implementations
import tensorflow as tf
import data_helpers
from tensorflow_serving.apis import predict_pb2
from tensorflow_serving.apis import prediction_service_pb2
from tensorflow.contrib import learn
import numpy as np


tf.flags.DEFINE_string("positive_data_file", "./data/rt-polaritydata/rt-polarity.pos", "Data source for the positive data.")
tf.flags.DEFINE_string("negative_data_file", "./data/rt-polaritydata/rt-polarity.neg", "Data source for the negative data.")
tf.flags.DEFINE_string('server', '192.168.99.100:8500',
                           'PredictionService host:port')
FLAGS = tf.flags.FLAGS

x_text=[]
y=[]
max_document_length=40


def main(_):


  testStr =["wisegirls is its low-key quality and genuine"]

  
  if x_text.__len__()==0:
      x_text, y = data_helpers.load_data_and_labels(FLAGS.positive_data_file, FLAGS.negative_data_file)
      max_document_length = max([len(x.split(" ")) for x in x_text])

  vocab_processor = learn.preprocessing.VocabularyProcessor(max_document_length)
  vocab_processor.fit(x_text)
  x = np.array(list(vocab_processor.fit_transform(testStr)))
  
  host, port = FLAGS.server.split(':')
  channel = implementations.insecure_channel(host, int(port))
  stub = prediction_service_pb2.beta_create_PredictionService_stub(channel)
  request = predict_pb2.PredictRequest()
  request.model_spec.name = "text_classified_model"
  request.model_spec.signature_name = 'textclassified'
  dropout_keep_prob = np.float(1.0)
  
  request.inputs['inputX'].CopyFrom(
  tf.contrib.util.make_tensor_proto(x, shape=[1,40],dtype=np.int32))
  
  request.inputs['input_dropout_keep_prob'].CopyFrom(
  tf.contrib.util.make_tensor_proto(dropout_keep_prob, shape=[1],dtype=np.float))
  
  result = stub.Predict(request, 10.0)  # 10 secs timeout
  print(result)


if __name__ == '__main__':
  tf.app.run()
outputs {
  key: "predictClass"
  value {
    dtype: DT_INT64
    tensor_shape {
      dim {
        size: 1
      }
    }
    int64_val: 1
  }
}
model_spec {
  name: "text_classified_model"
  version {
    value: 1
  }
  signature_name: "textclassified"
}

從上面的結果可以看出,我們傳入了一句話

wisegirls is its low-key quality and genuine

分類的結果

predictClass
int64_val: 1

分成第一類

這個真的是神經網路的部署呀。

啦啦啦 ,  Tensorflow真的很牛,上至瀏覽器,下到手機,一次訓練,一次匯出。處處執行。

沒有不敢想,只有不敢做

 The Full version can be find here

https://github.com/weizhenzhao/TextCNN_Tensorflow_Serving/tree/master

Thanks

WeiZhen