1. 程式人生 > >part-aligned系列論文:1707.Deeply-Learned Part-Aligned Representations for Person Re-Identification 論文筆記

part-aligned系列論文:1707.Deeply-Learned Part-Aligned Representations for Person Re-Identification 論文筆記

Deeply-Learned Part-Aligned Representations for Person Re-Identification一種超簡單有效的行人對齊識別網路!
inspired by attention model,propose a part-aligned human representation基於特徵圖
FCN全卷積提取特徵圖部分應用GoogLeNet部分網路,特徵圖輸入到K個分支的part net,整個model learnt from CUHK03,then 在其他資料集上fine-tune測試
be efficient in online matching and cheap in storage
採用triplet loss指導模型訓練,無需人體部位標籤資訊
在Market-1501,CUHK03,CUHK01和VIPeR取得了最好的效能效果
公式描述部分很容易看懂
設計多分支partnet(Part Map detector(一個卷積+sigmoid代替softmax)+全域性均值池化+線性維度縮減層(1個卷積層)),最後各part特徵被L2 normalization後進行特徵串接,符合行人由上倒下的序列關係,串接特徵更具區分性,採用簡單的
分spatial partition和Body part partition,該論文屬於後者歐氏距離做匹配度量
作者的方法側重在特徵的更有效的表達(deeply learnt feature representation),同其他方法(如PIE)可一定程度上結合度量學習方法提升效能

其他基於部件對齊的論文
CPM:S. Wei, V. Ramakrishna, T. Kanade, and Y. Sheikh. Convolutional pose machines. In CVPR, pages 4724–4732, 2016.
PIE:L. Zheng, Y. Huang, H. Lu, and Y. Yang. Pose invariant embedding for deep person re-identification. CoRR,abs/1701.07732, 2017(w/o using KISSME,uses ResNet-50,complicated fusion scheme)

使用複雜的匹配子網路解決對齊問題的論文
F. Wang, W. Zuo, L. Lin, D. Zhang, and L. Zhang. Joint learning of single-image and cross-image representations for person re-identification. In CVPR, June 2016
IDLA:E. Ahmed, M. Jones, and T. K. Marks. An improved deep learning architecture for person re-identification. In CVPR, 2015.
Deepreid

:W. Li, R. Zhao, T. Xiao, and X. Wang. Deepreid: Deep filter pairing neural network for person re-identification. In CVPR, 2014.
DCSL:Y. Zhang, X. Li, L. Zhao, and Z. Zhang. Semantics-aware deep correspondence structure learning for robust person reidentification. In IJCAI, pages 3545–3551, 2016.
PersonNet:L. Wu, C. Shen, and A. van den Hengel. Personnet: Personre-identification with deep convolutional neural networks. CoRR, abs/1601.07255, 2016.
Gated S-CNN:R. R. Varior, M. Haloi, and G. Wang. Gated siamese convolutional neural network architecture for human reidentification. In ECCV, pages 791–808, 2016
**SIR-CIR:**F. Wang, W. Zuo, L. Lin, D. Zhang, and L. Zhang. Joint learning of single-image and cross-image representations for person re-identification. In CVPR, June 2016
Deep Ranking:S.-Z. Chen, C.-C. Guo, and J.-H. Lai. Deep ranking for person re identification via joint representation learning. IEEE Trans. Image Processing, 25(5):2353–2367, 2016
SSDAL :C. Su, S. Zhang, J. Xing, W. Gao, and Q. Tian. Deep attributes driven multi-camera person re-identification. In ECCV, pages 475–491, 2016.
其他相關論文
作者參考的採用了LSTM注意網路在特徵圖階段進行匹配的論文:
R. R. Varior, M. Haloi, and G. Wang. Gated siamese convolutional neural network architecture for human reidentification. In ECCV, pages 791–808, 2016.
作者主要參考的其他兩篇注意網路論文:
H. Liu, J. Feng, M. Qi, J. Jiang, and S. Yan. End-to-end comparative attention networks for person re-identification. CoRR, abs/1606.04404, 2016.(spatial attention model)
K. Xu, J. Ba, R. Kiros, K. Cho, A. C. Courville, R. Salakhutdinov, R. S. Zemel, and Y. Bengio. Show, attend and tell:Neural image caption generation with visual attention. InICML, pages 2048–2057, 2015(注意機制模型,作者參考的一篇論文)
作者著重參考的其他相關的論文(need to train a part/pose segmentation or detection model from the labeled part mask/box or pose ground-truth,):
S. Bak, E. Corvee, F. Br ´ emond, and M. Thonnat. Person ´ re-identification using spatial covariance regions of human body parts. In AVSS, pages 435–440, 2010.
Y. Xu, L. Lin, W. Zheng, and X. Liu. Human re-identification by matching compositional template with cluster sampling. In ICCV, pages 3152–3159, 2013.
D. S. Cheng, M. Cristani, M. Stoppa, L. Bazzani, and V. Murino. Custom pictorial structures for re-identification.In BMVC, pages 1–11, 2011
L. Zheng, Y. Huang, H. Lu, and Y. Yang. Pose invariant embedding for deep person re-identification. CoRR, abs/1701.07732, 2017.
similarity/metric learning techniques for handling pose misalignment
D. Chen, Z. Yuan, G. Hua, N. Zheng, and J. Wang. Similarity learning on an explicit polynomial kernel feature map for person re-identification. In CVPR, pages 1565–1573, 2015.
D. Chen, Z. Yuan, B. Chen, and N. Zheng. Similarity learning with spatial constraints for person re-identification. In CVPR, June 2016.
Y. Shen, W. Lin, J. Yan, M. Xu, J. Wu, and J. Wang. Person re-identification with correspondence structure learning. In ICCV, 2015
partial person matching:W.-S. Zheng, X. Li, T. Xiang, S. Liao, J. Lai, and S. Gong.Partial person re-identification. In ICCV, December 2015.

以下論文基於假設:各bbox中行人影象各空間區域的空間分佈和行人姿態及相應位置是相似的(實際中,bbox被detector檢測,假設很難滿足,即spatial partition is not well aligned with human body parts)
box horizontal stripes
D. Yi, Z. Lei, S. Liao, and S. Z. Li. Deep metric learning for person re-identification. In ICLR, 2014.
D. Cheng, Y. Gong, S. Zhou, J. Wang, and N. Zheng. Person re identification by multi-channel parts-based cnn with improved triplet loss function. In CVPR, June 2016 R. R. Varior, B. Shuai, J. Lu, D. Xu, and G. Wang. A siamese long short-term memory architecture for human reidentification. In ECCV, pages 135–153, 2016.

box grids
W. Li, R. Zhao, T. Xiao, and X. Wang. Deepreid: Deep filter pairing neural network for person re-identification. In CVPR, 2014.(subsequent complex matching techniques很小程度上來消除誤對齊)
E. Ahmed, M. Jones, and T. K. Marks. An improved deep learning architecture for person re-identification. In CVPR, 2015.(subsequent complex matching techniques很小程度上來消除誤對齊)

論文介紹

簡介
通過閱讀作者的論文,大致可將Part-Aligned分成四類,即基於mask掩膜的空間條紋(spatial strips)分割,基於mask掩膜的空間網格(spatial grids)分割,基於畫素級細粒度分類的Person Body Segmentation(separate body part detection in reid)和基於特徵圖的Person Body Part Segmentation(可以實現端到端的訓練),前兩種是spatial partition-based local representation 方法。

行人body part 對齊的必要性,如圖:
這裡寫圖片描述

作者設計的part net:
這裡寫圖片描述

論文中FCN為全卷積網路簡寫,FC為全連線網路簡寫。
作者實驗:
The methods are separated into four categories: feature extraction (F), metric learning (M), deeply learnt feature representation (DF), deep learning with matching subnetwork (DMN).

Learnt body parts
這裡寫圖片描述

Body part partition and spatial partition
這裡寫圖片描述

Separate part segmentation
這裡寫圖片描述

The number of parts.
這裡寫圖片描述
Human segmentation and body part segmentation
這裡寫圖片描述
Comparison with non-human/part-segmentation
這裡寫圖片描述
Image feature map extraction networks
這裡寫圖片描述
Comparison with other attention models

這裡寫圖片描述

Comparison with State-of-the-Arts
這裡寫圖片描述
這裡寫圖片描述
這裡寫圖片描述
這裡寫圖片描述
這裡寫圖片描述