使用caffe的工具convert_imageset將資料集轉換為lmdb資料庫
阿新 • • 發佈:2019-01-03
本文預先將cifar10儲存為影象格式,分別存放於train和test資料夾中,其中的子資料夾0~9分別代表類標籤。
使用create_list.sh建立資料集的列表檔案
#!/usr/bin/env sh DATA="/home/tim/datasets/cifar10" cd $DATA rm -f train.txt rm -f test.txt find train/0 -name "*" | grep -i -E ".bmp|.jpg|.png" | sed "s/$/ 0/" >> train.txt find train/1 -name "*" | grep -i -E ".bmp|.jpg|.png" | sed "s/$/ 1/" >> train.txt find train/2 -name "*" | grep -i -E ".bmp|.jpg|.png" | sed "s/$/ 2/" >> train.txt find train/3 -name "*" | grep -i -E ".bmp|.jpg|.png" | sed "s/$/ 3/" >> train.txt find train/4 -name "*" | grep -i -E ".bmp|.jpg|.png" | sed "s/$/ 4/" >> train.txt find train/5 -name "*" | grep -i -E ".bmp|.jpg|.png" | sed "s/$/ 5/" >> train.txt find train/6 -name "*" | grep -i -E ".bmp|.jpg|.png" | sed "s/$/ 6/" >> train.txt find train/7 -name "*" | grep -i -E ".bmp|.jpg|.png" | sed "s/$/ 7/" >> train.txt find train/8 -name "*" | grep -i -E ".bmp|.jpg|.png" | sed "s/$/ 8/" >> train.txt find train/9 -name "*" | grep -i -E ".bmp|.jpg|.png" | sed "s/$/ 9/" >> train.txt find test/0 -name "*" | grep -i -E ".bmp|.jpg|.png" | sed "s/$/ 0/" >> test.txt find test/1 -name "*" | grep -i -E ".bmp|.jpg|.png" | sed "s/$/ 1/" >> test.txt find test/2 -name "*" | grep -i -E ".bmp|.jpg|.png" | sed "s/$/ 2/" >> test.txt find test/3 -name "*" | grep -i -E ".bmp|.jpg|.png" | sed "s/$/ 3/" >> test.txt find test/4 -name "*" | grep -i -E ".bmp|.jpg|.png" | sed "s/$/ 4/" >> test.txt find test/5 -name "*" | grep -i -E ".bmp|.jpg|.png" | sed "s/$/ 5/" >> test.txt find test/6 -name "*" | grep -i -E ".bmp|.jpg|.png" | sed "s/$/ 6/" >> test.txt find test/7 -name "*" | grep -i -E ".bmp|.jpg|.png" | sed "s/$/ 7/" >> test.txt find test/8 -name "*" | grep -i -E ".bmp|.jpg|.png" | sed "s/$/ 8/" >> test.txt find test/9 -name "*" | grep -i -E ".bmp|.jpg|.png" | sed "s/$/ 9/" >> test.txt
使用create_lmdb.sh建立lmdb資料庫
注意設定-shuffle引數為true
#!/usr/bin/env sh # This script converts the data into lmdb/leveldb format, # depending on the value assigned to $BACKEND. set -e EXAMPLE="/home/tim/datasets/cifar10" DATA="/home/tim/datasets/cifar10" BUILD=$CAFFE_ROOT/build/tools GRAY=false RESIZE=false if $RESIZE; then RESIZE_HEIGHT=32 RESIZE_WIDTH=32 else RESIZE_HEIGHT=0 RESIZE_WIDTH=0 fi BACKEND="lmdb" echo "Creating ${BACKEND}..." rm -rf $EXAMPLE/train_${BACKEND} rm -rf $EXAMPLE/test_${BACKEND} $BUILD/convert_imageset -backend=$BACKEND -gray=$GRAY -resize_width=$RESIZE_WIDTH -resize_height=$RESIZE_HEIGHT -shuffle=true $DATA/ $DATA/train.txt $EXAMPLE/train_${BACKEND} $BUILD/convert_imageset -backend=$BACKEND -gray=$GRAY -resize_width=$RESIZE_WIDTH -resize_height=$RESIZE_HEIGHT -shuffle=true $DATA/ $DATA/test.txt $EXAMPLE/test_${BACKEND} echo "Computing image mean..." $BUILD/compute_image_mean -backend=$BACKEND \ $EXAMPLE/train_${BACKEND} $EXAMPLE/mean.binaryproto echo "Done."