1. 程式人生 > >目標檢測中準確率評價指標

目標檢測中準確率評價指標

caffe中實現SSD準確率評價方法是TestDetection()函式。mAp指標值是每個類別的Average precision的平均值。

在計算Average Precision之前需要先計算出所有預測框與gt_bboxes的匹配。

SSD evaluation  layer以detection_output layer的輸出([image_id, label, confidence, xmin, ymin, xmax, ymax])作為輸入,同時輸出 [image_id, label, confidence, true_pos, false_pos]元組。

具體操作為:1)遍歷每個類別;2)將該類別的預測框按confidence降序排列;3)對於每個pred_bbox,找出與其有最大iou的gt_bbox;4)如果該gt_bbox之前沒有被分配且iou大於給定的閾值(比如0.5),那將該gt_bbox分配該給pred_bbox,設定該pred_bbox為true positive;否則設定該pred_bbox為false positive.

      for (LabelBBox::iterator iit = detections.begin();
           iit != detections.end(); ++iit) {
        int label = iit->first;
        if (label == -1) {
          continue;
        }
        vector<NormalizedBBox>& bboxes = iit->second;
        if (label_bboxes.find(label) == label_bboxes.end()) {
          // No ground truth for current label. All detections become false_pos.
          for (int i = 0; i < bboxes.size(); ++i) {
            top_data[num_det * 5] = image_id;
            top_data[num_det * 5 + 1] = label;
            top_data[num_det * 5 + 2] = bboxes[i].score();
            top_data[num_det * 5 + 3] = 0;
            top_data[num_det * 5 + 4] = 1;
            ++num_det;
          }
        } else {
          vector<NormalizedBBox>& gt_bboxes = label_bboxes.find(label)->second;
          // Scale ground truth if needed.
          if (!use_normalized_bbox_) {
            CHECK_LT(count_, sizes_.size());
            for (int i = 0; i < gt_bboxes.size(); ++i) {
              OutputBBox(gt_bboxes[i], sizes_[count_], has_resize_,
                         resize_param_, &(gt_bboxes[i]));
            }
          }
          vector<bool> visited(gt_bboxes.size(), false);
          // Sort detections in descend order based on scores.
          std::sort(bboxes.begin(), bboxes.end(), SortBBoxDescend);
          for (int i = 0; i < bboxes.size(); ++i) {
            top_data[num_det * 5] = image_id;
            top_data[num_det * 5 + 1] = label;
            top_data[num_det * 5 + 2] = bboxes[i].score();
            if (!use_normalized_bbox_) {
              OutputBBox(bboxes[i], sizes_[count_], has_resize_,
                         resize_param_, &(bboxes[i]));
            }
            // Compare with each ground truth bbox.
            float overlap_max = -1;
            int jmax = -1;
            for (int j = 0; j < gt_bboxes.size(); ++j) {
              float overlap = JaccardOverlap(bboxes[i], gt_bboxes[j],
                                             use_normalized_bbox_);
              if (overlap > overlap_max) {
                overlap_max = overlap;
                jmax = j;
              }
            }
            if (overlap_max >= overlap_threshold_) {
              if (evaluate_difficult_gt_ ||
                  (!evaluate_difficult_gt_ && !gt_bboxes[jmax].difficult())) {
                if (!visited[jmax]) {
                  // true positive.
                  top_data[num_det * 5 + 3] = 1;
                  top_data[num_det * 5 + 4] = 0;
                  visited[jmax] = true;
                } else {
                  // false positive (multiple detection).
                  top_data[num_det * 5 + 3] = 0;
                  top_data[num_det * 5 + 4] = 1;
                }
              }
            } else {
              // false positive.
              top_data[num_det * 5 + 3] = 0;
              top_data[num_det * 5 + 4] = 1;
            }
            ++num_det;
          }
        }
      }
    }

計算mAP的程式碼位於src/caffe/solver.cpp中的TestDetection的函式,該函式以vector<pair<float, int> > label_true_pos,vector<pair<float, int> > label_false_pos作為引數呼叫src/caffe/util/bbox_util.cpp檔案中的ComputeAP函式。ComputeAP計算出所有的precision和recall值。average precision的計算方式有11point(VOC2007 styl),MaxIntegral(VOC2012 or ILSVRC style),Integral(會比11ponit計算出來的值略大)。