雙目立體視覺的整體流程包括：影象獲取、雙目標定、雙目矯正、立體匹配、三維重建。

影象獲取

雙目相機拍攝獲取 左右目影象

雙目標定

《Learning OpenCV》中對於 Translation 和 Rotation 的圖示是這樣的:

雙目矯正

通過 OpenCV函式 stereoRectify，包括 畸變矯正 和 立體矯正 。

其中，
\begin{aligned}
P_1 =
\begin{bmatrix}
f & 0 & c_x & 0 \
0 & f & c_y & 0 \
0 & 0 & 1 & 0
\end{bmatrix}
\end{aligned}

\begin{aligned}
P_2 =
\begin{bmatrix}
f & 0 & c_x & T_x \cdot f \
0 & f & c_y & 0 \
0 & 0 & 1 & 0
\end{bmatrix}
\end{aligned}

通過 $P_2$ 可計算出 基線 長度:
\begin{aligned}
baseline = - T_x = - \frac{{P_2}_{03}}{f_x}
\end{aligned}

• ethz-asl/image_undistort: A compact package for undistorting images directly from kalibr calibration files. Can also perform dense stereo estimation.

立體匹配

視差計算

通過 OpenCV函式 stereoBM (block matching algorithm)，生成 視差圖(Disparity Map) (CV_16S or CV_32F)

disparity map from stereoBM of OpenCV :
It has the same size as the input images. When disptypeCV_16S, the map is a 16-bit signed single-channel image, containing disparity values scaled by 16. To get the true disparity values from such fixed-point representation, you will need to divide each disp element by 16. If disptypeCV_32F, the disparity map will already contain the real disparity values on output.

So if you’ve chosen disptype = CV_16S during computation, you can access a pixel at pixel-position (X,Y) by: short pixVal = disparity.at<short>(Y,X);, while the disparity value is float disparity = pixVal / 16.0f;; if you’ve chosen disptype = CV_32F during computation, you can access the disparity directly: float disparity = disparity.at<float>(Y,X);

• Disparity Map
• Disparity map post-filtering

三維重建

深度計算

深度計算公式如下，通過遍歷影象生成 深度圖
\begin{aligned}
Z = depth = \frac{f_x \cdot baseline}{disparity}
\end{aligned}
其中， $disparity$ 代表 視差圖 座標值

影象型別

• 單位meter --> 32FC1
• 單位millimeter --> 16UC1

三維點座標計算

• 根據 小孔成像模型，已知 $Z$ 和 相機內參 可計算出 三維點座標，從而生成 三維點雲
  \begin{aligned}
  \begin{cases}
  Z = depth \
  X = \frac{u-c_x}{f_x} \cdot Z \
  Y = \frac{v-c_y}{f_y} \cdot Z
  \end{cases}
  \end{aligned}