1. 程式人生 > >BN層和CONV層合併

BN層和CONV層合併

卷積層中

卷積權重: W,卷積偏置:B

卷積層運算:W \times X+B

BN 層中
均值:\mu ,方差:\delta,縮放因子:\gamma,偏移:\beta, 一個較小數(防止分母為0):\epsilon

 \large \mu \leftarrow \tfrac{1}{m}\sum_{i=1}^{m}x_i           \large \sigma^2 \leftarrow \tfrac{1}{m}\sum_{i=1}^{m}(x_i-\mu)^2

\large \hat{x_i} \leftarrow \frac{x_i-\mu}{\sqrt{\sigma^2+\epsilon }}        \large y_i \leftarrow \gamma \hat{x_i} + \beta

BN層和卷積層合併後:

\large \alpha = \frac{\gamma }{\sqrt{\sigma^2+\epsilon }}

\large W_{merged} = W\times \alpha

\large B_{merged} =B\times \alpha+(\beta-\mu\times a)