1. 程式人生 > >論文閱讀——《Crafting a Toolchain for Image Restoration by Deep Reinforcement Learning》

論文閱讀——《Crafting a Toolchain for Image Restoration by Deep Reinforcement Learning》

對於同一張圖片的不同區域,需要的denoise的網路是一樣的嗎?有些區域可能很簡單的網路就可以實現很好的效果,但有些區域需要比較複雜的網路才可以得到不錯的效果。

對於不同的圖片,也是如此,有些圖片需要複雜的網路,有些圖片不需要複雜的網路。

如何的自適應地去應對不同的condition?

existing dynamic networks treat all image regions as an integration

different paths for different regions.

如何選patch可以看作為如何選圖片類似嗎?

 

In contrast to existing methods that concentrate on a single type of distortion, we intend to handle a mix of multiple distortions

To address mixed distortions, we propose to restore the corrupted image step by step with a sequence of restoration tools.因此。如何去選擇工具箱,將會十分具備challenge

the choice of the restoration type, level and the processing order all influence the final performance.

The results indicate that minor changes in a toolchain can severely impact the restoration performance. 

除此以外,using improper tools may lead to unnatural outputs(使用不合適的工具會導致結果變化)

故此,合適的工具順序,合適的工具,對結果影響比較大。那麼如何來確定工具的順序,以及所選的工具是否合適呢(所估計出來的level是否合適)

When the tools are trained on specific tasks, we encounter another problem that none of the tools can perfectly handle the ‘middle state’, which refers to the intermediate result after several steps of processing.並且沒有一個工具可以完美的handle中間狀態

 

As most distortions are irreversible (不可逆), the restoration of their mixture is not a simple composition of the corresponding restorers.

New artifacts could be introduced in the middle states. For example, the deblurring operation will also enhance the noises, causing the following denoisers fail in removing the newly introduced artifacts.

  • treat the sequential tool selection problem as a Markov Decision Process (MDP) and solve it in a deep reinforcement learning manner.
  •  propose a training scheme to refine the agent and tools jointly so that the tools are more well-informed with the middle states observable by the agent.

所以關鍵就是強化學習選擇工具的過程,以及這個訓練的方案可以fine tune

 

關於TOOLBOX 

The toolbox contains a set of tools that might be applied to the corrupted image.

each tool is trained only on a narrow range of distortions

用小網路來準確handel一個小範圍的退化

After training the agent, all tools are jointly fine-tuned on the basis of the well-trained toolchains.

 

強化學習部分

Given an input image, the agent first selects a tool from the toolbox and uses it to restore the image, then the agent chooses another tool according to the previous result and repeats the restoration process until it decides to stop.

 

關於動態網路