# Real noise modeling/ noise map generation (image processing, deep learning)

I am working on a project with really noisy images. I have trained a detector that can detect the characters but fails in some cases (noise is high).

So far I have gone through many denoising, deblurring, super-resolution papers. The problem with denoising papers is that in almost all of them, they use a specified Gaussian noise to first add noise and trains the model on that. I have tried it but it doesn't work very well in my domain as the source of the noise in my images is different.

Let's say I have few thousand images (real-data with noise), is there any deep learning/image processing approach which helps me to get a noise map which I'll use to augment my clean images so that I can train denoising models.

• Do you access to noisy and denoised version of the same image? – usεr11852 Apr 16 '20 at 11:58
• I have only access to noisy images, or you can say I have two set of images, one which are really distorted and noisy, another one with less noise but for different set of images. – Zabir Al Nazi Apr 16 '20 at 12:28

Let's say you have $$m$$ noisy images $$I_N^{i};i \in [1...m ]$$. One way of accomplishing your end goal is as follows:
1. Run single-image learning-based denoising techniques on your noisy images to obtain corresponding clean images. You can use Deep Image Prior (code) or Noise2Void - Learning Denoising from Single Noisy Images (code). These methods learn the image intensity statistics for an image to denoise them as opposed to using a source-pair fixed distribution (e.g. Gaussian). Your model $$M(\cdot)$$ takes noisy images $$I_N^{i}$$, and produces corresponding clean images $$I_C^{i}$$.
2. Using the clean-noisy pairs following step 1 above, train a fully supervised model $$P(\cdot)$$ to produce noisy image, given a clean image as input. To do this, you can use a strong single-image superresolution architecture (several options here). Why superresolution? Because noise map has high frequencies, similar to the mask composited by superresolution architectures on top of the input image. These masks essentially enhance edges, which has a high frequency structure.
3. Using $$P(\cdot)$$, forward pass other clean images to create a new dataset for training your original denoising model, this is what you wanted to accomplish.
Noise map would simply be $$N^{i}=I_{N}^{i}-I_{C}^{i}$$. For training $$P(\cdot)$$ in step 2, you can apply mild data augmentation, e.g. rotation, scaling (zooming in/out), translation, mild brightness and contrast adjustments.