# Real noise modeling/ noise map generation (image processing, deep learning)

I am working on a project with really noisy images. I have trained a detector that can detect the characters but fails in some cases (noise is high).

So far I have gone through many denoising, deblurring, super-resolution papers. The problem with denoising papers is that in almost all of them, they use a specified Gaussian noise to first add noise and trains the model on that. I have tried it but it doesn't work very well in my domain as the source of the noise in my images is different.

Let's say I have few thousand images (real-data with noise), is there any deep learning/image processing approach which helps me to get a noise map which I'll use to augment my clean images so that I can train denoising models.

• Do you access to noisy and denoised version of the same image? Commented Apr 16, 2020 at 11:58
• I have only access to noisy images, or you can say I have two set of images, one which are really distorted and noisy, another one with less noise but for different set of images. Commented Apr 16, 2020 at 12:28

Let's say you have $$m$$ noisy images $$I_N^{i};i \in [1...m ]$$. One way of accomplishing your end goal is as follows:

1. Run single-image learning-based denoising techniques on your noisy images to obtain corresponding clean images. You can use Deep Image Prior (code) or Noise2Void - Learning Denoising from Single Noisy Images (code). These methods learn the image intensity statistics for an image to denoise them as opposed to using a source-pair fixed distribution (e.g. Gaussian). Your model $$M(\cdot)$$ takes noisy images $$I_N^{i}$$, and produces corresponding clean images $$I_C^{i}$$.
2. Using the clean-noisy pairs following step 1 above, train a fully supervised model $$P(\cdot)$$ to produce noisy image, given a clean image as input. To do this, you can use a strong single-image superresolution architecture (several options here). Why superresolution? Because noise map has high frequencies, similar to the mask composited by superresolution architectures on top of the input image. These masks essentially enhance edges, which has a high frequency structure.
3. Using $$P(\cdot)$$, forward pass other clean images to create a new dataset for training your original denoising model, this is what you wanted to accomplish.

Noise map would simply be $$N^{i}=I_{N}^{i}-I_{C}^{i}$$. For training $$P(\cdot)$$ in step 2, you can apply mild data augmentation, e.g. rotation, scaling (zooming in/out), translation, mild brightness and contrast adjustments.

• thanks so far, I will have to run some experiments to verify your thought process, I will let you know. Commented Apr 16, 2020 at 12:29
• Please let me know if you have additional questions. I would appreciate if you can award me this bounty at the end if you think my answer was reasonable, I need it to ask a bounty question of my own.
– PAF
Commented Apr 18, 2020 at 14:21
• I have accepted your answer, even though I haven't tested it yet but as there were not any better answer and your answer seems reasonable. Commented Apr 19, 2020 at 9:31