I came across "Tikhonov regularization" and I have bare knowledge on it. It seems that it is a type of regularization that is important for deconvolution. Are there any good resources and examples? Also, can you explain briefly what it is?


Deconvolution is part of a more general class of problems called inverse problems. In the case of deconvolution you want to recover the original image from a given one which is affected by noise after being altered by some process/system modeled through a filter, that is, $$ observed(x) = h * input(x) + \epsilon $$ where the noise if often assumed to be white noise. Notice that this problem is ill-posed. First, assume that the image is not affected by noise. h is usually a blur filter which is band limited. If you calculate the Fourier transform, $$ Observed(\omega) = H(\omega)Input(\omega) $$ which is not defined for values outside of the band. Hence, you need to introduce and additional bound on the filter in order to be able to solve it. If you have noise in addition, $$ Input(\omega) = \frac{Output(\omega)}{H(\omega)} + \frac{Noise(\omega)}{H(\omega)} $$ so that variations in the noise produce very different solutions.

The idea of Tikhonov regularization is to stabilize the problem by adding some constraint on the possible solutions. This constraint presents some a priori knowledge about the problem. Concretely, you would solve, $$ H(f) = \sum_{i}\left(f(x_{i})-y_{i}\right)^{2} + \lambda ||Df||^{2} $$ where $D$ is a differential operator like for example $\frac{d^{2}}{dx^{2}}$. This condition is basically imposing some smoothness on the filter response. See the paper by Poggio et al. (Regularization Neworks and Neural Networks Architectures) for a detailed derivation and some concrete cases.

Now it can be proved that this results in the following solution, $$ f(x) = \sum_{i}c_{i}G(x-x_{i}) $$ where $G$ is the Green function (a.k.a. kernel in the context of regression) associated with the regularizer. By means of cross-validation you can search for good values of $\lambda$ and the order of the differential operator.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.