# Optimal regularization for non linear optimization

What would be the optimal (Bayesian?) solution for fitting a model $f(x)$ to data $h$, given the following assumptions:

• $h$ is a vector with $N$ elements
• $h$ has Gaussian noise with known covariance $\sigma_h I$
• $x$ is known to have an a-priori Gaussian distribution, with covariance $C_x$
• the first- and second-order derivatives of $f(x)$ are known

My understanding is that:

• The maximum likelihood solution is given by minimizing $\sum_{i=1}^N (f(x,i)-h_i)^2$.

• The MAP estimate is given by: $\sum_{i=1}^N \frac{1}{\sigma_c}(f(x,i)-h_i)^2 + \lambda x^t C_x^{-1} x$

Both can be optimized using non-linear optimization techniques such as BFGS.
So the question comes down to:

Is there a way to determine the optimal value for $\lambda$ ?

-

In all similar problems I dealt with there was no "optimal" value for regularisation coefficient.

And now it seems to me that the only solution in your case is using Cross Validation procedure.

-