Minimizing relative error (or mean square error) and maximizing likelihood I'm not a statistician, so I would appreciate an answer in the simplest possible words.
I've read that, in some sense, when we minimize the mean square error, we are maximizing the likelihood.
This seems to have more sense if we consider the noise in the system as pure additive Gaussian noise (i.e. $y = f + ae$, where $a$ is the standard deviation of the noise, $e\sim N(0,1)$ and $f$ is the predicted output).
However, when the noise in the system is proportional to the predicted output 
(i.e. $y = f + bfe$) I was thinking that the relative error $RE= \mid\frac{y_{obs}-y_{pred}}{y_{obs}}\mid$ would make more sense.
So, my questions are:


*

*Would it be correct to say that by minimizing the relative error we are maximizing the likelihood in a system with proportional noise?

*If I have a combination of additive and proportional noise ($y = f + (a+bf)e$) would it still be correct to say that minimizing the relative error maximizes the likelihood? If not the relative error, what would be the cost function in this case?
 A: If the noise is Gaussian the maximum likelihood estimator is still a weighted least squares estimator. (minimizing squared relatives in your case)
If the noise is Laplacian and proportional the maximum likelihood estimator is the absolute relative error.
So the objective function of the MLE depends on the error distribution and its variance structure.
If the standard deviation of the errors are proportional to the observations, then your objective function $Q$ looks in your notation like $Q=\frac{(y_{\text{obs}}-y_{\text{pred}})^2}{y_{\text{obs}}^2}$
A: I just derived this, so I hope its OK - my first answer here - you can check the derivation below.
So, if the noise is a linear function of the observed variable (e.g. $y_{obs} = y_{pred} + (ay_{obs} + b)X$ where $X$ is normally distributed with zero mean and unit variance), then maximizing the likelihood is the same as minimizing:
$$
\frac{(y_{obs} - y_{pred})^2}{(ay_{obs} + b)^2}
$$
Notice, that if $a=0$, the denominator becomes constant and it suffices to minimize the squared difference. On the other hand if $b=0$, it suffices to minimize the relative error (the $1/a^2$ factor is a constant, which is irrelevant).

Derivation: Your assumption translates to $y_{obs}$ being normally distributed with mean $\mu = y_{pred}$ and variance $\sigma = ay_{obs} + b$. The likelihood for a single point is thus simply the value of the probability density function of the normal distribution:
$$
f(y_{obs}) = \frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{(y_{obs} - y_{pred})^2}{2\sigma^2}}
$$
Since likelihood is positive everywhere, we can maximize the log-likelihood instead, which is (substituting for $\sigma$)
$$
log(f(y_{obs})) =-log(\sqrt{2\pi}) -log(ay_{obs} + b) -\frac{(y_{obs} - y_{pred})^2}{2(ay_{obs} + b)^2}
$$
Since the first two terms do not depend on the model, we can ignore them in the maximization, as well as the $\frac{1}{2}$ factor. So we are left with maximizing
$$
-\frac{(y_{obs} - y_{pred})^2}{(ay_{obs} + b)^2}
$$
Which is the same is minimizing the the value at the beginning of the answer.
