1
$\begingroup$

Likely question:In Bayesian linear regression, why do we assume parameter prior has zero mean?.


But, I just focus on can we assume $p(w \mid \alpha) = \mathcal{N}(w \mid \mu, \alpha^{-1})$, and the reason of zero mean.

If mean of $w$ is $\mu$, we should minimum of $$\frac{\beta}{2} \sum_{n=1}^N\{y(x_n,w)-t_n\}^2+\frac{\alpha}{2}(w-\mu)^T(w-\mu)$$

so the result is not Ridge Regression(just my opinion).

My Question: can we assume $w$ is not zero mean? Does the result of whether the mean of $w$ is zero lead to different result?

Reason of ask the question: I didn't find the answer in the question above. I also want to know the reason \ undermeaning of assume zero mean.


This problem is from book PRML 1.2.5 Curve fitting re-visited Page28.

Apologize for not being word-perfect in English.

$\endgroup$
2
$\begingroup$

Slightly too long for a comment, not sure if this consistutes a complete answer.

Philosophically, there is no reason you could not set $\mu \ne 0$. I think the reason people tend to use $\mu = 0$ is that there isn't any other natural default choice. By setting $\mu = 0$, you are shrinking towards a model in which $w = 0$, which has a more parsimonious feel to it. That is, we are shrinking towards a model in which $w$ is not included in the model. There is also something of a rotational invariance sort of argument: one way of expressing ignorance about the predictors is to say that any orthogonal transformation of the predictors should have the same prior. This implies we should take $\mu = 0$ and use the covariance $\Sigma = \alpha^{-1} I$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.