I was reading Bishop's Pattern Recognition and Machine Learning (PRML) and I am not completely sure I understand Bayesian (polynomial) curve-fitting. This might be an elementary question, but I expected both frequentist and Bayesian approaches to give the same results for curve-fitting. However, I notice that there is an additional term in the variance (see equation 1.71) for Bayesian curve-fitting, compared to equation 1.64, which is obtained by frequentist measures.

For a quick reference: Equation (1.64) is $p(t|x, \textbf{w}_{\text{ML}}, \beta_{\text{ML}}) = \mathcal{N}(t| y(x, \textbf{w}_{\text{ML}}), \beta^{-1})$ and Eq. (1.71) is $s^2(x) = \beta^{-1} + \phi(x)^T\textbf{S}\phi(x)$ where $\textbf{S} = \alpha \textbf{I} + \beta\sum_{n=1}^{N}\phi(x_n)\phi(x)^T$ and $s^2$ is the variance for the Bayesian polynomial fit.

Am I missing something? Is there a reason for this additional term, and if so, how is it accommodated in the frequentist viewpoint? Or am I completely wrong in expecting the two viewpoints to give the same results? If so, why? Are they not just different ways of looking at statistics?

As you can probably tell from the question, I don't have much formal background in stats and am basically self-studying the book, so any help would be appreciated.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.