I want to estimate a simple Regression. I have 20 observations and 10 regression parameters. The degrees of freedom are too small to get reliable point estimates and p-values. I found the following statement:

Another advantage of MCMC estimation is the problem of degree of freedom. In the maximum likelihood estimation, if the parameters number is large compared with the observation, the model becomes unstable and the obtained result would not be reliable. But we do not need to worry about it if we use MCMC estimation. In this note, for example, the number of parameteres to be estimated is larger than n, which is the observation number." (Estimating Markov Switching model using Gibbs sampling with a statistical computing software R)

Would bayesian inference (MCMC) of the Regression solve my problem? Or is the statement only true in the specific case of the Markov switching model?


As you stated, in some ways, the Bayesian inference solves your problem. However, it has to be considered that, roughly speaking, the less data, the more the priors speak. So in the presence of very little data, the prior distributions would have more influence, and consequently your inference can be highly conditioned by the choice of prior. In that sense, the Bayesian inference does not (always) solve your problem. Note that this property is not related to MCMC but to Bayesian inference.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.