1
$\begingroup$

I'm comparing Bayesian (generalized) linear methods vs Frequentist ones in the case when $p>n$ ($p,n$ being respectively number of regressors, number of samples).

In the frequentist context when $p$ > $n$ I cannot invert the design matrix $X$ since it is not full rank. Of course there are methods like Ridge Regression to deal with this problem, but I don't want to consider them here. So in that case there is an identifiability issue.

My question is this:

Is there any case in which the same problem (impossibility of finding regression coefficients when $p>n$) arises in a Bayesian context? For instance, considering the simple linear model, when I specify the priors as Normal-Invgamma (like here) I have no need to invert the design matrix to obtain the posterior parameters, solving in that way the problem which occurs in the frequentist case.

I'm dealing with pretty complex multilevel models and I let STAN do the computations, so matrix algebra (which is the problem in the linear context) is not something I care about, may be wrongly. Thanks to @Tim I read Michael Betancourt answer in stanusersgroup look here and what I understood is that the prior imposition constraints the posterior sampling to a convex set, not always solving the identifiability problem,but at least allowing to spot high correlations between the regressors.

Thank you in advance

Improve this question
$\endgroup$
7
  • $\begingroup$ Inverting matrices has nothing to do with Bayesian vs frequentist methods... Bayesians as well use matrix algebra. Moreover, there are frequentist methods for p > n cases. $\endgroup$ – Tim♦ Apr 19 '17 at 8:33
  • 1
    $\begingroup$ Then please make your question more precise: that you are asking if there are problems with n > p when working with generalized linear models when using Bayesian vs frequentist methods. $\endgroup$ – Tim♦ Apr 19 '17 at 8:40
  • $\begingroup$ @Tim Do you think the question is clear enough now?Thank you $\endgroup$ – Tommaso Guerrini Apr 19 '17 at 12:35
  • 1
    $\begingroup$ Yes, but (1) Bayesian estimation also uses matrix algebra, so this has nothing to do with frequentist methods; (2) it is also about model identifability. $\endgroup$ – Tim♦ Apr 19 '17 at 12:39
  • 1
    $\begingroup$ Check e.g. en.wikipedia.org/wiki/Bayesian_linear_regression $\endgroup$ – Tim♦ Apr 19 '17 at 14:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.