9
$\begingroup$

My overall question is: why use bayesglm instead of other classification methods?

Note:

  1. I'm only interested in prediction.
  2. I have a decent amount of data (~ 100,000 obs).

I feel like the sample size is large enough the parameters of a regular logistic regression are going to be normally distributed (CLT). What would I gain by specifying priors? My hunch is that it will only matter for a small dataset, but I don't have any theoretical or applied evidence.

$\endgroup$
3
  • 5
    $\begingroup$ Your intuition about the relationship between sample size and priors is correct. On the other hand, Bayesian logistic regression can solve the problem of infinite parameter estimates resulting from perfect separation. $\endgroup$
    – Sycorax
    Commented Oct 23, 2013 at 20:39
  • 1
    $\begingroup$ Logistic regression is not a classification algorithm. It is a probability prediction algorithm. $\endgroup$ Commented Jul 23, 2019 at 14:17
  • 1
    $\begingroup$ What Sycorax mentions is one of the most important reasons you would want to use a Bayesian model in a large-sample setting. If your logistic regression has lots of predictors, especially predictors with low variance, consider having priors over the regression coefficients. $\endgroup$ Commented Jul 23, 2019 at 14:18

1 Answer 1

11
$\begingroup$

In engineering, as well as supply chain risk management, "engineering knowledge" --eg an educated persons best guess-- may be the best data you have. For example, the likelihood of a tsunami occurring and disrupting the supply chain, without additional data, can be estimated by an expert in the subject (there are better methods for constructing priors). As time passes, tsunamis occur and, as a result, we gain more data, and can update our priors (engineering knowledge) with posteriors (priors adjusted for new data). At some point, there will be so much data that the initial prior is irrelevant, and no matter whom made the prediction, you will have equal predictions of likelihood.

It is my belief that if you have that much data, a "traditional" Frequentist approach is (typically) preferable to the Bayesian approach (of course others will disagree, especially with choosing between statistical philosophies rather than sticking to one and selecting an appropriate method). Note that it is entirely possible (and occurs often) that the Frequentist approach yields similar/identical results to the Bayesian.

That said, when the difference in methods is a line of code, why not implement multiple methods and compare the results yourself?

$\endgroup$
1
  • 1
    $\begingroup$ Thanks! Good explanation of some aspects of Bayesian thinking - not something I'm very familiar with. $\endgroup$
    – wcampbell
    Commented Oct 23, 2013 at 20:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.