8
$\begingroup$

My overall question is: why use bayesglm instead of other classification methods?

Note:

  1. I'm only interested in prediction.
  2. I have a decent amount of data (~ 100,000 obs).

I feel like the sample size is large enough the parameters of a regular logistic regression are going to be normally distributed (CLT). What would I gain by specifying priors? My hunch is that it will only matter for a small dataset, but I don't have any theoretical or applied evidence.

$\endgroup$
  • 4
    $\begingroup$ Your intuition about the relationship between sample size and priors is correct. On the other hand, Bayesian logistic regression can solve the problem of infinite parameter estimates resulting from perfect separation. $\endgroup$ – Sycorax Oct 23 '13 at 20:39
  • 1
    $\begingroup$ Logistic regression is not a classification algorithm. It is a probability prediction algorithm. $\endgroup$ – Brash Equilibrium Jul 23 '19 at 14:17
  • 1
    $\begingroup$ What Sycorax mentions is one of the most important reasons you would want to use a Bayesian model in a large-sample setting. If your logistic regression has lots of predictors, especially predictors with low variance, consider having priors over the regression coefficients. $\endgroup$ – Brash Equilibrium Jul 23 '19 at 14:18
8
$\begingroup$

In engineering, as well as supply chain risk management, "engineering knowledge" --eg an educated persons best guess-- may be the best data you have. For example, the likelihood of a tsunami occurring and disrupting the supply chain, without additional data, can be estimated by an expert in the subject (there are better methods for constructing priors). As time passes, tsunamis occur and, as a result, we gain more data, and can update our priors (engineering knowledge) with posteriors (priors adjusted for new data). At some point, there will be so much data that the initial prior is irrelevant, and no matter whom made the prediction, you will have equal predictions of likelihood.

It is my belief that if you have that much data, a "traditional" Frequentist approach is (typically) preferable to the Bayesian approach (of course others will disagree, especially with choosing between statistical philosophies rather than sticking to one and selecting an appropriate method). Note that it is entirely possible (and occurs often) that the Frequentist approach yields similar/identical results to the Bayesian.

That said, when the difference in methods is a line of code, why not implement multiple methods and compare the results yourself?

$\endgroup$
  • $\begingroup$ Thanks! Good explanation of some aspects of Bayesian thinking - not something I'm very familiar with. $\endgroup$ – wcampbell Oct 23 '13 at 20:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.