I do struggle with the most basic starting point of Bayesian statistics: why is using a prior useful?
It seems to me that if anything they hurt much more than help. Moreover, Bayesians always say things like "the more evidence you get the less important a prior becomes". So why use them in the first place?
Especially if you have start with a prior that is really far-off, then you will be hurting your estimates.
To me a "Frequentist" approach seems much simpler and more straightforward.
Example 1 -- Are French people Rude?
Imagine that I am interested in estimating if French people are rude (in a binary way: Rude vs Non-Rude).
Imagine that the true parameter $\theta$ is 0.3, 30% of French are rude.
I have a random sample of 100 people with data on "rudeness".
A straightforward "Frequentist" approach would be: compute your confidence intervals on the sample and ... you done. We know what is the probability that the true $\theta$ lies within our CI, and we know that our sample average, on average, thanks to the Central Limit Theorem, will be close to the true parameter.
true_theta = 0.3 set.seed(111) # population X = rbinom(n = 10000, size = 1, prob = true_theta) # sample of 100 x = sample(X, size = 100) p_hat = prop.table(table(x)) margin_error = 1.96 * sqrt( (p_hat*(1-p_hat)) / 100 ci_low = p_hat - margin_error ci_high = p_hat + margin_error data.frame(p_hat, ci_low, ci_high)
Now Bayesian will try to incorporate priors into this. Why?
We can imagine that most people will have beliefs that French are rude. (Using a Beta distribution) based on personal experience, someone would typically have a prior distribution like this: they know 10 French people and 7 are rude. So that is their prior.
The results are shown in the Figure above. On the bottom panel, I draw the Confidence Intervals around the sample mean.
It seems to me that the Frequentist approach will gives us a more precise and immediate answer to this question.
What am I missing here?
Example 2 -- Forecast Knock Out
Imagine I am interested in forecasting Knocked out (KO) in the Ultimate Fighting Championship (UFC).
My basic Frequentist approach would go like this.
Study the sport and see what variables play a role in predicting a fighter knocking another fighter (e.g. age of the fighters, winning streak, home advantage, …).
Then I will sample 30 UFC events and start build my model. I would use a simple logistic regression with maximum likelihood. I don’t have prior per see, but I have a model of reality that I built on theoretical knowledge, like any scientists do, Bayesian or not. My initial model would be that age and style of fighting (Muay Thai fighter vs BJJ) would be the most important factors predicting a KO.
I retrieve estimates from my logistic regression. Then I would cross-validate, take another sample and apply my initial model to the new data and see how it perform.
Let’s imagine that it doesn’t perform too well. I will then try to improve my model in studying more closely the sport. I find that accounting for an interaction between injury and age of the fighter is one of the most important predictors a KO. I re-run my regression with this and again cross validate, and now the model perform well.
Now I have a forecasting model telling me the probability of a KO and I can use it to bet money.
Furthermore, I know that certain special events, like a New Year's Eve event, will have more KOs, so I can adjust my model according to that fact.
Where would a prior, in the Bayesian sense, be useful here? Frequentists like all scientists use model to test reality and improve their models over time, but the difference is that they don’t put a formal initial probability on things. I still do not see why, you would want to do that.
# code for the Figure theta_range <- seq(0, 1, by = 0.01) prior <- dbeta(x = theta_range, shape1 = 7, shape2 = 3) # observed success success_obs = table(x) failure_obs = table(x) # sampling distribution # likelihood <- dbinom(x = success_obs, size = 100, prob = theta_range) * 100 # posterior posterior_density <- likelihood * prior df <- data.frame(theta = theta_range, likelihood = likelihood, prior_dist = prior, posterior_distribution = posterior_density) df_melt = melt(df, id.vars = 'theta') df_melt %>% ggplot(aes(x = theta, y = value, color = variable)) + geom_line(size = 1.2) + scale_x_continuous(breaks = seq(0, 1, by = 0.1)) + geom_vline(xintercept = p_hat, size = 1.5) + geom_vline(xintercept = ci_low, colour = 'gray', size = 1.5) + geom_vline(xintercept = ci_high, colour = 'gray', size = 1.5) + theme_minimal() + ggtitle("Bayesian + Frequentist Confidence Intervals")