2
$\begingroup$

After reading about the bayesian approach, I'm wondering how they would interpret a null finding from a regression coefficient. I ask because in a frequentist approach, if the p<.05 you'd say that there was an association between two variables, and if p>.05 you'd say that it wasn't.

For example, you could imagine someone using frequentist statistics saying something like (where b is a regression coefficient): participant height was associated with participant weight, b=.30, p<.05. In the case of a null effect, it might be something like: participant height was not associated with participant shoe size, b=.05, p=.40.

Now in the bayesian context, it seems like there's no easy way to describe associations. Maybe, if using something like a credible interval to describe the posterior distribution for a regression parameter, you could say that associations with a 95% credible interval excluding zero are strong associations but that associations with 95% credible intervals including zero are weak ones?

I guess my question is how does a bayesian describe their result when they get a situation like this from the frequentist context: participant height was not associated with participant shoe size, b=.05, p=.40.

$\endgroup$
  • 1
    $\begingroup$ If p>0.05, you DO NOT say that an association doesn't exist. Typically we say "fail to reject the null hypothesis" but that doesn't mean the null hypothesis is true. $\endgroup$ – jaradniemi Aug 23 '18 at 20:10
2
$\begingroup$

I would strongly oppose the answer of @Dave Harris.

What do frequentists do?

The make assumptions about the true underlying relations of the random variables, compute some new realization of a random variable that has a certain distribution they know (due to the assumptions on the underlying relations) and then see how 'probable' it is that this random variable produced this concrete realization. So this is in no way a 'direct' assertion of the probability of the parameter being zero or so. In short: "It seems to be accepted by the community" to transfer the assertion to some object/setting/... topic and then make this related object/setting/... precise.

What do Bayesians do?

Let us say that we chose the model $$Z = \beta_0 + \beta_1X + \beta_2Y + \text{error}$$ and we are given realizations (training set) $x_1, y_1, z_1, ..., x_N, y_N, z_N$. We then make some natural assumptions (the touples $W_i = (X_i, Y_i, Z_i)$ are independent of each other and so on). So far everything is as in the frequentists approach. Here comes the 'new' step: We imagine the $\beta_0, \beta_1, \beta_2$ being outcomes of random variables $B = (B_0, B_1, B_2)$ as well and put a rasonable so-called prior on these, i.e. we assume that $B$ has some density $f_B$. Then from the assumptions on the structure of the model and the independence and so son we can explicitly write down the conditional density $p(B|W)$ where $W = (W_1, ..., W_N) = (X_1, Y_1, Z_1, ..., X_N, Y_N, Z_N)$. Then the Bayesian viewpoint is that we do not search for the parameter that best explains the generation of the data (i.e. we do not maximize $p(W;\beta_0,\beta_1,\beta_2)$ which we can somewhat interpret as $p(W|B)$) but rather we compute the point in the density of the switched order $p(B|W)$ that is maximal (i.e. what is the parameter that has the highest probability given the data that we observed). Those two viewpoints are linked by the following formula $$p(B|W) = \frac{p(W|B)p(B)}{p(W)}$$ What we are after in the end is a more or less complete description of $p(B|W)$. In the very concrete example above we could, for example, get the distribution of $B_1$ (which encodes the relation of $X$ to $Z$) given the data, i.e. $p(B_1|W)$ by marginalizing out $B_0$ and $B_2$. We could get different pictures then. Ill provide three examples and then I state how I interpret them:

Situation 1 Situation 1

Situation 2 Situation 2

Situation 3 Situation 3

Situation 1: The mean and highest peak of the density is around 0 so the model suspects $\beta_1$ to be zero. The fact that there is almost no deviation from that value (i.e. the density put very much mass around a very small space around 0) tells us that the model is very certain that $\beta_1=0$. This corresponds to th case where the Null hypothesis can be rejected, we can safely claim that $\beta_1=0$.

Situation 2: The mean and highest peak of the density is around 0 so the model suspects $\beta_1$ to be zero. However, the peak is much smaller and the density puts a lot of mass outside that region around 0. Hence, the model suspects that $\beta_1=0$ but it 'cant prove it'/it is uncertain. This corresponds to the situation where the Null hypothesis may be right but we do not have enough evidence to reject it. In this case we tell our boss that (using this model) it might be true that $\beta_1=0$ but we cannot prove it. So either we need to change the model (this can be dangerous and could correspond to 'p-Hacking') or we need to collect more data.

Situation 3: The mean and highest peak of the density is around -2 so the model suspects $\beta_1$ to be $-2$. However, it is not as certain as in situation 1. So like in the frequentists approach (they set the acceptance threshold to p=0.05 which is absolutely arbitrary! Why not 0.01 or 0.0001?) it is up to you to define a measure/threshold that applies to the density in order to decide whether or not you want to state that $\beta_1=-2$ in that case. However, if the question is whether or not $\beta_1=0$ then again, the mass the density puts around $0$ is VERY little so we can safely claim that $\beta_1 \neq 0$.

$\endgroup$
2
$\begingroup$

The p-value is a statistic used in frequentist methodology, and its proper interpretation is fixed by its definition. Since the p-value is not a posterior probability, Bayesians would tend not to bother using this quantity at all. (If they decided to use if for some reason, it would have the same interpretation as a frequentist would give it ---i.e., the correct interpretation.)

Within Bayesian statistics, competing hypotheses about the model parameters are generally tested by comparing the posterior probability of competing parametric hypotheses under a given prior.$^\dagger$ This is usually done using Bayes factor to write the ratio of posterior probabilities for a given ratio of priors. Given observed data $\mathbf{x}$ and disjoint parametric hypotheses $\theta \in \Theta_0$ and $\theta \in \Theta_A$ we have:

$$\frac{\mathbb{P}(\theta \in \Theta_0|\mathbf{x})}{\mathbb{P}(\theta \in \Theta_A|\mathbf{x})} = \underbrace{\frac{p(\mathbb{x}|\theta \in \Theta_0)}{p(\mathbb{x}|\theta \in \Theta_A)}}_{\text{Bayes factor}} \cdot \frac{\mathbb{P}(\theta \in \Theta_0)}{\mathbb{P}(\theta \in \Theta_A)}$$

Bayes factor can be calculated from the likelihood $L_\mathbf{x}$ and prior $\pi$ as:

$$BF_\mathbf{x} \equiv BF_\mathbf{x}(\Theta_0,\Theta_A) \equiv \frac{p(\mathbb{x}|\theta \in \Theta_0)}{p(\mathbb{x}|\theta \in \Theta_A)} = \frac{\int_{\Theta_0} L_\mathbf{x}(\theta) \pi(\theta) d\theta}{\int_{\Theta_A} L_\mathbf{x}(\theta) \pi(\theta) d\theta} \cdot \frac{\int_{\Theta_A} \pi(\theta) d\theta}{\int_{\Theta_0} \pi(\theta) d\theta}.$$

Once you have calculated Bayes factor for your parametric hypotheses, you can easily obtain the posterior probability ratio that obtains from any given prior probabilities for those parameters. If we take $\alpha = \mathbb{P}(\theta \in \Theta_0)$ to be the prior probability of the null hypothesis, and we assume that there are only two possible hypotheses (i.e., the null and alternative parameter spaces partition the whole parameter space), then we have the posterior probability:

$$\mathbb{P}(\theta \in \Theta_0|\mathbf{x}) = \frac{\alpha BF_\mathbf{x}}{(1-\alpha) + \alpha BF_\mathbf{x}}.$$

This result allows you to substitute any prior probability $0 \leqslant \alpha \leqslant 1$ for the null hypothesis and get the corresponding posterior probability of that hypothesis.


$^\dagger$ There are some papers that explore "Bayesian p-values" which are quantities that are somewhat similar to the frequentist p-values except that they are calculated as posterior quantities (see e.g., Meng 1994, Gelman 2003 and Gelman 2012). These are less commonly used than the standard posterior parametric comparison using Bayes factor.

$\endgroup$
0
$\begingroup$

I like @Ben (though I don't think we need to rely on Bayes factors necessarily) and @FabianWerner's answers. I'd like to add that Bayesian statistics allow us to not be so strict: An association of exactly zero is highly improbable in most settings. So instead of having a rigid straw-man null hypothesis of "r = 0," (the null hypothesis that frequentist statistics assume and test against), we can get a posterior distribution and see regions where we are most confident that the parameter lies.

If the posterior distribution of the parameter is centered tightly around zero, we can believe that there's probably a practically null relationship (e.g., close enough to zero that we don't care).

Frequentists can't do this—they assume the null is true. A p-value is the probability we observe our data (or more extreme data), given that the null hypothesis is true. It can't tell us anything about the null, because it is (a) a rather meaningless null in most situations, and (b) it assumes this null is true. This comes up a lot when we have such big samples that almost anything is significant in frequentist statistics. Since the null hypothesis is a parameter being precisely 0.0000000..., then even a small deviation can be "significant." I like that Bayesian statistics allow us to look at probabilities of where the parameter is likely to be, so we can see how likely it is that the parameter falls in a range of values we don't care about. If my client wants at least a one percentage point increase, I can specify how likely it is that we get a one percentage point increase, how likely it is that we get -1 to +1 change (where we don't really care—a null association for all intents and purposes), and how likely it is to get an increase of -1 or less (a backlash effect).

$\endgroup$
-2
$\begingroup$

Let us consider the following model, $$z=\beta_xx+\beta_yy+\alpha.$$ Bayesian hypotheses are combinatoric. The possible hypotheses are $$z=\alpha,$$ $$z=\beta_xx+\alpha,$$ $$z=\beta_yy+\alpha,$$ and $$z=\beta_xx+\beta_yy+\alpha.$$ If $\beta_x=0,$ that is equivalent to removing $\beta_xx$ from the equations that are possible. In addition to a prior for the parameters, you would need to create a prior for each possible model.

Because parameters are random variables, for those on a continuum such as the real numbers, $\Pr(\beta_x=0)=0$ since it is a countable point in an uncountable set. There is no direct method to test a no association hypothesis in Bayesian methods.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.