I am fitting Bayesian models (using R and rstanarm). Beyond estimating the effect of each predictor (and extracting pointwise indices such as median, MAD and 90% CI), I am also interested in having a general index of effect existence (telling if the effect is "different" (in the common sense) from 0). Two ways I can imagine of doing this are:

  1. Extracting the probability that the effect is in one particular direction (i.e., the area under the density curve of one "side" (from 0) of the posterior distribution (e.g., the probability that an effect is positive vs. the probabiltiy that the effect is null or opposite)).
  2. Performing a Bayesian t.test of the posterior distribution against mu=0 and computing the Bayes factor, the odds that the effect distribution is different from 0. Unfortunately, due to the large size of the posterior distribution (for example 4000 draws), the Bayes factor is almost always infinite, even if the distribution heavily overlaps 0 and the opposite side.

What could I do?


The problem is not in the methods to answer, but in the question:

telling if the effect is different from 0

The effect or the coefficient or whatever have a distribution density. But given, that 0 is a point on a real scale, it has a probability of zero to begin with. There is an infinite number of points on the real scale and if each of them had a probability $P > 0$, than the sum of their probabilites would be larger then one, which cannot be.

The coefficient or the effect being in any given interval, has a probability, the coefficent or the effect being one exact value is zero. Please read about the concept of "Region of Practical Equivalence (ROPE)" and you will find good answers how to proceed.

vs. the probabiltiy that the effect is null or opposite

The probability of the effect being null or opposite is identical with the probability of it being opposite. Still, this is a valid way to proceed.

E D I T: After I posted my answer, you edited the question from "different from 0" to "different (in the common sense)". Now this is a mathematical question and it needs a precise definition of different. "different in the common sense" is not precise. You can compute with numbers, not with senses. You will have to define, what difference is "different enough", and that is what you do, when you define the ROPE (which I advised to look up above). Trying to test, whether a true real value is really different from a point (e.g., zero) is one of the oddities of Frequentism, that brings people to Bayesian methods.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.