Skip to main content
Adding some positive outcome from following the advice in the answer.
Source Link
stijn
  • 578
  • 3
  • 14

We were asked by a reviewer to provide p-values as to better understand the model estimates in our bayesian multilevel model. The model is a typical model of multiple observations per participant in an experiment. We estimated the model with Stan, so we can easily compute additional posterior statistics. Currently, we are reporting (visually and in tables) the mean estimate and the 0.025 and 0.975 quantiles.

My response so far would include:

  1. P-values are inconsistent with bayesian models, i.e. $P(X|\theta) \neq P(\theta|X).$
  2. Based on the posterior, we can calculate the probability of parameters being larger (smaller) than 0. This looks a bit like a traditional p-value.

My question is whether this is a response that can satisfy a reviewer or will it only cause more confusion?


Update 10-Oct: We rewrote the paper with the advice in the answer in mind. The paper is accepted so I will reiterate my earlier comment that this was really helpful advice!

We were asked by a reviewer to provide p-values as to better understand the model estimates in our bayesian multilevel model. The model is a typical model of multiple observations per participant in an experiment. We estimated the model with Stan, so we can easily compute additional posterior statistics. Currently, we are reporting (visually and in tables) the mean estimate and the 0.025 and 0.975 quantiles.

My response so far would include:

  1. P-values are inconsistent with bayesian models, i.e. $P(X|\theta) \neq P(\theta|X).$
  2. Based on the posterior, we can calculate the probability of parameters being larger (smaller) than 0. This looks a bit like a traditional p-value.

My question is whether this is a response that can satisfy a reviewer or will it only cause more confusion?

We were asked by a reviewer to provide p-values as to better understand the model estimates in our bayesian multilevel model. The model is a typical model of multiple observations per participant in an experiment. We estimated the model with Stan, so we can easily compute additional posterior statistics. Currently, we are reporting (visually and in tables) the mean estimate and the 0.025 and 0.975 quantiles.

My response so far would include:

  1. P-values are inconsistent with bayesian models, i.e. $P(X|\theta) \neq P(\theta|X).$
  2. Based on the posterior, we can calculate the probability of parameters being larger (smaller) than 0. This looks a bit like a traditional p-value.

My question is whether this is a response that can satisfy a reviewer or will it only cause more confusion?


Update 10-Oct: We rewrote the paper with the advice in the answer in mind. The paper is accepted so I will reiterate my earlier comment that this was really helpful advice!

Tweeted twitter.com/#!/StackStats/status/593028951506812929
Source Link
stijn
  • 578
  • 3
  • 14

How to respond to reviewers asking for p-values in bayesian multilevel model?

We were asked by a reviewer to provide p-values as to better understand the model estimates in our bayesian multilevel model. The model is a typical model of multiple observations per participant in an experiment. We estimated the model with Stan, so we can easily compute additional posterior statistics. Currently, we are reporting (visually and in tables) the mean estimate and the 0.025 and 0.975 quantiles.

My response so far would include:

  1. P-values are inconsistent with bayesian models, i.e. $P(X|\theta) \neq P(\theta|X).$
  2. Based on the posterior, we can calculate the probability of parameters being larger (smaller) than 0. This looks a bit like a traditional p-value.

My question is whether this is a response that can satisfy a reviewer or will it only cause more confusion?