4
$\begingroup$

Suppose we have a sample from some population of people and we want to perform Bayesian regression of height vs weight using this sample. Suppose the true relationship between height $y$ and weight $x$ is: $$ y = \beta_0 + \beta_1 x^2 = 3 + 2x^2. $$ The plot of the sample data is: enter image description here

To implement a Bayesian linear regression model on this data I assume Gaussian priors $\beta_0 \sim \mathcal{N}(\mu_0,\sigma_0)$ and $\beta_1 \sim \mathcal{N}(\mu_1,\sigma_1)$ and that the error is normally distributed with mean zero and constant variance.

Now it remains to choose the parameters of the priors. Suppose I know a doctor who is an expert on the height/weight relationship so I visit him to obtain his expert knowledge.

Here is where I am having issues:

  1. The parameter $\mu_1$ represents the expected value of a variable that indicates how much a person's height increases as their weight squared goes up by one unit. Is a doctor really going to be able to give a good answer to this equation? We are dealing with a medical professional here, not a statistician/mathematician. I am taking a simplified example for the sake of convenience here, in practise we could have a multiple regression model with many predictors and be asking a field expert for their opinion on the mean of the several coefficients.
  2. The variable $\sigma_1$ is worse again. Are we really expected to believe that a medical professional will not just be able to provide a good guess for $\mu_1$ but that they can also give a good guess for the standard deviation of how much a person's height increases as their weight squared goes up by one unit? It seems way too complicated to ask this of statistician/mathematician?
  3. We also require the doctor to give an estimate for the covariance of $\beta_0$ and $\beta_1$. A doctor will know this?
  4. Suppose we extend our model to multiple regression of height $y$ vs weight $x$ and age $w$ (and assume a linear relationship for age): $$ y = \beta_0 + \beta_1 x^2 + \beta_2 w. $$ To incorporate the expert opinion of the doctor we now require him to to give us entries for the covariance matrix of $\beta_0,\beta_1$, and $\beta_2$. This seems to be a completely ridiculous proposition.

So how does it work in practise when a Bayesian statistican wants to generate priors for the regression coefficients based the opinion of some expert in a particular field? It seems to me that an expert could only be realistically be expected to give a decent estimate of $\mu_0$ and $\sigma_0$. Asking for the other information seems like a lost cause.

Yet incorporating expert information is often highlighted as one of the advantages of Bayesian over Frequentist statistics. Is it truly only a theoretical advantage, or can we actually make use of it in the real world?

If the difficulty is because I have somehow overcomplicated matters, can you explain to me what questions I should ask the doctor to get good parameters for the priors in the model for height vs weight? How exactly should I phrase the questions?

$\endgroup$
4
  • 4
    $\begingroup$ "If the difficulty is because I have somehow overcomplicated matters ... " No -- it is a genuine difficulty. How to choose priors is one of the more discussed topics in Bayesian statistics. $\endgroup$ Feb 10, 2021 at 13:55
  • $\begingroup$ @JohnColeman From what I can see the implications and motivations for choosing priors are often discussed but of the countless examples I've looked through hardly any, actually none as far as I can recall, discuss the real-world practicalities of obtaining these priors. Namely, that unless you are dealing with a statistician/mathematician the expert is not going to be in a position to give good answers. $\endgroup$
    – csss
    Feb 10, 2021 at 16:29
  • $\begingroup$ Suppose for the sake of example you were dealing with an expert who had the mathematical sophistication and domain knowledge to answer your questions. You ask them for their input on the coefficient priors. They will answer based on their knowledge of the literature. If they want to give a really good answer they will go consult a reference book or some paper to give a more accurate answer. But I could just get the information from the literature myself directly and bypass the expert completely. So it seems to me the answer to my question is to set priors based on results in the literature. $\endgroup$
    – csss
    Feb 10, 2021 at 16:34
  • 1
    $\begingroup$ It seems that "Elicitation" is used for the process whereby expert opinion is used to inform priors. For example, Elicitation by design in ecology: Using expert opinion to inform priors for Bayesian statistical models. For the reasons that you give, elicitation can't be as simple as asking an expert for the numerical value of a statistical parameter. $\endgroup$ Feb 10, 2021 at 17:12

1 Answer 1

2
$\begingroup$

This can be a challenging problem and one has to think quite carefully about it. There's multiple frameworks for how to elicit expert judgments such as SHELF or Delphi.

Asking non-statistical experts (and for that matter statisticians) about some quantity of interest in a way that is not a natural way of thinking for them is not a good idea. And, people mostly don't think in terms of regression equations (in fact it's a very unnatural way for most people). Instead, you could ask doctors for their judgements on the plausible heights a person that weighs $x$ kilograms and is $z$ years old. You can ask about a set of $(x, z)$ combinations that span the space of plausible values (i.e. you probably don't want to ask about totally unrealistic values) and identify the terms in your regression equation well - and perhaps even let you double-check whether your experts a-priori really believe in the functional form that you are proposing. The width of their distributions of course reflects both the uncertainty on coefficients, as well as $\sigma$, so it's important to ask about judgements when $y$ is low and high to clearly get at $\sigma$. It gets a bit trickier, if heteroskedascity is possible.

Doing it this way also ensures that you get a joint prior that avoids completely absurd prior beliefs (e.g. independent priors on $\beta_0$, $\beta_1$ and $\beta_2$ could possibly put a decent amount of prior weight on totally implausible predictions like heights <0 m or >3 m for humans).

$\endgroup$
3
  • $\begingroup$ This is another issue I have with Bayesian priors. If I ask a doctor for his opinion on the coefficient priors most likely he wont be able to give me an answer for the reasons above. He will then consult a book/literature and find an existing plot of height vs weight and base his opinion on that, after all that's where his info ultimately comes from. But I can do that myself without any need for the doctor at all. Expert opinions for every field can be found in the literature and will be far more accurate than some off the top of the head guess by some arbitrarily chosen real-world expert. $\endgroup$
    – csss
    Feb 10, 2021 at 16:22
  • $\begingroup$ So it seems the optimal way of getting the priors is to take them from the literature directly myself or indirectly via the doctor. But then my priors are biased towards whatever statistical study was used to generate the plot in some reference book/journal article. $\endgroup$
    – csss
    Feb 10, 2021 at 16:23
  • 1
    $\begingroup$ One possible approach for that concern is to give your experts an evidence dossier with the literature and ask them for their judgments on how your particular setting may differ. If the settings are totally identical, then using a previous large study to inform yours makes sense, of course (why ask people, when you have a lot of directly relevant data). $\endgroup$
    – Björn
    Feb 10, 2021 at 23:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.