I had a course of Bayesian statistics, but I don't understand it at all. I calculated a posteriori distribution and Bayesian decision rules of something like that, but I don't understand what is this all about. For example in Bayesian statistics we assume that we've got a random variable $$X \sim p(\theta)$$ and $\theta$ is also a random variable: $$ \theta \sim p(\lambda).$$ We can also assume that $\lambda$ is a random variable as well $$\lambda \sim p(\mu)$$ and so on, so on, but we must stop at one point and assume that the parameter is just a number/vector of numbers. Let's assume that $$\lambda \sim p(\mu)$$ and $\mu$ is just a number/vector of numbers. Could you explain me if we estimate $\mu$ somehow or we have to just assume its value (for instance that $\mu=5$)?


Unless you’re using empirical Bayesian approach, you don’t “estimate” the priors. Prior is your best guess about the distribution of the parameter. If you are familiar with some previous research results, or theoretical considerations, that suggest what the distribution could be, then you use it. If you don’t, you guess. If you don’t have any clue at all, you can use vague priors, that are uniformish over wide range of parameter values, but still you need to make decisions on things like range of the values to consider.

How do you guess them? Based on the model itself, for example if you have the trivial regression model

$$ E[y|x] = \beta x $$

Than $\beta$ multiplied by possible values of $x$ should give you values alike the ones $y$ has. For example, say that $x$ is human height and $y$ is human weight. I guess, the higher people are, the more they weight, on average. According to Google, average human weight is 62 kg and average height is between 163 cm for women and 176.5 cm for men, so the prior we choose for $\beta$ should have mean roughly equal to

$$ 62 \approx \beta \times 170 $$

you can solve this for $\beta$ to find a reasonable value. This is an algebraic exercise. Next, you'd need to choose also the range of the prior, for example, if it is Gaussian, then it should have such standard deviation that the $\beta \pm 2\sigma$ range should be big enough that plugging-in different heights would give reasonable weights. This doesn't need to be super precise, since this is your initial guess that would be corrected, using Bayes theorem, when confronted with data. The above reasoning would not be based on your data, but on external sources of information, otherwise you would be using same data twice (in prior and likelihood) and be prone to overfitting to it.

  • $\begingroup$ Thank you for your reply. In your example $\beta$ is a fixed number and $x$ is a random variable or conversely? :P Could you give a numeric example? :) $\endgroup$ – P Lrc Jun 26 '20 at 18:49
  • $\begingroup$ @PLrc not exactly, see my edit for example. $\endgroup$ – Tim Jun 26 '20 at 20:07
  • $\begingroup$ It's easier to justifying using an uninformative prior rather than uniform one. $\endgroup$ – Neil G Jun 26 '20 at 20:30
  • $\begingroup$ Thank you for your explanations. I think I'm starting to understand it :) One more question: does it all what you said apply to nonhierarchical models as well? $\endgroup$ – P Lrc Jun 26 '20 at 21:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.