Generally, the uncertainty can be categorized into aleatory and epistemic according to whether it can be reduced or not. In Bayesian statistics, one "true fixed parameter" is presumed as discussions here, so it seems that the aleatory uncertainty in parameter is not considered.

To be more specific, let's see the Bayesian linear normal model. See Slide 16 here, the posterior variance of parameter $\beta$ is $\overline{V}=\frac{1}{V^{-1}+\sum x_{i}^{2}}$, in which $x_{i}$ is the independent variable. When more and more data $x_{i}$ are collected, the variance of $\beta$ can be arbitrarily small (Influence of $h$ is neglectable). Therefore, only epistemic uncertainty in $\beta$ is considered.

Here is my question: suppose $\beta$ has some aleatory uncertainty, for example $\beta$ can randomly change over time, is there a way to recover its aleatory uncertainty in Bayesian setting?

  • $\begingroup$ Greater clarity might be found if you considered that if $\beta$ is changing a Bayesian wouldn't use a single symbol for it. e.g. changes over time would be indicated by writing $\beta_t$ for its value at time $t$, and then its behavior would be described by writing down some model for how $\beta_t$ behaves over time (does it evolve over time, for example according to a random walk? Is it independently randomly selected at each time an observation is made? Does it vary cyclically? Is it related to some other variable?) $\endgroup$ – Glen_b -Reinstate Monica Jun 14 '15 at 2:27
  • $\begingroup$ A random walk assumption on $\beta_{t}$ can be found link, but the $\beta_{t}$ will have different mean and variance at different time instances. My concern is that, when $\beta$ is random but stationary, how we can model it. $\endgroup$ – bbl Jun 14 '15 at 7:17
  • $\begingroup$ Viewed this way, $\beta_t$ is not a single parameter, but an unobservable process whose values are themselves related by parameters. [See, for example, hidden Markov models for one example of the kind of thing that's possible.] --- once you think in these terms, the distinction you make disappears, and the original question shifts to one of modelling the way $\beta_t$ changes. There's all manner of models you might consider. You might come up with some model class from considering the problem at hand (eg. via subject knowledge), and then it reduces to estimation on your particular data. $\endgroup$ – Glen_b -Reinstate Monica Jun 14 '15 at 7:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.