0
$\begingroup$

I am reading this article an am a bit confused about one of the terms.

My understanding is we are trying to estimate a gaussian distribution Θ from which a certain value we are interested in is drawn.

We make an initial assumption, our "prior", about such distribution and label it p(Θ).

We know (by bayes' theorem) that p(Θ|data) ~ p(data|Θ)p(Θ). Where p(Θ|data) is our updated believe after observing some real world-data about the value we are interested in.

What I don't fully get is the meaning of p(data|Θ). This to me reads as "probability of observing the real value given it is drawn from its distribution". But why would this be a probability? Is it a probability simply because we assume our measurements would be noisy/subject to stochastic?

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ Think about a simple case: flipping coins where the true probability of "heads" is unknown. Call it $\theta$. The data are the sequence of tosses such as HTHHTTTH. The likelihood of the data is $\theta (1 - \theta) \theta \theta (1 - \theta) (1 - \theta) (1 - \theta) \theta$. The probabililty of observing what we observed is this function of $\theta$. In this case we have no noise in the observation (measurement error) and it's still a probability. $\endgroup$
    – Frank Harrell
    Jul 3, 2019 at 10:53

2 Answers 2

Reset to default
1
$\begingroup$

It is not noise per se, but your data has a data generating process associated with it, otherwise you will just be observing a constant value for your data. It could be a normal distribution with some unknown mean $\theta$ but known variance $\sigma^2$, so that your data has an associated probability density $N(\theta, \sigma^2)$.

I would define $p(\text{data}\rvert \theta)$ as the probability of observing the data that you have observed, given a certain value of the parameter $\theta$. It is not the "real value of data" as you've stated, because our data follows a data generating process and has no one "real value". The "real value" is associated with $\theta$, and we are trying to infer this "true value" of $\theta$ based on updating our prior beliefs with the data that we observe.

Improve this answer
$\endgroup$
0
$\begingroup$

In Bayesian theory, things on right side of the equation are related to prior, meaning that they are something about your initial belief. So $p(Data|\theta)$ should read given that your initial belief of the distribution of $\theta$ (a random variable has support [0,1]), the probability of observing the specific data which is the likelihood of the data.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.