2
$\begingroup$

I would like to know why in the below formula the prior distribution of theta is not conditioned on X (observations):

enter image description here

In my understanding, the correct formula should be:

P(theta | X, y) = P(y| X, theta) P(theta| X) / P(y|X)

But I think I am missing something.

$\endgroup$
  • 1
    $\begingroup$ Why do you think this should be correct? $\endgroup$ – Tim Sep 9 at 17:21
  • 1
    $\begingroup$ P(theta | y) = P(y | theta) P(theta) / P(y) right? We added new condition to the formula P(theta | y, X). The entire formula has changed but the peior P(theta). Why does this make sense? If we condition to X, then both likelihood and prior should be conditioned to X. $\endgroup$ – Amir Jalilifard Sep 9 at 17:23
  • $\begingroup$ Is this some sort of regression where X is used to explain Y? $\endgroup$ – TrynnaDoStat Sep 9 at 17:49
4
$\begingroup$

This must be in the context of a regression model $P(y \mid X, \theta)$ where $X$ is considered a constant. This is usually done for reasons of inference, as explained in What is the difference between conditioning on regressors vs. treating them as fixed?.

Then, for inference on $\theta$ after seeing $X$, it does not matter if you condition $p(\theta)$ on $X$ or not, since you treat $X$ as fixed and known anyhow.

| cite | improve this answer | |
$\endgroup$
3
$\begingroup$

Regressors in regression are often assumed to be fixed. In other words, they are fixed, real vectors and not random variables. Typically, they are used to predict or explain the random variable $Y$. Notice, that regressors don't have to be fixed but this is the case in ordinary least squares and many of its variants.

Assuming this is regression and $X$ are regressors and $Y$ is the dependent variable, $P(\theta|X) = P(\theta)$ because $X$ is just some fixed value.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.