Skip to main content

How is P$P(D;$\theta$D;\theta) = P(D|$\theta$D|\theta)$?

Tweeted twitter.com/StackStats/status/1100636819014729728
typos
Source Link
rranjik
  • 227
  • 1
  • 9

I started reading about maximum likelihood estimator and Bayesian statistics recently. I understand that given a statistical model $(X, (P_\theta))$ where, $\theta$ belongs to a large parameter space $\Theta$, the KL divergence between the $P_\theta$ and $P_\theta*$ ($\theta^*$ being the true parameter we would like to find) is minimised for the $\theta$ that maximises $\prod_{i=1}^{n}p_\theta(X_i)$. Assuming the events are independent and identically distributed, this amounts to maximising the the joint probability $P[X_1=x_1, X_2=x_2, ...,X_n=x_n].$ (the independence assumption allows to equate this to the product of the individual elements)

The Bayesian approach, accounts for the prior belief in the distribution of $\theta$, $P(\theta)$ and maximises $P(\theta|X)$, which by Bayes rule is equivalent to maximising, $P(X|\theta)P(\theta)/P(X)$. I understood things up to this part. After this, the $P(X|\theta)$ is called the "likelihood" and is replaced by $P[X_1=x_1, X_2=x_2, ...,X_n=x_n]$, which is just the product the individual probabilities of the X's in the distribution $P_\theta$. Does this mean that $P[X_1=x_1, X_2=x_2, ...,X_n=x_n]$ is actually $P_\theta[X_1=x_1, X_2=x_2, ...,X_n=x_n]$, i.e probabilities given $\theta$, or something like that ?

I'm not very good at probability and distributions, and my understanding is that the object $P(X|\theta)$ is called conditional probability, and the object $P[X_1=x_1, X_2=x_2, ...,X_n=x_n] (that\ equals \prod_{i=1}^{n}p_\theta(X_i) by\ independence) $$P[X_1=x_1, X_2=x_2, ...,X_n=x_n]$ (that equals $\prod_{i=1}^{n}p_\theta(X_i)$ by independence) is called the joint probability and they are very different things. I have seen authors use $P(X;\theta)$ for the joint probability in maximum likelihood in some cases. I'm confused why the joint probability and the conditional probability are considered to be equal ?

I started reading about maximum likelihood estimator and Bayesian statistics recently. I understand that given a statistical model $(X, (P_\theta))$ where, $\theta$ belongs to a large parameter space $\Theta$, the KL divergence between the $P_\theta$ and $P_\theta*$ ($\theta^*$ being the true parameter we would like to find) is minimised for the $\theta$ that maximises $\prod_{i=1}^{n}p_\theta(X_i)$. Assuming the events are independent and identically distributed, this amounts to maximising the the joint probability $P[X_1=x_1, X_2=x_2, ...,X_n=x_n].$ (the independence assumption allows to equate this to the product of the individual elements)

The Bayesian approach, accounts for the prior belief in the distribution of $\theta$, $P(\theta)$ and maximises $P(\theta|X)$, which by Bayes rule is equivalent to maximising, $P(X|\theta)P(\theta)/P(X)$. I understood things up to this part. After this, the $P(X|\theta)$ is called the "likelihood" and is replaced by $P[X_1=x_1, X_2=x_2, ...,X_n=x_n]$, which is just the product the individual probabilities of the X's in the distribution $P_\theta$. Does this mean that $P[X_1=x_1, X_2=x_2, ...,X_n=x_n]$ is actually $P_\theta[X_1=x_1, X_2=x_2, ...,X_n=x_n]$, i.e probabilities given $\theta$, or something like that ?

I'm not very good at probability and distributions, and my understanding is that the object $P(X|\theta)$ is called conditional probability, and the object $P[X_1=x_1, X_2=x_2, ...,X_n=x_n] (that\ equals \prod_{i=1}^{n}p_\theta(X_i) by\ independence) $ is called the joint probability and they are very different things. I have seen authors use $P(X;\theta)$ for the joint probability in maximum likelihood in some cases. I'm confused why the joint probability and the conditional probability are considered to be equal ?

I started reading about maximum likelihood estimator and Bayesian statistics recently. I understand that given a statistical model $(X, (P_\theta))$ where, $\theta$ belongs to a large parameter space $\Theta$, the KL divergence between the $P_\theta$ and $P_\theta*$ ($\theta^*$ being the true parameter we would like to find) is minimised for the $\theta$ that maximises $\prod_{i=1}^{n}p_\theta(X_i)$. Assuming the events are independent and identically distributed, this amounts to maximising the the joint probability $P[X_1=x_1, X_2=x_2, ...,X_n=x_n].$ (the independence assumption allows to equate this to the product of the individual elements)

The Bayesian approach, accounts for the prior belief in the distribution of $\theta$, $P(\theta)$ and maximises $P(\theta|X)$, which by Bayes rule is equivalent to maximising, $P(X|\theta)P(\theta)/P(X)$. I understood things up to this part. After this, the $P(X|\theta)$ is called the "likelihood" and is replaced by $P[X_1=x_1, X_2=x_2, ...,X_n=x_n]$, which is just the product the individual probabilities of the X's in the distribution $P_\theta$. Does this mean that $P[X_1=x_1, X_2=x_2, ...,X_n=x_n]$ is actually $P_\theta[X_1=x_1, X_2=x_2, ...,X_n=x_n]$, i.e probabilities given $\theta$, or something like that ?

I'm not very good at probability and distributions, and my understanding is that the object $P(X|\theta)$ is called conditional probability, and the object $P[X_1=x_1, X_2=x_2, ...,X_n=x_n]$ (that equals $\prod_{i=1}^{n}p_\theta(X_i)$ by independence) is called the joint probability and they are very different things. I have seen authors use $P(X;\theta)$ for the joint probability in maximum likelihood in some cases. I'm confused why the joint probability and the conditional probability are considered to be equal ?

typos
Source Link
rranjik
  • 227
  • 1
  • 9

I started reading about maximum likelihood estimator and Bayesian statistics recently. I understand that given a statistical model $(X, (P_\theta))$ where, $\theta$ belongs to a large parameter space $\Theta$, the KL divergence between the $P_\theta$ and $P_\theta*$ ($\theta^*$ being the true parameter we would like to find) is minimised for the $\theta$ that maximises $\prod_{i=1}^{n}p_\theta(X_i)$. Assuming the events are independent and identically distributed, this amounts to maximising the the joint probability $P[X_1=x_1, X_2=x_2, ...,X_n=x_n].$ (the independence assumption allows to equate this to the product of the individual elements)

The Bayesian approach, accounts for the prior belief in the distribution of $\theta$, $P(\theta)$ and maximises $P(\theta|X)$, which by Bayes rule is equivalent to maximising, $P(X|\theta)P(\theta)/P(X)$. I understood things up to this part. After this, the $P(X|\theta)$ is called the "likelihood" and is replaced by $P[X_1=x_1, X_2=x_2, ...,X_n=x_n]$, which is just the product the individual probabilities of the X's in the distribution $P_\theta$. Does this mean that $P[X_1=x_1, X_2=x_2, ...,X_n=x_n]$ is actually $P_\theta[X_1=x_1, X_2=x_2, ...,X_n=x_n]$, i.e probabilities given $\theta$, or something like that ?

I'm not very good at probability and distributions, and my understanding is that the object $P(X|\theta)$ is called conditional probability, and the object $P[X_1=x_1, X_2=x_2, ...,X_n=x_n] (that\ equals \prod_{i=1}^{n}p_\theta(X_i) by\ independence) $ is called the joint probability and they are very different things. I have seen authors use $P(X;\theta)$ for the joint probability in maximum likelihood in some cases. I'm confused why the joint probability and the conditional probabiltyprobability are considered to be equal ?

I started reading about maximum likelihood estimator and Bayesian statistics recently. I understand that given a statistical model $(X, (P_\theta))$ where, $\theta$ belongs to a large parameter space $\Theta$, the KL divergence between the $P_\theta$ and $P_\theta*$ ($\theta^*$ being the true parameter we would like to find) is minimised for the $\theta$ that maximises $\prod_{i=1}^{n}p_\theta(X_i)$. Assuming the events are independent and identically distributed, this amounts to maximising the the joint probability $P[X_1=x_1, X_2=x_2, ...,X_n=x_n].$ (the independence assumption allows to equate this to the product of the individual elements)

The Bayesian approach, accounts for the prior belief in the distribution of $\theta$, $P(\theta)$ and maximises $P(\theta|X)$, which by Bayes rule is equivalent to maximising, $P(X|\theta)P(\theta)/P(X)$. I understood things up to this part. After this, the $P(X|\theta)$ is called the "likelihood" and is replaced by $P[X_1=x_1, X_2=x_2, ...,X_n=x_n]$, which is just the product the individual probabilities of the X's in the distribution $P_\theta$. Does this mean that $P[X_1=x_1, X_2=x_2, ...,X_n=x_n]$ is actually $P_\theta[X_1=x_1, X_2=x_2, ...,X_n=x_n]$, i.e probabilities given $\theta$, or something like that ?

I'm not very good at probability and distributions, and my understanding is that the object $P(X|\theta)$ is called conditional probability, and the object $P[X_1=x_1, X_2=x_2, ...,X_n=x_n] (that\ equals \prod_{i=1}^{n}p_\theta(X_i) by\ independence) $ is called the joint probability and they are very different things. I have seen authors use $P(X;\theta)$ for the joint probability in maximum likelihood in some cases. I'm confused why the joint probability and the conditional probabilty are considered to equal ?

I started reading about maximum likelihood estimator and Bayesian statistics recently. I understand that given a statistical model $(X, (P_\theta))$ where, $\theta$ belongs to a large parameter space $\Theta$, the KL divergence between the $P_\theta$ and $P_\theta*$ ($\theta^*$ being the true parameter we would like to find) is minimised for the $\theta$ that maximises $\prod_{i=1}^{n}p_\theta(X_i)$. Assuming the events are independent and identically distributed, this amounts to maximising the the joint probability $P[X_1=x_1, X_2=x_2, ...,X_n=x_n].$ (the independence assumption allows to equate this to the product of the individual elements)

The Bayesian approach, accounts for the prior belief in the distribution of $\theta$, $P(\theta)$ and maximises $P(\theta|X)$, which by Bayes rule is equivalent to maximising, $P(X|\theta)P(\theta)/P(X)$. I understood things up to this part. After this, the $P(X|\theta)$ is called the "likelihood" and is replaced by $P[X_1=x_1, X_2=x_2, ...,X_n=x_n]$, which is just the product the individual probabilities of the X's in the distribution $P_\theta$. Does this mean that $P[X_1=x_1, X_2=x_2, ...,X_n=x_n]$ is actually $P_\theta[X_1=x_1, X_2=x_2, ...,X_n=x_n]$, i.e probabilities given $\theta$, or something like that ?

I'm not very good at probability and distributions, and my understanding is that the object $P(X|\theta)$ is called conditional probability, and the object $P[X_1=x_1, X_2=x_2, ...,X_n=x_n] (that\ equals \prod_{i=1}^{n}p_\theta(X_i) by\ independence) $ is called the joint probability and they are very different things. I have seen authors use $P(X;\theta)$ for the joint probability in maximum likelihood in some cases. I'm confused why the joint probability and the conditional probability are considered to be equal ?

Source Link
rranjik
  • 227
  • 1
  • 9
Loading