Skip to main content
added 5 characters in body
Source Link
Zen
  • 25.1k
  • 4
  • 86
  • 125

I'll use a simplified notation in this answer. If you're doing classical statistics, $\theta$ is not a random variable. Hence, the notation $p(x;\theta)$ is describing a member of a family of probability functions or densities $\{p_\theta(x)\}_{\theta\in\Theta}$, in which $\Theta$ is the parameter space. In a Bayesian analysis, $\theta$ is a random variable, and $p(x\mid\theta)$ is a conditional probability function or density, which models your uncertainty about $x$ for each possible value of $\theta$. After you're done with your experiment, there is no longer uncertainty about $x$ (it becomes data/information you know about), and you look at $p(x\mid \theta)=L_x(\theta)$ as a function of $\theta$, for this "fixed" data $x$. This likelihood function $L_x(\theta)$ lives in the intersection between the classical and Bayesian styles of inference. In my opinion, the Bayesian way is better understood in terms of conditional independence. I suggest that you write down and explore the likelihood function for the Bernoulli model; graph it; think about it's meaning before and after the experiment is conducted. You mentioned that a Bayesian maximizes the posterior $\pi(\theta\mid x)$. That's not necessarily the case. There are other ways to summarize the posterior distribution. Essentially, the chosen summary depends on the introduction of a loss function. Check Robert's Bayesian Choice to learn all the gory details.

I'll use a simplified notation in this answer. If you're doing classical statistics, $\theta$ is not a random variable. Hence, the notation $p(x;\theta)$ is describing a member of a family of probability functions or densities $\{p_\theta(x)\}_{\theta\in\Theta}$, in which $\Theta$ is the parameter space. In a Bayesian analysis, $\theta$ is a random variable, and $p(x\mid\theta)$ is a conditional probability function or density, which models your uncertainty about $x$ for each possible value of $\theta$. After you're done with your experiment, there is no longer uncertainty about $x$ (it becomes data/information you know about), and you look at $p(x\mid \theta)=L_x(\theta)$ as a function of $\theta$, for this "fixed" data $x$. This likelihood function $L_x(\theta)$ lives in the intersection between the classical and Bayesian styles of inference. In my opinion, the Bayesian way is better understood in terms of conditional independence. I suggest you write down and explore the likelihood function for the Bernoulli model; graph it; think about it's meaning before and after the experiment is conducted. You mentioned that a Bayesian maximizes the posterior $\pi(\theta\mid x)$. That's not necessarily the case. There are other ways to summarize the posterior distribution. Essentially, the chosen summary depends on the introduction of a loss function. Check Robert's Bayesian Choice to learn all the gory details.

I'll use a simplified notation in this answer. If you're doing classical statistics, $\theta$ is not a random variable. Hence, the notation $p(x;\theta)$ is describing a member of a family of probability functions or densities $\{p_\theta(x)\}_{\theta\in\Theta}$, in which $\Theta$ is the parameter space. In a Bayesian analysis, $\theta$ is a random variable, and $p(x\mid\theta)$ is a conditional probability function or density, which models your uncertainty about $x$ for each possible value of $\theta$. After you're done with your experiment, there is no longer uncertainty about $x$ (it becomes data/information you know about), and you look at $p(x\mid \theta)=L_x(\theta)$ as a function of $\theta$, for this "fixed" data $x$. This likelihood function $L_x(\theta)$ lives in the intersection between the classical and Bayesian styles of inference. In my opinion, the Bayesian way is better understood in terms of conditional independence. I suggest that you write down and explore the likelihood function for the Bernoulli model; graph it; think about it's meaning before and after the experiment is conducted. You mentioned that a Bayesian maximizes the posterior $\pi(\theta\mid x)$. That's not necessarily the case. There are other ways to summarize the posterior distribution. Essentially, the chosen summary depends on the introduction of a loss function. Check Robert's Bayesian Choice to learn all the gory details.

added 77 characters in body
Source Link
Zen
  • 25.1k
  • 4
  • 86
  • 125

I'll use a simplified notation in this answer. If you're doing classical statistics, $\theta$ is not a random variable. Hence, the notation $p(x;\theta)$ is describing a member of a family of probability functions or densities $\{p_\theta(x)\}_{\theta\in\Theta}$, in which $\Theta$ is the parameter space. In a Bayesian analysis, $\theta$ is a random variable, and $p(x\mid\theta)$ is a conditional probability function or density, which models your uncertainty about $x$ for each possible value of $\theta$. After you're done with your experiment, there is no longer uncertainty about $x$ (it becomes data/information you know about), and you look at $p(x\mid \theta)=L_x(\theta)$ as a function of $\theta$, for this "fixed" data $x$. This likelihood function $L_x(\theta)$ lives in the intersection between the classical and Bayesian styles of inference. In my opinion, the Bayesian way is better understood in terms of conditional independence. I suggest you write down and explore the likelihood function for the Bernoulli model; graph it; think about it's meaning before and after the experiment is conducted. You mentioned that a Bayesian maximizes the posterior $\pi(\theta\mid x)$. That's not necessarily the case. There are other ways to summarize the posterior distribution. Essentially, the chosen summary depends on the introduction of a loss function. Check Robert's Bayesian Choice to learn all the gory details.

I'll use a simplified notation in this answer. If you're doing classical statistics, $\theta$ is not a random variable. Hence, the notation $p(x;\theta)$ is describing a member of a family of probability functions or densities $\{p_\theta(x)\}_{\theta\in\Theta}$, in which $\Theta$ is the parameter space. In a Bayesian analysis, $\theta$ is a random variable, and $p(x\mid\theta)$ is a conditional probability function or density. After you're done with your experiment, there is no longer uncertainty about $x$ (it becomes data/information you know about), and you look at $p(x\mid \theta)=L_x(\theta)$ as a function of $\theta$, for this "fixed" data $x$. This likelihood function $L_x(\theta)$ lives in the intersection between the classical and Bayesian styles of inference. In my opinion, the Bayesian way is better understood in terms of conditional independence. I suggest you write down and explore the likelihood function for the Bernoulli model; graph it; think about it's meaning before and after the experiment is conducted. You mentioned that a Bayesian maximizes the posterior $\pi(\theta\mid x)$. That's not necessarily the case. There are other ways to summarize the posterior distribution. Essentially, the chosen summary depends on the introduction of a loss function. Check Robert's Bayesian Choice to learn all the gory details.

I'll use a simplified notation in this answer. If you're doing classical statistics, $\theta$ is not a random variable. Hence, the notation $p(x;\theta)$ is describing a member of a family of probability functions or densities $\{p_\theta(x)\}_{\theta\in\Theta}$, in which $\Theta$ is the parameter space. In a Bayesian analysis, $\theta$ is a random variable, and $p(x\mid\theta)$ is a conditional probability function or density, which models your uncertainty about $x$ for each possible value of $\theta$. After you're done with your experiment, there is no longer uncertainty about $x$ (it becomes data/information you know about), and you look at $p(x\mid \theta)=L_x(\theta)$ as a function of $\theta$, for this "fixed" data $x$. This likelihood function $L_x(\theta)$ lives in the intersection between the classical and Bayesian styles of inference. In my opinion, the Bayesian way is better understood in terms of conditional independence. I suggest you write down and explore the likelihood function for the Bernoulli model; graph it; think about it's meaning before and after the experiment is conducted. You mentioned that a Bayesian maximizes the posterior $\pi(\theta\mid x)$. That's not necessarily the case. There are other ways to summarize the posterior distribution. Essentially, the chosen summary depends on the introduction of a loss function. Check Robert's Bayesian Choice to learn all the gory details.

Source Link
Zen
  • 25.1k
  • 4
  • 86
  • 125

I'll use a simplified notation in this answer. If you're doing classical statistics, $\theta$ is not a random variable. Hence, the notation $p(x;\theta)$ is describing a member of a family of probability functions or densities $\{p_\theta(x)\}_{\theta\in\Theta}$, in which $\Theta$ is the parameter space. In a Bayesian analysis, $\theta$ is a random variable, and $p(x\mid\theta)$ is a conditional probability function or density. After you're done with your experiment, there is no longer uncertainty about $x$ (it becomes data/information you know about), and you look at $p(x\mid \theta)=L_x(\theta)$ as a function of $\theta$, for this "fixed" data $x$. This likelihood function $L_x(\theta)$ lives in the intersection between the classical and Bayesian styles of inference. In my opinion, the Bayesian way is better understood in terms of conditional independence. I suggest you write down and explore the likelihood function for the Bernoulli model; graph it; think about it's meaning before and after the experiment is conducted. You mentioned that a Bayesian maximizes the posterior $\pi(\theta\mid x)$. That's not necessarily the case. There are other ways to summarize the posterior distribution. Essentially, the chosen summary depends on the introduction of a loss function. Check Robert's Bayesian Choice to learn all the gory details.