Return to Answer

The problem presented here is not a problem of comparing models of different dimensions. The model can be more formally specified as follows

$Z \sim \text{Bernoulli}(0.5)$

$X_1 \sim \text{Uniform}(0, \alpha)$

$X_2 | Z = 1, X_1=x_1 \sim \text{Uniform}(0, \alpha)$

$X_2 | Z = 0, X_1=x_1 \sim \delta_{x_1}$

$\delta_{x_1}$ is the Dirac measure or Dirac delta (density) function (point mass) at $x_1$; you can think of it as a $\text{Normal}(x_1, \sigma^2)$ distribution as $\sigma^2\to 0$. This is the key component missing from your original model.

Next we plug everything into Bayes formula

$P(Z | X_1, X_2) \propto P(Z)P(X_1 | Z)P(X_2 | Z, X_1)$

If $X_1=X_2$, then

$P(Z=1 | X_1=x_1, X_2 = x_1) \propto 0.5 (\alpha^{-1})\alpha^{-1} $

$P(Z=0 | X_1=x_1, X_2 = x_1) \propto P(Z=0, X_1=x_1, X_2 = x_1) = 0.5(\alpha^{-1}) \infty = \infty $, thus the posterior is a $\text{Bernoulli}(0.0)$.

Otherwise

$P(Z=1 | X_1=x_1, X_2 = x_2) \propto 0.5(\alpha^{-1})\alpha^{-1}$

$P(Z=0 | X_1=x_1, X_2 = x_2) \propto 0.5(\alpha^{-1})0 = 0$ implying the posterior is a $\text{Bernoulli}(1.0)$.

The problem presented here is not a problem of comparing models of different dimensions. The problem is that the model is incomplete. We can amend the model by defining a joint distribution for $X_1, X_2$ using a conditional distribution as follows:

$Z \sim \text{Bernoulli}(0.5)$

$X_1 \sim \text{Uniform}(0, \alpha)$

$X_2 | Z = 1, X_1=x_1 \sim \text{Uniform}(0, \alpha)$

$X_2 | Z = 0, X_1=x_1 \sim \delta_{x_1}$

$\delta_{x_1}$ is the Dirac measure or Dirac delta (density) function (point mass) at $x_1$; you can think of it as a $\text{Normal}(x_1, \sigma^2)$ distribution with $\sigma^2\to 0$. This is the key component missing from your original model.

Next we plug everything into Bayes formula

$P(Z | X_1, X_2) \propto P(Z)P(X_1 | Z)P(X_2 | Z, X_1)$

If $X_1=X_2$, then

$P(Z=1 | X_1=x_1, X_2 = x_1) \propto 0.5 (\alpha^{-1})\alpha^{-1} $

$P(Z=0 | X_1=x_1, X_2 = x_1) \propto P(Z=0, X_1=x_1, X_2 = x_1) = 0.5(\alpha^{-1}) \infty = \infty $, thus the posterior is a $\text{Bernoulli}(0.0)$.

Otherwise

$P(Z=1 | X_1=x_1, X_2 = x_2) \propto 0.5(\alpha^{-1})\alpha^{-1}$

$P(Z=0 | X_1=x_1, X_2 = x_2) \propto 0.5(\alpha^{-1})0 = 0$ implying the posterior is a $\text{Bernoulli}(1.0)$.

The Spike and slab (regression setting) model is an indirectly related topic that may be of interest to you. In short, it is a mixture model consisting of a discrete component and a continuous component.

The problem presented here is not a problem of comparing models of different dimensions. The model can be more formally specified as follows

$Z \sim \text{Bernoulli}(0.5)$

$X_1 \sim \text{Uniform}(0, \alpha)$

$X_2 | Z = 1, X_1=x_1 \sim \text{Uniform}(0, \alpha)$

$X_2 | Z = 0, X_1=x_1 \sim \delta_{x_1}$

$\delta_{x_1}$ is the Dirac measure or Dirac delta (density) function (point mass) at $x_1$; you can think of it as a $\text{Normal}(x_1, \sigma^2)$ distribution as $\sigma^2\to 0$. This is the key component missing from your original model.

Next we plug everything into Bayes formula

$P(Z | X_1, X_2) \propto P(Z)P(X_1 | Z)P(X_2 | Z, X_1)$

If $X_1=X_2$, then

$P(Z=1 | X_1=x_1, X_2 = x_1) \propto 0.5 (\alpha^{-1})\alpha^{-1} $

$P(Z=0 | X_1=x_1, X_2 = x_1) \propto P(Z=0, X_1=x_1, X_2 = x_1) = \infty $, thus the posterior is a $\text{Bernoulli}(0.0)$.

Otherwise

$P(Z=1 | X_1=x_1, X_2 = x_2) \propto 0.5(\alpha^{-1})\alpha^{-1}$

$P(Z=0 | X_1=x_1, X_2 = x_2) \propto 0.5(\alpha^{-1})0 = 0$ implying the posterior is a $\text{Bernoulli}(1.0)$.

The problem presented here is not a problem of comparing models of different dimensions. The model can be more formally specified as follows

$Z \sim \text{Bernoulli}(0.5)$

$X_1 \sim \text{Uniform}(0, \alpha)$

$X_2 | Z = 1, X_1=x_1 \sim \text{Uniform}(0, \alpha)$

$X_2 | Z = 0, X_1=x_1 \sim \delta_{x_1}$

$\delta_{x_1}$ is the Dirac measure or Dirac delta (density) function (point mass) at $x_1$; you can think of it as a $\text{Normal}(x_1, \sigma^2)$ distribution as $\sigma^2\to 0$. This is the key component missing from your original model.

Next we plug everything into Bayes formula

$P(Z | X_1, X_2) \propto P(Z)P(X_1 | Z)P(X_2 | Z, X_1)$

If $X_1=X_2$, then

$P(Z=1 | X_1=x_1, X_2 = x_1) \propto 0.5 (\alpha^{-1})\alpha^{-1} $

$P(Z=0 | X_1=x_1, X_2 = x_1) \propto P(Z=0, X_1=x_1, X_2 = x_1) = 0.5(\alpha^{-1}) \infty = \infty $, thus the posterior is a $\text{Bernoulli}(0.0)$.

Otherwise

$P(Z=1 | X_1=x_1, X_2 = x_2) \propto 0.5(\alpha^{-1})\alpha^{-1}$

$P(Z=0 | X_1=x_1, X_2 = x_2) \propto 0.5(\alpha^{-1})0 = 0$ implying the posterior is a $\text{Bernoulli}(1.0)$.

First, let's define the model

$Z \sim \text{Bernoulli}(0.5)$

$X_1 \sim \text{Uniform}(0, \alpha)$

$X_2 | Z = 1, X_1=x_1 \sim \text{Uniform}(0, \alpha)$

$X_2 | Z = 0, X_1=x_1 \sim \delta_{x_1}$

$\delta_{x_1}$ is the Dirac measure or Dirac delta (density) function (point mass) at $x_1$; you can think of it as a $\text{Normal}(x_1, \sigma^2)$ distribution as $\sigma^2\to 0$.

Next we plug everything into Bayes formula

$P(Z | X_1, X_2) \propto P(Z)P(X_1 | Z)P(X_2 | Z, X_1)$

If $X_1=X_2$, then

$P(Z=1 | X_1=x_1, X_2 = x_1) \propto 0.5 (\alpha^{-1})\alpha^{-1} $

$P(Z=0 | X_1=x_1, X_2 = x_1) = \infty $, thus the posterior is a $\text{Bernoulli}(0.0)$.

Otherwise

$P(Z=1 | X_1=x_1, X_2 = x_2) \propto 0.5(\alpha^{-1})\alpha^{-1}$

$P(Z=0 | X_1=x_1, X_2 = x_2) \propto 0.5(\alpha^{-1})0 = 0$ implying the posterior is a $\text{Bernoulli}(1.0)$.

The problem presented here is not a problem of comparing models of different dimensions. The model can be more formally specified as follows

$Z \sim \text{Bernoulli}(0.5)$

$X_1 \sim \text{Uniform}(0, \alpha)$

$X_2 | Z = 1, X_1=x_1 \sim \text{Uniform}(0, \alpha)$

$X_2 | Z = 0, X_1=x_1 \sim \delta_{x_1}$

$\delta_{x_1}$ is the Dirac measure or Dirac delta (density) function (point mass) at $x_1$; you can think of it as a $\text{Normal}(x_1, \sigma^2)$ distribution as $\sigma^2\to 0$. This is the key component missing from your original model.

Next we plug everything into Bayes formula

$P(Z | X_1, X_2) \propto P(Z)P(X_1 | Z)P(X_2 | Z, X_1)$

If $X_1=X_2$, then

$P(Z=1 | X_1=x_1, X_2 = x_1) \propto 0.5 (\alpha^{-1})\alpha^{-1} $

$P(Z=0 | X_1=x_1, X_2 = x_1) \propto P(Z=0, X_1=x_1, X_2 = x_1) = \infty $, thus the posterior is a $\text{Bernoulli}(0.0)$.

Otherwise

$P(Z=1 | X_1=x_1, X_2 = x_2) \propto 0.5(\alpha^{-1})\alpha^{-1}$

$P(Z=0 | X_1=x_1, X_2 = x_2) \propto 0.5(\alpha^{-1})0 = 0$ implying the posterior is a $\text{Bernoulli}(1.0)$.