I'm trying to solve a homework problem and I'm not getting the answer I expect.
The problem is from Pattern Classification by Duda,Hart,Stork (problem 3.47)
Consider $\mathcal{D} = \left\{\begin{pmatrix}1\\1\end{pmatrix},\begin{pmatrix}3\\3\end{pmatrix},\begin{pmatrix}2\\*\end{pmatrix} \right\}$, sampled from a two-dimensional (separable) distribution $p(x_1,x_2)=p_{1}(x_1)p_{2}(x_2)$, with \begin{equation} p_{1}(x) \sim \begin{cases} \frac{1}{\theta_1}e^{-x/\theta_1} & \text{if}~x \geq 0\\0 & \text{otherwise}\end{cases}. \end{equation} and \begin{equation} p_{2}(x) \sim \begin{cases} \frac{1}{\theta_2} & \text{if}~0 \leq x \leq \theta_2 \\ 0 & \text{otherwise}\end{cases} \end{equation} As usual, $*$ represents a missing feature value.
(a) Start with an initial estimate of $\theta^0=\begin{pmatrix}2\\4\end{pmatrix}$ and analytically calculate $Q(\theta; \theta^0)$ - The E step of the EM algorithm. Be sure to consider the normalization of your distribution.
(b) Find the $\theta$ that maximizes your $Q(\theta; \theta^0)$ - The M step.
$Q$ is defined as
\begin{equation} Q(\theta; \theta^i) = \mathbb{E}_{\mathcal{D}_b}\left[ \ln p(\mathcal{D}_g, \mathcal{D}_b ; \theta) | \mathcal{D}_g ; \theta^i \right] \end{equation}
where $\mathcal{D}_g$ is the "good" (observed) data and $\mathcal{D}_b$ is the "bad" (missing) data. Note that $\mathcal{D}=\mathcal{D}_g \cup \mathcal{D}_b$.
I'm assuming i.i.d. sampling and we are given that the features are independent. For this reason I would expect that EM converges to the maximum likelihood estimates which is $\theta=(2,3)^T$. So I expect my solution for part (b), $\theta^1$, to be closer to $(2,3)^T$ than $\theta^0$. However this doesn't seem to be the case.
Attempt
In this problem $\mathcal{D}_g$ contains 5 observations and $\mathcal{D}_b = \{ x_{32} \}$ is the missing data. I assume the domain of $Q$ is $\theta \in \mathbb{R}^2_{++}$. Also let $\theta = (\theta_1, \theta_2)^T$ and $\theta^0 = (\theta_0^1, \theta_0^2)^T$.
\begin{align} Q(\theta; \theta^i) &= \mathbb{E}_{\mathcal{D}_b}\left[ \ln p(\mathcal{D}_g, \mathcal{D}_b ; \theta) | \mathcal{D}_g ; \theta^0 \right] \\ &= \mathbb{E}_{x_{32}}\left[ \ln p(1,1;\theta) + \ln p(3,3;\theta) + \ln p_{1}(2;\theta_1) + \ln p_{2}(x_{32};\theta_2) | \mathcal{D}_g ; \theta^0 \right] \\ &= \ln p(1,1;\theta) + \ln p(3,3;\theta) + \ln p_{1}(2;\theta_1) + \mathbb{E}_{x_{32}} \left[ \ln p_{2}(x_{32};\theta_2) | \mathcal{D}_g ; \theta^0 \right] \\ \end{align}
Consider the expression in two parts $Q(\theta; \theta^0) = f(\theta) + g(\theta; \theta^0)$
where
\begin{align} f(\theta) &= \ln p(1,1;\theta) + \ln p(3,3;\theta) + \ln p_{1}(2;\theta_1) \\ &= \begin{cases} -3 \ln \theta_1 - \frac{6}{\theta_1} - 2 \ln \theta_2 & \text{if}~\theta_2 \geq 3 \\ -\infty & \text{otherwise} \end{cases}. \end{align}
assuming $\ln(0) = -\infty$, and
\begin{align} g(\theta; \theta^0) &= \mathbb{E}_{x_{32}} \left[ \ln p_{2}(x_{32};\theta_2) | \mathcal{D}_g ; \theta^0 \right] \\ &= \int_{-\infty}^{\infty} {\left( \ln p_{2}(x_{32};\theta_2)\right) p_{2}(x_{32} | \mathcal{D}_g ; \theta^0) dx_{32}} \end{align}
Note that $x_{32}$ is independent of $\mathcal{D}_g$ because i.i.d. and $p(x_1,x_2)=p_1(x_1)p_2(x_2)$. Therefore $p_{2}(x_{32} | \mathcal{D}_g ; \theta^0) = p_{2}(x_{32} ; \theta^0_2) = 1/4$ if $x_{32} \in [0,4]$, and $0$ otherwise so then
\begin{align} g(\theta; \theta^0) &= \frac{1}{4} \int_{0}^{4} \ln p_{2}(x_{32} ; \theta_2) dx_{32} \hspace{50pt} \text{(almost surely)} \\ &= \begin{cases} -\ln \theta_2 & \text{if}~\theta_2 \geq 4 \\ -\infty & \text{otherwise} \end{cases} \end{align}
Therefore \begin{equation} Q(\theta ; \theta^0) = \begin{cases} -3\ln\theta_1 - \frac{6}{\theta_1} - 3\ln\theta_2 & \text{if}~\theta_2 \geq 4 \\ -\infty & \text{otherwise} \end{cases} \end{equation}
Clearly $\theta^1 = \text{arg}\max_{\theta} Q(\theta ; \theta^0) = (2,4)^T$.
Again this solution disagrees with my belief that $\theta^1$ should be closer to $(2,3)$ than $\theta^0$. Is this belief wrong? Have I made a mistake somewhere? I really appreciate if someone could help me understand. I think there may be a mistake in the calculation of $g(\theta ; \theta^0)$ but I can't see it.