# Direct MLE vs Estimating a parameter and then doing MLE

I'm attempting to solve a channel estimation problem for a wireless communications system with equiprobable discrete transmitter inputs $x \in \{+1,-1\}$ and receiver output $y \in \mathbb{R}$. The channel multiplies $x$ by the random variable $h\in \mathbb{R}$ and AWGN $n\in \mathbb{R}$ is added to produce $y$

$$y = hx+n\,.$$

I have chosen to use the MAP detector to decide at the receiver which value of $x$ was transmitted given $y$

\begin{align} \hat{x} &= \underset{x}{\text{argmax}}\ p(x|y) \\ &= \underset{x}{\text{argmax}}\ p(y|x)p(x) \\ &= \underset{x}{\text{argmax}}\ p(y|x) \,. \end{align}

In general I see two approaches to solving a problem like this:

1) I could first compute an ML estimate of $h$ and then maximize $p(y|x,h)$

2) I could maximize $p(y|x)$ directly

The first approach appears to be easier to solve for my particular problem.

What I really need help with is this: It's not clear to me if approach 1 or 2 above produce equal or different estimates in general. Can someone show me how they are different or the same? Can we use the invariance of the MLE to somehow tie these two approaches together?

• In your first choice, if $h$ is uniform, the there isn't a unique ML estimator. Are you using of a posterior of some sort? Apr 22, 2018 at 5:02
• Sorry no. I didn't explain my assumptions for the estimation process of $h$. To estimate $h$ in approach 1, I assume a known training sequence $x_0$ is sent first and that over this training phase the value of $h$ remains constant. During this training phase the receiver estimates $h$ and then uses this value to detect the following unknown data $x$. The value of $h$ does not change during this entire process. When $h$ changes, the training process repeats itself. Does this help? Apr 22, 2018 at 5:47

From your explanation, you are indeed using a posterior distribution of $h$, namely $p(h|X)$ for training data $X$ and $h_{\textrm{MAP}}$ as an estimate for $h$. In general, without seeing a full description of your model, it's unlikely that the two techniques will be numerically equivalent. However, I suspect that the first approach is consistent, and if it is, then both should converge towards the same "true" values as the size of the training set increases. I suggest running some tests with sythentic data/simple model to verify this is indeed the case.
As Bjorn mentions, there is a risk of making incorrect predictions by using an estimated parameter while not specifying/modelling your uncertainty in that estimate. The second method handles this by integrating over all possible values of $h$. The resulting predictive distribution may be more conservative, but in doing so reflects the uncertainty represented in the training set. As such, I would expect the full method to have greater predictive power on unseen signals, especially if you make use of the whole predictive posterior via sampling rather than using its mode. In the first method, you run the risk to modelling the training data (and its particular error structure) too closely (overfitting) and losing predictive power, even if your model matches the training data exactly. Which highlights that there are two kinds of accuracy we might be interested in, accuracy with respect to training data, or accuracy when making predictions. For accuracy with respect to prediction, I would say the later method is at least as powerful as the former, and is likely more powerful, for a given level of uncertainty.
• Do I understand the difference between the two approaches properly though? The first approach amounts to estimating $h$ and then choosing the $x$ that maximizes $p(y|x,h)$. The second approach amounts to choosing the $x$ that maximizes $p(y|x)=\int p(y,h|x)dh=\int p(y|x,h)p(h|x)dh=E_{h|x}(p(y|x,h)|x)$. Thinking of it this way I feel the first approach might be more accurate if the estimate of $h$ is accurate, because I am not averaging over all possible $h$ to determine $x$. I would appreciate it if someone could help me understand the trade offs between these two approaches better. Apr 23, 2018 at 0:15
• I've editted my answer to address your questions. Does $h$ change with respect to the input $x$? If not, $p(h|x)=p(h)$, your prior on $h$. Apr 23, 2018 at 16:45
• $h$ and $x$ are independent, so $p(h|x)=p(h)$ as you mentioned. Apr 24, 2018 at 6:47