# Non-parametric MLE?

I have a question regarding non-parametric MLE as follows, and I am not sure whether my reasoning makes sense.

What I know about MLE usually concerns maximizing a formula of the form $$\theta = \operatorname*{argmax}_\theta P(X|\theta)$$, but now this model has no explicit parameters. I am not sure what we are maximizing in this case.

My first attempt would be trying to maximize $$P(Y)$$ and $$P(X_i|Y)$$ for $$i = \{1,2,3\}$$. Let $$q = P(Y=1)$$, then we can write the objective function as

$$\sum_{k=1}^{n} q^{y^{(k)}} (1-q)^{1-y^{(k)}} \prod_{i=1}^{3} P(X_i^{(k)}|y^{(k)})$$

This is equivalent with maximizing

\begin{align} L & = \sum_{k=1}^{n} (y^{(k)}\log q + (1-y^{(k)}) \log (1-q) + \sum_{i=1}^{3} \log P(X_i^{(k)}|y^{(k)})) \\ & = \sum_{y^{(k)}=1}^{\#\{Y=1\}} \big[log q + \sum_{i=1}^{3}log P(X_i^{(k)}|1)\big] + \sum_{y^{(k')}=0}^{\#\{Y=0\}} \big[log (1-q) + \sum_{i=1}^{3}log P(X_i^{(k')}|0)\big] \\ & = \#\{Y=1\}log q + \sum_{y^{(k)}=1}^{\#\{Y=1\}} \sum_{i=1}^{3}log P(X_i^{(k)}|1) + \#\{Y=0\}log (1-q) + \sum_{y^{(k')}=0}^{\#\{Y=0\}} \sum_{i=1}^{3}log P(X_i^{(k')}|0) \end{align}

Taking the partial derivative $$\displaystyle \frac{\partial L}{\partial q}$$ and setting it to zero, I got $$q = \displaystyle \frac{\#\{Y=1\}}{n}$$, which matches with the solution. Is my reasoning so far correct?

## 1 Answer

Your steps and the overall likelihood are correct (except the typo: $$\sum$$ should be replaced by $$\prod$$), however cumbersome. The objective function to maximize has actually nothing to do with $$X_i$$, so we can cancel the unrelated part from the beginning:

$$L(q)\propto \prod_{k=1}^n q^{y_k}(1-q)^{1-y_k}$$ This would lead to the same answer as yours. Note this isn't a non-parametric method. The parameter $$\theta$$ is the prior probability coefficient, $$q$$. For $$\pi_{iy}=P(X_i|Y=y)$$, the situation is similar. While conducting further analysis, make sure you simplify the objective as much as possible.