0
$\begingroup$

I have a question regarding non-parametric MLE as follows, and I am not sure whether my reasoning makes sense. enter image description here enter image description here

What I know about MLE usually concerns maximizing a formula of the form $\theta = \operatorname*{argmax}_\theta P(X|\theta)$, but now this model has no explicit parameters. I am not sure what we are maximizing in this case.

My first attempt would be trying to maximize $P(Y)$ and $P(X_i|Y)$ for $i = \{1,2,3\}$. Let $q = P(Y=1)$, then we can write the objective function as

$$\sum_{k=1}^{n} q^{y^{(k)}} (1-q)^{1-y^{(k)}} \prod_{i=1}^{3} P(X_i^{(k)}|y^{(k)}) $$

This is equivalent with maximizing

\begin{align} L & = \sum_{k=1}^{n} (y^{(k)}\log q + (1-y^{(k)}) \log (1-q) + \sum_{i=1}^{3} \log P(X_i^{(k)}|y^{(k)})) \\ & = \sum_{y^{(k)}=1}^{\#\{Y=1\}} \big[log q + \sum_{i=1}^{3}log P(X_i^{(k)}|1)\big] + \sum_{y^{(k')}=0}^{\#\{Y=0\}} \big[log (1-q) + \sum_{i=1}^{3}log P(X_i^{(k')}|0)\big] \\ & = \#\{Y=1\}log q + \sum_{y^{(k)}=1}^{\#\{Y=1\}} \sum_{i=1}^{3}log P(X_i^{(k)}|1) + \#\{Y=0\}log (1-q) + \sum_{y^{(k')}=0}^{\#\{Y=0\}} \sum_{i=1}^{3}log P(X_i^{(k')}|0) \end{align}

Taking the partial derivative $\displaystyle \frac{\partial L}{\partial q}$ and setting it to zero, I got $q = \displaystyle \frac{\#\{Y=1\}}{n}$, which matches with the solution. Is my reasoning so far correct?

$\endgroup$
0
$\begingroup$

Your steps and the overall likelihood are correct (except the typo: $\sum$ should be replaced by $\prod$), however cumbersome. The objective function to maximize has actually nothing to do with $X_i$, so we can cancel the unrelated part from the beginning:

$$L(q)\propto \prod_{k=1}^n q^{y_k}(1-q)^{1-y_k}$$ This would lead to the same answer as yours. Note this isn't a non-parametric method. The parameter $\theta$ is the prior probability coefficient, $q$. For $\pi_{iy}=P(X_i|Y=y)$, the situation is similar. While conducting further analysis, make sure you simplify the objective as much as possible.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.