2
$\begingroup$

I have used an EM algorithm to fit a finite mixture of linear regression to my data, and cluster them into $k$ clusters.

Now that I have my clusters with the estimated parameters $\beta_k$ and $\sigma_k$, my question is how to use my model to predict new observations $x_{new}$?

I am thinking that I need to calculate the posterior probability of $p(z'=k|x_{new},\hat \theta)$, and somehow calculate the predicted $y'$ as $$y'=\sum_k p(z'=k|x') \; (\hat \beta_k^T x')$$

However, I am not sure if this is completely correct. Can anyone shed some light on this?

Improve this question
$\endgroup$
5
  • $\begingroup$ Your estimated parameters should be $\beta_k$ and $\sigma_k$ for each cluster $k$, as well as cluster weights $\lambda_k$ (i.e. prior probabilities over the mixture components). In that case your forecast is a weighted average of the forecasts from each component (weighted by $\lambda_k$). $\endgroup$
    – Adrian
    Commented Apr 13, 2017 at 15:25
  • $\begingroup$ @Adrian Then in this case we don't consider the probability of the new observation belonging to a cluster $k$, which sounds a little bit naive? (I am not sure maybe I am wrong). $\endgroup$
    – H_A
    Commented Apr 13, 2017 at 15:38
  • $\begingroup$ You do account for the probability of belonging to different clusters / components -- that's what the mixing proportions $\lambda_k$ are doing. In most models the mixing proportions (i.e. prior probabilities over mixture components) do not vary with $x$, although in a more general model they might. $\endgroup$
    – Adrian
    Commented Apr 13, 2017 at 16:20
  • $\begingroup$ @Adrian Thanks. What I was thinking was that we may be able to calculate the probability of the label ($p(z_{new}=k$)for the new observation based on the given data ($p(z_{new}=k|X)$, to get a new set of $\lambda_k$, and then use them as the mixing probabilities. But maybe I am mistaken. $\endgroup$
    – H_A
    Commented Apr 13, 2017 at 16:34
  • $\begingroup$ in a "plain vanilla" mixture of regressions, $\Pr[z = k | X = x] = \lambda_k$ for all $x$. Is that the case in your model, or are you doing something fancier where the lambdas vary with $X$? $\endgroup$
    – Adrian
    Commented Apr 13, 2017 at 17:57

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.