7
$\begingroup$

I would be very grateful for some advice on how to model mixture distributions with R.

Given a problem to create a ranking of graduate students by their yearly income after completing their education, what are some suited models for this task?

Specifically, my data has a distribution with a point mass at 0 (the majority of graduates doesn't find, or start, a full-time job right away). The rest of the data is sort of nicely distributed. The data $x$ was transformed $\log(x+1)$. histogram

  • My first approach was a simple regression model
  • My second approach were two models (one for classifying whether they get a job - is very weak, and second to predict the income) Simply chaining these two models works much worse than the simple model.

My next step would be a Bayesian mixture model to predict income. I was thinking about fitting a mixture with 2 Gaussians, where I would set the mean for one of them to be known as equal to 0. Would that make sense? Has anyone a good experience with some package?

Another problem might be that I am always predicting the income using a regression and building a ranking from that, rather than running an ordinal regression. What is the best way handle this situation - if the target variable (income) that the ranking is based on is itself available for training data?

Full disclosure: This is a fictitious scenario, as I cannot discuss the exact details of the real case.

$\endgroup$
  • $\begingroup$ For fitting mixture models I use mixtools package in R and it works well $\endgroup$ – omidi Feb 4 '14 at 14:43
2
$\begingroup$

Mixture models of only two distributions, $D_1$ and $D_2$, can be considered to have the density $p D_1+(1-p) D_2$, where $0<p<1$. Now $p\neq0,1$ because it would no longer be a mixture. In general for any mixture distribution, software is available to find the best mixture model, e.g., FindDistribution in Mathematica, or mixtools in R as suggested by @omidi. However, in the case given by the OP, there is no overlap between distributions, as a salary of zero is not a salary. There is no particular need to add one to the income to take logarithms, as there is no need to take logarithms. Instead, all that need be done is to assign a Dirac $\delta$ for the $D_1$, zero salaries, i.e., $p \delta(x=0) +(1-p)D_2$, and then find $p$ and $D_2$. Finding $p$ is trivial, as $p=\frac{N_{no}}{N_{no}+N_{yes}}$, where $N_{yes}$ and $N_{no}$ are the number of subjects with and without income, respectively. Finding the best distribution for those with incomes can use software, or prior literature to search for models. However, even then, the distribution of those with income, $D_2$ here, can itself a mixture distribution. Moreover, use of an empirical distribution may be enough to answer the some of the questions the OP needs to answer, and although it's nice to have a theoretical distribution, it would not be absolutely required.

That is, the final distribution could be $p \delta(0) +(1-p)D_{Emp}(x)$, where $p=\frac{N_{no}}{N_{no}+N_{yes}}$. To be clear, that formulation is identical to what the data is, as the empirical distribution is $\frac{\delta(x_i)}{N_{yes}}$.

Other questions: "...set the mean for one of them [Sic, Gaussian distributions] to be known as equal to 0." That is what the Dirac $\delta(0)$ is; although the first distribution used to create it was historically Cauchy, it makes little difference which limit one uses to create the $\delta$, because its standard deviation is zero. Regarding using a Gaussian for the second distribution, not a good idea as the right tail of income is notoriously heavy, e.g., see Pareto distribution. If a theoretical distribution is desired in R for the OP's data for those having an income, see this.

Finally, I do not fully understand "Another problem might be that I am always predicting the income using a regression and building a ranking from that, rather than running an ordinal regression. What is the best way handle this situation - if the target variable (income) that the ranking is based on is itself available for training data?"

If one desires to predict income using regression, transforming variables may be desirable, but no transformation of $p \delta(0) +(1-p)D_{Emp}(x)$ will have "nice" residuals. That is, the residuals will not be Gaussian or homoscedastic, so the usual default regression techniques will not yield accurate answers. Maybe the way to treat this is to use classifiers to find the probability of finding a job with a yes/no Y-axis variable, and if yes use glm to find what that salary is for a best theoretical distribution of non-zero salaries. That is, one can then use classifiers to determine predictors of non-zero salary, and when finished, one then has two cases, predictors of getting a job, and predictors of salary if one secures a position.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.