# How to fit nonparametric mixtures of regression models in a statistical program? [closed]

I am interested in estimating a nonparametric finite mixture regression model, explained for example here:

https://methodology.psu.edu/media/techreports/09-93.pdf

But I don't know with which program the simulation study (p. 10) or the data set (p. 15) were analyzed.

Does anybody have an idea how to fit a nonparametric mixture regression model for example in R or SAS?

I can't speak for R but SAS has several approaches to nonparametric mixture regression. The core of them has to be PROC FMM (for finite mixture models). Here is an introduction to this approach that includes a comparison with kernel density estimation:

Finite mixture models provide a flexible framework for analyzing a variety of data. Suppose your objective is to describe the distribution of a response variable. If the corresponding data are multimodal, skewed, heavy-tailed, or exhibit kurtosis, they may not be representative of most known distributions. In this case, you often use a nonparametric method such as kernel density estimation to describe the distribution. A kernel density estimate generates a smoothed, numerical approximation to the unknown distribution function and estimates the distribution’s percentiles. Although this approach is useful, it might not be the most concise way to describe an unknown distribution. A finite mixture model provides a parametric alternative that describes the unknown distribution in terms of mixtures of known distributions. A finite mixture model also enables you to assess the probabilities of events or simulate draws from the unknown distribution the same way you do when your data are from a known distribution.

This quote is taken from this document: https://support.sas.com/resources/papers/proceedings12/328-2012.pdf

Of course your interest is in nonparametric mixture models but the suggestion of using kernel density estimation for nonparametric mixture modeling opens up a wide range of SAS options. Proc KDE is the core procedure for kernel density estimation. This paper introduces a macro for Smoothing-Spline-Based Functional Mixed Effects Models ... file:///C:/Documents%20and%20Settings/Administrator/My%20Documents/Downloads/v43c01.pdf

This paper describes an adaptation of Proc Mixed for spline smoothing in parametric and nonparametric mixture models... http://www2.sas.com/proceedings/sugi28/268-28.pdf

Here is the abstract:

Mixed models are an extension of regression models that allows for incorporation of random effects. The application of mixed-effects models to practical data analysis has greatly expanded with consequent development of theory and computer software. It also turns out that mixed models are closely related to smoothing. Nonparametric regression models, especially the general smoothing spline models, are well known for their ability to fit an arbitrary mean response function. This paper describes the use of the MIXED procedure for fitting nonparametric or semi-parametric regression models. Compared with such SAS procedures as PROC LOESS and PROC TPSPLINE, the use of the PROC MIXED allows the fitting of a wide spectrum of complex non-parametric and semiparametric regression models with simultaneous modeling of trends and covariance structure.

Google searches turned up many more papers of an ancillary nature in areas such as semi-parametric survival analysis, spline estimation with TSPLINE, kernel density clustering, and so on, all using SAS procedures.

• I appreciate your support. I am working with proc fmm at the moment and I have estimated a mixture of linear regression models. But reading the docs of proc fmm I couldn't find any hint how to implement the nonlinear version. Do you have any ideas? Regarding proc mixed, I checked the docs and I am not sure how to use it. Because I thought that one could apply a mixed model to consider that our data can be clustered by a "given" variable. But in my case I am interested in clustering the data. So I want to get the latent group variable. Is it possible via proc mixed? – Da_Stat May 20 '16 at 9:36
• Proc Mixed presumes pre-existing clusters. If your interest is in latent clustering, there is an external proc that does latent variable classification in the sense of Lazarsfeld latent clustering but its terrible. Just to be clear "Lazersfeld" latent clustering presumes categorical information is being used for the classification. If that's what you're referring to then there is an R module polCA that does that. A better software tool is Latent Gold which does latent mixtures of variables classification. Website here... statisticalinnovations.com – Mike Hunter May 20 '16 at 10:31