I'm trying to fit some data to a Gaussian + Uniform mixture model.
This model has three parameters: the mean and standard deviation of the Gaussian, and the relative weights of the distributions (which sum to 1).
The idea underlying this data is that each observation is assumed to be drawn from either of these two distributions. The relative weights determine the probability that each observation was drawn from a particular distribution.
I'm using the mle function in Matlab to attempt to do this estimate (maximum likelihood estimation), and have made good progress, however I've reached a stumbling block, and I think it has to do with the way I've modeled the mixture distribution.
First, I've created some simulated data from these two distributions:
Data = [5+3*randn(2000,1); unifrnd(-100,100,2000,1)];
This creates observations drawn from both a Gaussian with Mean of 5, standard deviation of 3, and from a uniform distribution over the closed interval [-100 100]. As the data set contains equal observations from each distribution (each has 2000 observations), the corresponding relative weights for a model of this data are 0.5 and 0.5 for the Gaussian and Uniform distribution, respectively.
Next, I've defined my custom probability density function, which the mle function will then use to estimate the model parameters:
MixPDF = @(x,m,s,weight) (weight*normpdf(x,m,s) + (1-weight)*(1/length(x)));
here, x refers to the input vector for the function, and m and s are the mean and standard deviation of the Gaussian component. I've attempted to model the uniform component as a constant that is scaled by both the (1-weight) factor, and by the reciprocal of the length of the input vector. This is my attempt at normalizing the area under the uniform component to be 1, when the weight of the uniform component (1-weight) is 1.
However, I'm pretty sure I've not done this properly. I think what I need to do is specify the model in such a way that the area underneath the entire mixed distribution will always be equal to 1. And I'm not sure how to do this.
When I go ahead and use the mle procedure, as follows:
p = mle(Data, 'pdf', MixPDF, 'start', [0 1 0.5],'lowerbound', [-20 0.1 0], 'upperbound', [20 50 1])
I get an output that is somewhat accurate, but the accuracy is heavily dependent upon the way I scale the uniform component in the MixPDF model. The output also varies depending upon the interval over which I simulate the uniform component of the data.
I'd really appreciate any guidance on how to properly specify the model such that the area underneath the distribution is properly normalized (assuming this is my issue, of course!)
MixPDF = @(x,m,s,weight) (weight*normpdf(x,m,s) + (1-weight*(1/(Upper-Lower)));
I then tested the area under distribution with an input vector spanning [Lower Upper]. Given that my simulated data has this same range, I think that's why it works. I think I should be ok so long as any actual data I have falls within the range. Will have to think on this more, but thanks for clarifying. $\endgroup$ – spacediver Sep 21 '15 at 2:03