0
$\begingroup$

First of all I profusely apologise for the lack of suitable ways to express myself, I lack the formal data science background and am trying to learn as I go along, so finding the right terminology is a bit tricky for me.. I'm trying to run MLE across a great number of data sets (basically a matrix of pixels), with the idea to predict the optimum parameter of a known model in each pixel. According to the literature in my field, MLE should be the best estimator for a data closely related to mine (it has to do with counting statistics, hence Poisson distribution), and this indeed works fine. Overall, the estimated parameters across the entire matrix should roughly form a normal distribution.The very best I can do is get a histogram which looks like this:

enter image description here

As you see the estimator is hugely biased in favour of some values, and hence the overall image doesn't correspond to reality. It seems to be hitting the lower boundary of the estimated parameter, which I've set to 0.1. The greater counts I have, the less the bias is, but this is the maximum counts I can get. The loglikelihood for such poisson data I also got from the literature, and I got it to work fine for an established model, but not for the new model I am trying now. The model is overall an exponential decay (e^-x/t) with some additional parameters, I have simplified it as much as possible such that only t is to be estimated. I am minimizing the loglikelihood with Nelder-Mead, as it seemed to yield the best results with the established model.

Could anyone kindly point me to the right direction in terms of any suggestions what to try or even better what I have to read, since there is a lot of information on what MLE is and how it works, but troubleeshooting is tricky to start from scratch .. I've been working on this for months and now I am stuck, so any advice at all will be greatly appreciated.

EDIT: in response to questions below

The model is [ 1/3* I * (1 + 2(r*exp(-x/t)) ], where "I" is an array (another decay), r is a fixed parameter, x is the independent variable (time axis of the decay) and t is the parameter we want to find - t should show the rate of the decay basically.

the loglikelihood i am using (adapted from the literature for a similar model) is [ 2 * sum (y * log(y/yhat))) .. Where y is the actual raw data and yhat is the model with the predicted parameter to be optimised. It is essentially minimizing the residuals (sort of)..

Further clarification on the histogram: I have a matrix of elements (pixels), each pixel contains an array. Each array is an exponential decay. I am performing MLE on each and every array/pixel/element in this matrix (around 90 000 in total), by a nested for loop. When this is completed, there should be a parameter estimated for each pixel in this matrix, hence I am expecting around 90 000 estimated parameters. When plotted as a histogram, these should give normal distribution. However, the histogram above shows that too many of my optimizations (around 25 000) are hitting the low boundary of the estimated parameter, around 0.1, and some others (around 10 000) are biased for some specific value around 10. Ideally I am expecting a normal distribution around 4, but only a small amount of the fits obey this.

$\endgroup$
3
  • 3
    $\begingroup$ Can you write your model in more detail? That might help us understand the problem better. Also, write the log-likelihood that you are trying to maximize. $\endgroup$ Commented Oct 26, 2021 at 3:26
  • $\begingroup$ The model is [ 1/3* I * (1 + 2(r*exp(-x/t)) ], where "I" is an array, r is a fixed parameter, x is the independent variable (time axis of the decay) and t is the parameter we want to find - t should show the rate of the decay basically.. $\endgroup$
    – ISquared
    Commented Oct 27, 2021 at 0:46
  • $\begingroup$ the loglikelihood i am using (adapted from the literature for a similar model) is [ 2 * sum (y * log(y/yhat))) .. Where y is the actual raw data and yhat is the model with the predicted parameter to be optimised. It is essentially minimizing the residuals (sort of).. $\endgroup$
    – ISquared
    Commented Oct 27, 2021 at 8:27

1 Answer 1

0
$\begingroup$

I am confused with your plot. Your plot shows a histogram, but from question, it seems the parameter is coming from MLE.

If the parameter from MLE then there is only one number, why there is a distribution?

From the plot, it seems your data is a mixture of different classes.

For example, you are measuring the weight of cat, elephant and a coin. And you have a lot of exact same coins, which cause the spike on the left. In addition, the cats and elephants are roughly follow normal distribution, but with different variance.

Therefore, I would recommend to check your data, and build 3 models instead of one.

$\endgroup$
1
  • $\begingroup$ I addded further clarification to the question, thank you for taking the time! $\endgroup$
    – ISquared
    Commented Oct 27, 2021 at 8:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.