Weighted linear regression from log-likelihood?

Question

I have 500 time-series and for each one, I compute the best (aka max likelihood) parameter α in a model that I'm testing, along with the corresponding log-likelihood.

Now I wish to uncover a linear relationship between my α and some other property of the time-series, say β. What are my options regarding linear regressors which would somehow take into account the log-likelihoods? In other words, how do I translate log-likelihood into some sort of weighting or error-bars in a rigorous and meaningful way?

In the example below, for instance, most high-alpha points have low log-likelihoods and the correct slope is therefore smaller than a simple linear regression would yield.

edit: For a more concrete example, let's assume my time-series are almost sinusoids with different periods $T$ and also have, say, a wavelength $\lambda$ assigned to them which should scale linearly with the period of oscillation. So my MLE routine returns $T_{1} = 42$ seconds with log-likelihood $1000$ and I know for this time series that $\lambda_{1} = 521 \pm 5$ nm. Some other series might return $T_{2} = 33$ seconds, log-likelihood $10$ and I know $\lambda_{2} = 314 \pm 7$ nm. How do I take the quality of the period estimate into account when fitting $T$ vs $\lambda$?

Answer 1

The log likelihood can be negative so you have to throw that idea out straight away.

You should probably use inverse variance weighting instead.

The idea is that if the $\alpha$ you estimated has a high variance, it should have a low weight. This might match your intuition since the likelihood of a linear regression model is a function of the variance.

In fact, inverse variance weighting in a linear regression model leads to the best linear unbiased estimate (BLUE) of the trend-line, if that matters at all (sometimes a little bias isn't so bad).

Answer 2

I don't think you should try to use log-likelihood as some kind of weighting, the reason is that you're comparing different sets of data.

If $\alpha_1$ is the max likelihood estimate (MLE) for data set 1 and $\alpha_2$ is the MLE for data set two, then these are the best for their data. The values of the likelihood may be different but they are the best parameters for their data.

If you're after error bounds on your $\alpha$ vs $\beta$ regression, bootstrap your data.

Better yet, you can set up a hierarchical model where you model the dependence between $\alpha$ and $\beta$ explicitly. Without more information it's hard to give a concrete example though.