# Maximum likelihood estimation under heteroskedasticity (and relation to OLS)

I have a question about MLE and how it relates to OLS. I know how to relate OLS and MLE when the noise is normal and homoskedastic. I can apply the same reason for heteroskedastic noise. My question is that, clearly, the noise terms are no longer identical (though still independent). So, we can apply MLE even when the distributions are not identical?

To make it more clear, if I take two samples, can I apply MLE when one sample's noise is normal, and the other's is Laplace? If not, why does MLE work for heteroskedastic noise?

Is it because, when estimating the parameter vector, we can simply take the different variances as known scaling factors* uniform variance $\sigma^2$, giving rise to the weights?

• Yes, you can apply maximum likelihood estimation (MLE) even if the distributions aren't identical. If you can write down the likelihood function, you can search for what parameters give the maximum likelihood. But you need to understand the data generating process, what drives the heteroskedasticity well enough to write down a likelihood function (that's sensibly parameterized). Eg. letting the variance of each observation $\sigma^2_i$ be a different, unknown parameter will almost certainly not be a useful approach. – Matthew Gunn Jan 31 '17 at 7:53
• Wait, but isn't that what we do in weighted least squares? Except for the unknown part (we assume each variance is known). Unless you are telling me that I can't estimate all the (unknown) parameters if they are all unknown (if that even makes sense). – learning Jan 31 '17 at 9:07
• Also, I guess from your explanation, I can apply MLE for heteroskedastic noise in my example because I have known variances and am estimating only one unknown. Is my reason for this correct (along with the final paragraph in the question)? – learning Jan 31 '17 at 9:10

If you are using R, there are some packages that allows for separate modeling of expectation and variance, among them, dglm and gamlss. See for instance Simulate linear regression with heteroscedasticity and Is it possible to calculate variable confidence intervals, conditional on $\hat{Y}$ to address heteroscedasticity?