Is there a theoretical maximum to the number of parameters that can be estimated with maximum likelihood estimation?

Question

Is there a theoretical upper limit to the number of parameters that be estimated with maximum likelihood estimation? My understanding is no, but that if you have too many parameters it may not be possible to find one set of parameters that uniquely optimizes the log-likelihood.

Assuming the above is correct, practically speaking, how can I determine if my model has too may parameters? Are there tests or guidelines regarding how many observed data points I need relative to my number of parameters?

Answer 1

Theoretical? No. Let’s look at an example where the number of parameters is unbounded.

If we assume $iid$ Gaussian error terms in a linear model, OLS coincides with maximum likelihood estimation. If there are more observations than regression parameters, then we have a model matrix $X$ that has full rank. Consequently, $(X^TX)$ exists, and the usual a$\hat\beta_{ols}=(X^TX)^{-1}X^Ty$ exists and is equivalent to maximum likelihood estimation.

Since there is no theoretical limit to the number of observations, there is no theoretical limit to the number of parameters that can be estimated.

Moving beyond this example, you are correct that having many parameters can result in the MLE not being unique, yes, but that does not keep MLEs from existing.