My regression problem is properly formulated, but is encountering serious computational difficulties.

Dependent: $Y$ = multinomial

Independent: $X_1, \dots, X_{90}$ = linearly independent set of variables. (I verified the independence. Afterall, I defined these variables).

Consider design matrix $X$, Hessian $H$, and gradient $G$.


condition_number($H$) = $10^9$

Variance Inflation Factors (VIFs): all of them $ < 5.5$ except for one variable which has $VIF = 15.6$

eigenvalues$(H)$: ranges from $-10^{10}$ more-or-less-smoothly to $-17.3$

This causes parameter estimation to go whacky - any sort of Newton-Raphson approxmation encounters numerical problems when computing $ H^{-1} \cdot G$

Any suggestions or ideas?


The large condition number is saying that your independent data are not independent but are colinear to a dangerous degree. Look at the variance proportions of the largest condition index. See if some variables can be deleted. If not, you can look at partial least squares or ridge regression, or search on "colinearity" for other options.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.