Questions tagged [condition-number]

The condition number (also known as condition index) is a diagnostic tool for collinearity in regression models. A regression model has as many condition numbers as it has independent variables. Each is defined as the square root of the ratio of the largest eigenvalue to the eigenvalue for the respective variable. Condition numbers over 30 are considered to be signs of problematic collinearity.

13 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
6
votes
0answers
5k views

Cause of a high condition number in a python statsmodels regression?

I'm pretty new to regression analysis, and I'm using python's statsmodels to look at the relationship between GDP/health/social services spending and health outcomes (DALYs) across the OECD. Just to ...
4
votes
0answers
415 views

Choosing the basis functions in a linear regression

I have two random variables $X$ and $Y$ and I'm trying to model $\mathbb{E}[Y|X]$. To this end, I'd like to pick a collection of functions $f_1, f_2 \dots f_n : \mathbb{R} \to \mathbb{R}$ and then ...
4
votes
0answers
306 views

Proximal Gradient Descent and Proximal Coordinate descent for Lasso Problem

Why is proximal coordinate descent much less affected by bad conditioning than proximal gradient descent? For example, we can consider this problem : $\min_x \frac{1}{2}\|Ax-b\|^2_2 + \lambda\|x\|_1$ ...
2
votes
0answers
346 views

Should the intercept be included when you check the condition index?

Many sources state that a condition index >30 constitutes a multicollinearity problem. When I've tried to implement this check in practice, I've realized that the condition index (and VIFs) change ...
1
vote
0answers
81 views

Is condition number important when fitting splines?

I am fitting data using p-splines. The authors of p-splines, Eilers and Marx, remark that there is no technical requirement to have a small number of knots and in fact you can have many more knots ...
1
vote
0answers
404 views

How to interpret differences in VIF and condition number?

In my present data, the Variance Inflation Factors suggest lack of substential multicolliniearity (<1,7). However, the condition number of 28 is almost at the critical value of 30. How do I ...
1
vote
0answers
753 views

Propogation of error in a matrix inversion

I'm trying to find the deterministic error bounds for some parameters calculated through distance geometry. The equation can be simplified to the following form: $ \left[\begin{matrix} x_1 \\ x_2 \\ ...
0
votes
0answers
18 views

Comparing numerical stability and computing bounds on the condition number of learned weights

I have an empirical risk minimization problem with two equivalent losses that solves it, $f_1(x; \theta_1)$ and $f_2(x ; \theta_2)$, where $x$ is the data and $\theta$ are the model parameters (in ...
0
votes
0answers
8 views

How to measure changes in condition indices over time

I am trying to understand how adding data, one observation at a time, affects the condition indices of a model. A similar question is how adding individual observations affects the principal ...
0
votes
0answers
15 views

QR decomposition used for estimating condition numbers

I have read that the QR decomposition is often used to estimate the condition number of a matrix but I don't understand... what is the benefit of using the QR decomposition for this? Is it purely ...
0
votes
1answer
135 views

Kappa condition number in R

I have read that the kappa function in R does not always explicitly calculate the condition number of a matrix, but rather, estimates the 2 norm of a matrix or a QR ...
0
votes
0answers
14 views

OLS Multicolleniarity

I have a pretty simple task to estimate ols multiple regression. I need a measure of multicolleniarity. Is condition number a good measure and what criteria exists fot its value?
0
votes
0answers
40 views

Problems with ill-conditioned matrix for Dantzig Selector

I'm using the Dantzig Selector on high dimensional data and my matrix is ill conditioned (condition number on the order of magnitude of 10^17). I know there are ways to improve the conditioning, ...