Questions tagged [condition-number]

The condition number (also known as condition index) is a diagnostic tool for collinearity in regression models. A regression model has as many condition numbers as it has independent variables. Each is defined as the square root of the ratio of the largest eigenvalue to the eigenvalue for the respective variable. Condition numbers over 30 are considered to be signs of problematic collinearity.

Filter by
Sorted by
Tagged with
2
votes
1answer
120 views

Why does the condition number of the covariance matrix explode as number of variables increases?

From asset returns of $N$ stocks, the symmetric covariance matrix sized $N\times N$ is constructed, which treats the asset returns as variables. When the number of variables $N$ is fairly low like $N=...
0
votes
0answers
21 views

Condition number and leverage

Condition number is defined using the eigenvalues of $X^\top X$ while leverage values are the diagonal elements of the projection matrix. How are the two related? How can rescaling help, because aren'...
0
votes
0answers
9 views

Variance inflation factor vs condition number [duplicate]

We know the Variance inflation factor and condition number both help to measure multicollinearity. Which one should we use when?
0
votes
0answers
12 views

What is the meaning of the regressor characteristic root?

As described by Greene's Econometric Analysis (7th Edition), the regressor matrix's condition number measures how singular the matrix is. Therefore, the condition number is a measure of ...
0
votes
0answers
19 views

Condition numbers, invertibility and multicollinearity

The following is an excerpt from Greene's Regression Analysis (Seventh Edition): a) What does it mean to be "difficult" to invert a matrix accurately? Shouldn't all matrixes be either ...
0
votes
0answers
18 views

Does Ridge Regression generally improve the condition number?

While learning about Ridge regression and its applications I found a test question about impact of Ridge regression on the condition number. As far as I understand, Ridge regression can decrease the ...
1
vote
0answers
17 views

Multicollinearity negating $\beta$

In Regression coefficients that flip sign after including other predictors, ars's answer states that "Basically, if your variables are positively correlated, then the coefficients will be ...
1
vote
0answers
35 views

Comparing numerical stability and computing bounds on the condition number of learned weights

I have an empirical risk minimization problem with two equivalent losses that solves it, $f_1(x; \theta_1)$ and $f_2(x ; \theta_2)$, where $x$ is the data and $\theta$ are the model parameters (in ...
0
votes
0answers
21 views

QR decomposition used for estimating condition numbers

I have read that the QR decomposition is often used to estimate the condition number of a matrix but I don't understand... what is the benefit of using the QR decomposition for this? Is it purely ...
0
votes
1answer
239 views

Kappa condition number in R

I have read that the kappa function in R does not always explicitly calculate the condition number of a matrix, but rather, estimates the 2 norm of a matrix or a QR ...
2
votes
1answer
167 views

How are the condition numbers of a design matrix and its correlation matrix related?

Given a design matrix $X$ for a linear regression model, what is the relationship between the condition number of $X$ and its correlation matrix $R$? I would be interested in the case of a centered ...
3
votes
1answer
1k views

Why does matrix condition number change drastically when a constant is added?

If I create a regression model design matrix with 3 uncorrelated variables, I get a small condition number as expected. MWE: ...
4
votes
0answers
574 views

Choosing the basis functions in a linear regression

I have two random variables $X$ and $Y$ and I'm trying to model $\mathbb{E}[Y|X]$. To this end, I'd like to pick a collection of functions $f_1, f_2 \dots f_n : \mathbb{R} \to \mathbb{R}$ and then ...
3
votes
2answers
2k views

Deep Learning: Condition Number and Poor Conditioning

I am reading the following section of the book Deep Learning. Can you provide an intuitive explanation of the above section? I don't quite understand the statement "When this number is large, matrix ...
4
votes
0answers
329 views

Proximal Gradient Descent and Proximal Coordinate descent for Lasso Problem

Why is proximal coordinate descent much less affected by bad conditioning than proximal gradient descent? For example, we can consider this problem : $\min_x \frac{1}{2}\|Ax-b\|^2_2 + \lambda\|x\|_1$ ...
1
vote
0answers
82 views

Is condition number important when fitting splines?

I am fitting data using p-splines. The authors of p-splines, Eilers and Marx, remark that there is no technical requirement to have a small number of knots and in fact you can have many more knots ...
2
votes
2answers
3k views

Multicolinearity and Condition number of logistic regresison

It seems to be common to take a "high" condition number as a sign for multicolinearity in regression analysis. For linear models I'm totally convinced that this is a good idea, but is there any ...
10
votes
0answers
8k views

Cause of a high condition number in a python statsmodels regression?

I'm pretty new to regression analysis, and I'm using python's statsmodels to look at the relationship between GDP/health/social services spending and health outcomes (DALYs) across the OECD. Just to ...
1
vote
1answer
1k views

Condition number calculation in R

If I understood correctly, the condition number should be a product of Frobenious norms of a matrix and its inverse. In R if I do the following: ...
0
votes
1answer
229 views

Wrong choice of covariance function?

I am applying Gaussian process regression (GPR) model on some data assuming covSEiso (a.k.a. RBF) covariance function. I made sure there are no identical data points (but there are similar data ...
5
votes
2answers
891 views

When is it appropriate to override the default reciprocal condition number tolerance for solve() in R?

I am estimating a GMM IV model, where I'm creating a weighting matrix by taking the inverse of Z'Z, where Z is a matrix of instruments. For certain combinations of instruments, when I try to compute ...
2
votes
0answers
413 views

Should the intercept be included when you check the condition index?

Many sources state that a condition index >30 constitutes a multicollinearity problem. When I've tried to implement this check in practice, I've realized that the condition index (and VIFs) change ...
1
vote
0answers
44 views

Problems with ill-conditioned matrix for Dantzig Selector

I'm using the Dantzig Selector on high dimensional data and my matrix is ill conditioned (condition number on the order of magnitude of 10^17). I know there are ways to improve the conditioning, ...
1
vote
0answers
527 views

How to interpret differences in VIF and condition number?

In my present data, the Variance Inflation Factors suggest lack of substential multicolliniearity (<1,7). However, the condition number of 28 is almost at the critical value of 30. How do I ...
8
votes
2answers
11k views

How do you interpret the condition number of a correlation matrix

I have two correlation matrices, one with a condition number of 9 and the other with a condition number of 70. From what i have read, it will appear that the first matrix is better conditioned than ...
5
votes
1answer
174 views

Automatically fixing ill-conditioning or collinearity

I'm backtesting a regression model, which entails running it on a bunch of bootstrap samples of a "rewound" version of our data set. Unfortunately, in some of these resamplings, I end up getting some "...
1
vote
0answers
880 views

Propogation of error in a matrix inversion

I'm trying to find the deterministic error bounds for some parameters calculated through distance geometry. The equation can be simplified to the following form: $ \left[\begin{matrix} x_1 \\ x_2 \\ ...
1
vote
2answers
993 views

Condition number of data matrix and stability of OLS estimates

I have a multivariate regression model $Y=X\beta ' + \epsilon$. The variables in the $X$ matrix have very different scales and hence the condition number of $X'X$ is huge (order of trillions). I ...