112 views

### When is the determinant of a covariance matrix is 0? [duplicate]

Any covariance matrix $A$ must be non-negative definite or semi-positive definite. This means that its deteraminant should always $|A|\ge0$. In case $|A|=0$, what would happen? or what does this mean ...
37k views

### Is there an intuitive explanation why multicollinearity is a problem in linear regression?

The wiki discusses the problems that arise when multicollinearity is an issue in linear regression. The basic problem is multicollinearity results in unstable parameter estimates which makes it very ...
23k views

### How to visualize what canonical correlation analysis does (in comparison to what principal component analysis does)?

Canonical correlation analysis (CCA) is a technique related to principal component analysis (PCA). While it is easy to teach PCA or linear regression using a scatter plot (see a few thousand examples ...
43k views

### Why does correlation matrix need to be positive semi-definite and what does it mean to be or not to be positive semi-definite?

I have been researching the meaning of positive semi-definite property of correlation or covariance matrices. I am looking for any information on Definition of positive semi-definiteness; Its ...
15k views

### How does Factor Analysis explain the covariance while PCA explains the variance?

Here is a quote from Bishop's "Pattern Recognition and Machine Learning" book, section 12.2.4 "Factor analysis": According to the highlighted part, factor analysis captures the covariance between ...
7k views

### What is an example of perfect multicollinearity?

What is an example of perfect collinearity in terms of the design matrix $X$? I would like an example where $\hat \beta = (X'X)^{-1}X'Y$ can't be estimated because $(X'X)$ is not invertible.
20k views

### Qualitative variable coding in regression leads to “singularities”

I have an independent variable called "quality"; this variable has 3 modalities of response (bad quality; medium quality; high quality). I want to introduce this independent variable into my multiple ...
47k views

### When can we speak of collinearity

In linear models we need to check if a relationship exists among the explanatory variables. If they correlate too much then there is collinearity (i.e., the variables partly explain each other). I am ...
9k views

### Is every correlation matrix positive definite?

I'm talking here about matrices of Pearson correlations. I've often heard it said that all correlation matrices must be positive semidefinite. My understanding is that positive definite matrices must ...
14k views

### What are the assumptions of factor analysis?

I want to check if I really understood [classic, linear] factor analysis (FA), especially assumptions that are made before (and possibly after) FA. Some of the data should be initially correlated and ...
8k views

### Why does Ridge Regression work well in the presence of multicollinearity?

I am learning about ridge regression and know that ridge regression tends to work better in the presence of multicollinearity. I am wondering why this is true? Either an intuitive answer or a ...
4k views

### Combining principal component regression and stepwise regression

I want to use a combination of principal component analysis (PCA) and stepwise regression to develop a predictor model. I have 5 independent variables (which are correlated among each other to ...
15k views

### Relationship Between Correlation and Multicollinearity [duplicate]

Suppose I've a model such as $Y = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \cdots + \beta_k X_k + \epsilon$. Now, there's high correlation between $X_1$ & $X_2$ (say over 60% but below 75%). Does ...