Linked Questions

0
votes
0answers
84 views

When is the determinant of a covariance matrix is 0? [duplicate]

Any covariance matrix $A$ must be non-negative definite or semi-positive definite. This means that its deteraminant should always $|A|\ge0$. In case $|A|=0$, what would happen? or what does this mean ...
85
votes
9answers
32k views

Is there an intuitive explanation why multicollinearity is a problem in linear regression?

The wiki discusses the problems that arise when multicollinearity is an issue in linear regression. The basic problem is multicollinearity results in unstable parameter estimates which makes it very ...
70
votes
4answers
21k views

How to visualize what canonical correlation analysis does (in comparison to what principal component analysis does)?

Canonical correlation analysis (CCA) is a technique related to principal component analysis (PCA). While it is easy to teach PCA or linear regression using a scatter plot (see a few thousand examples ...
34
votes
3answers
38k views

Why does correlation matrix need to be positive semi-definite and what does it mean to be or not to be positive semi-definite?

I have been researching the meaning of positive semi-definite property of correlation or covariance matrices. I am looking for any information on Definition of positive semi-definiteness; Its ...
37
votes
2answers
14k views

How does Factor Analysis explain the covariance while PCA explains the variance?

Here is a quote from Bishop's "Pattern Recognition and Machine Learning" book, section 12.2.4 "Factor analysis": According to the highlighted part, factor analysis captures the covariance between ...
17
votes
2answers
19k views

Qualitative variable coding in regression leads to “singularities”

I have an independent variable called "quality"; this variable has 3 modalities of response (bad quality; medium quality; high quality). I want to introduce this independent variable into my multiple ...
12
votes
3answers
6k views

What is an example of perfect multicollinearity?

What is an example of perfect collinearity in terms of the design matrix $X$? I would like an example where $\hat \beta = (X'X)^{-1}X'Y$ can't be estimated because $(X'X)$ is not invertible.
15
votes
3answers
44k views

When can we speak of collinearity

In linear models we need to check if a relationship exists among the explanatory variables. If they correlate too much then there is collinearity (i.e., the variables partly explain each other). I am ...
10
votes
3answers
7k views

Is every correlation matrix positive definite?

I'm talking here about matrices of Pearson correlations. I've often heard it said that all correlation matrices must be positive semidefinite. My understanding is that positive definite matrices must ...
11
votes
3answers
11k views

What are the assumptions of factor analysis?

I want to check if I really understood [classic, linear] factor analysis (FA), especially assumptions that are made before (and possibly after) FA. Some of the data should be initially correlated and ...
13
votes
1answer
6k views

Why does Ridge Regression work well in the presence of multicollinearity?

I am learning about ridge regression and know that ridge regression tends to work better in the presence of multicollinearity. I am wondering why this is true? Either an intuitive answer or a ...
5
votes
1answer
4k views

Combining principal component regression and stepwise regression

I want to use a combination of principal component analysis (PCA) and stepwise regression to develop a predictor model. I have 5 independent variables (which are correlated among each other to ...
3
votes
2answers
14k views

Relationship Between Correlation and Multicollinearity [duplicate]

Suppose I've a model such as $Y = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \cdots + \beta_k X_k + \epsilon$. Now, there's high correlation between $X_1$ & $X_2$ (say over 60% but below 75%). Does ...
6
votes
2answers
500 views

Intuition for consequences of multicollinearity

So we have a regression equation with one explained variable and 10 explanatory variables. What I have read so far: Multicollinearity doesnt affect the regression of the model as a whole. But if we ...
6
votes
1answer
2k views

Does the first principal component differ from simply computing the mean of all variables?

I was just wondering if the first principal component, while I am trying to find it for a dataset of 18 variables, is different from simply adding all variables and finding the mean? I.e. to compute ...

15 30 50 per page