Linked Questions

1
vote
0answers
44 views

is there a geometric interpretation of canoncial correlation analysis? [duplicate]

I am looking for a geometric interpretation of CCA. Especially one that relies on the fact we are doing singular value decomposition, which has the geometric interpretation of a rotation, scaling and ...
1011
votes
28answers
603k views

Making sense of principal component analysis, eigenvectors & eigenvalues

In today's pattern recognition class my professor talked about PCA, eigenvectors and eigenvalues. I understood the mathematics of it. If I'm asked to find eigenvalues etc. I'll do it correctly like ...
73
votes
6answers
21k views

Is there any good reason to use PCA instead of EFA? Also, can PCA be a substitute for factor analysis?

In some disciplines, PCA (principal component analysis) is systematically used without any justification, and PCA and EFA (exploratory factor analysis) are considered as synonyms. I therefore ...
49
votes
4answers
13k views

Why does the correlation coefficient between X and X-Y random variables tend to be 0.7

Taken from Practical Statistics for Medical Research where Douglas Altman writes in page 285: ...for any two quantities X and Y, X will be correlated with X-Y. Indeed, even if X and Y are ...
66
votes
1answer
70k views

What correlation makes a matrix singular and what are implications of singularity or near-singularity?

I am doing some calculations on different matrices (mainly in logistic regression) and I commonly get the error "Matrix is singular", where I have to go back and remove the correlated variables. My ...
34
votes
4answers
42k views

X and Y are not correlated, but X is significant predictor of Y in multiple regression. What does it mean?

X and Y are not correlated (-.01); however, when I place X in a multiple regression predicting Y, alongside three (A, B, C) other (related) variables, X and two other variables (A, B) are significant ...
40
votes
3answers
58k views

Suppression effect in regression: definition and visual explanation/depiction

What is a suppressor variable in multiple regression and what might be the ways to display suppression effect visually (its mechanics or its evidence in results)? I'd like to invite everybody who has ...
37
votes
2answers
14k views

How does Factor Analysis explain the covariance while PCA explains the variance?

Here is a quote from Bishop's "Pattern Recognition and Machine Learning" book, section 12.2.4 "Factor analysis": According to the highlighted part, factor analysis captures the covariance between ...
24
votes
4answers
19k views

What is the relationship between regression and linear discriminant analysis (LDA)?

Is there a relationship between regression and linear discriminant analysis (LDA)? What are their similarities and differences? Does it make any difference if there are two classes or more than two ...
20
votes
2answers
34k views

Methods to compute factor scores, and what is the “score coefficient” matrix in PCA or factor analysis?

As per my understanding, in PCA based on correlations we get factor (= principal component in this instance) loadings which are nothing but the correlations between variables and factors. Now when I ...
17
votes
1answer
15k views

What is the proper association measure of a variable with a PCA component (on a biplot / loading plot)?

I am using FactoMineR to reduce my data set of measurements to the latent variables. The variable map above is clear for me to interpret, but I am confused when ...
20
votes
1answer
11k views

How LDA, a classification technique, also serves as dimensionality reduction technique like PCA

In this article , the author links linear discriminant analysis (LDA) to principal component analysis (PCA). With my limited knowledge, I am not able to follow how LDA can be somewhat similar to PCA. ...
10
votes
3answers
19k views

Can the scaling values in a linear discriminant analysis (LDA) be used to plot explanatory variables on the linear discriminants?

Using a biplot of values obtained through principal component analysis, it is possible to explore the explanatory variables that make up each principle component. Is this also possible with Linear ...
9
votes
1answer
5k views

Data space, variable space, observation space, model space (e.g. in linear regression)

Suppose we have the data matrix $\mathbf{X}$, which is $n$-by-$p$, and the label vector $Y$, which is $n$-by-one. Here, each row of the matrix is an observation, and each column corresponds to a ...
9
votes
2answers
670 views

Is there an elegant/insightful way to understand this linear regression identity for multiple $R^2$?

In linear regression I have come across a delightful result that if we fit the model $$E[Y] = \beta_1 X_1 + \beta_2 X_2 + c,$$ then, if we standardize and centre the $Y$, $X_1$ and $X_2$ data, $$R^...

15 30 50 per page