Linked Questions
1
vote
0
answers
92
views
is there a geometric interpretation of canoncial correlation analysis? [duplicate]
I am looking for a geometric interpretation of CCA. Especially one that relies on the fact we are doing singular value decomposition, which has the geometric interpretation of a rotation, scaling and ...
- 11
84
votes
6
answers
33k
views
Is there any good reason to use PCA instead of EFA? Also, can PCA be a substitute for factor analysis?
In some disciplines, PCA (principal component analysis) is systematically used without any justification, and PCA and EFA (exploratory factor analysis) are considered as synonyms.
I therefore ...
- 849
53
votes
4
answers
23k
views
Why does the correlation coefficient between X and X-Y random variables tend to be 0.7
Taken from Practical Statistics for Medical Research where Douglas Altman writes in page 285:
...for any two quantities X and Y, X will be correlated with X-Y.
Indeed, even if X and Y are ...
- 1,487
97
votes
1
answer
118k
views
What correlation makes a matrix singular and what are implications of singularity or near-singularity?
I am doing some calculations on different matrices (mainly in logistic regression) and I commonly get the error "Matrix is singular", where I have to go back and remove the correlated variables. My ...
- 1,391
54
votes
3
answers
91k
views
Suppression effect in regression: definition and visual explanation/depiction
What is a suppressor variable in multiple regression and what might be the ways to display suppression effect visually (its mechanics or its evidence in results)? I'd like to invite everybody who has ...
- 56.6k
40
votes
4
answers
60k
views
X and Y are not correlated, but X is significant predictor of Y in multiple regression. What does it mean?
X and Y are not correlated (-.01); however, when I place X in a multiple regression predicting Y, alongside three (A, B, C) other (related) variables, X and two other variables (A, B) are significant ...
- 5,054
41
votes
2
answers
21k
views
How does Factor Analysis explain the covariance while PCA explains the variance?
Here is a quote from Bishop's "Pattern Recognition and Machine Learning" book, section 12.2.4 "Factor analysis":
According to the highlighted part, factor analysis captures the covariance between ...
- 3,501
51
votes
2
answers
70k
views
Multiple regression or partial correlation coefficient? And relations between the two
I don't even know if this question makes sense, but what is the difference between multiple regression and partial correlation (apart from the obvious differences between correlation and regression, ...
- 239
31
votes
4
answers
29k
views
What is the relationship between regression and linear discriminant analysis (LDA)?
Is there a relationship between regression and linear discriminant analysis (LDA)? What are their similarities and differences? Does it make any difference if there are two classes or more than two ...
- 861
21
votes
1
answer
33k
views
What is the proper association measure of a variable with a PCA component (on a biplot / loading plot)?
I am using FactoMineR to reduce my data set of measurements to the latent variables.
The variable map above is clear for me to interpret, but I am confused when ...
- 345
24
votes
1
answer
53k
views
Methods to compute factor scores, and what is the "score coefficient" matrix in PCA or factor analysis?
As per my understanding, in PCA based on correlations we get factor (= principal component in this instance) loadings which are nothing but the correlations between variables and factors. Now when I ...
- 453
27
votes
1
answer
15k
views
How LDA, a classification technique, also serves as dimensionality reduction technique like PCA
In this article
, the author links linear discriminant analysis (LDA) to principal component analysis (PCA). With my limited knowledge, I am not able to follow how LDA can be somewhat similar to PCA.
...
- 6,435
12
votes
3
answers
28k
views
Can the scaling values in a linear discriminant analysis (LDA) be used to plot explanatory variables on the linear discriminants?
Using a biplot of values obtained through principal component analysis, it is possible to explore the explanatory variables that make up each principle component. Is this also possible with Linear ...
- 1,713
12
votes
1
answer
9k
views
Data space, variable space, observation space, model space (e.g. in linear regression)
Suppose we have the data matrix $\mathbf{X}$, which is $n$-by-$p$, and the label vector $Y$, which is $n$-by-one. Here, each row of the matrix is an observation, and each column corresponds to a ...
- 1,082
10
votes
2
answers
2k
views
Is there an elegant/insightful way to understand this linear regression identity for multiple $R^2$?
In linear regression I have come across a delightful result that if we fit the model
$$E[Y] = \beta_1 X_1 + \beta_2 X_2 + c,$$
then, if we standardize and centre the $Y$, $X_1$ and $X_2$ data,
$$R^...
- 5,325