5
$\begingroup$

I've tried reading many explanations of CCA, and I don't understand it. For example, on Wikipedia, it refers to two "vectors" $a$ and $b$ such that $\rho = \text{corr}(a^{\top} X, b^{\top} Y)$ is maximal. But if $a$ and $X$ are vectors, isn't $a^{\top} X$ a scalar? What does it mean for two scalars to be correlated?

Other explanations use matrix notation without any dimensions, e.g.

enter image description here

Can someone explain CCA with a small example and all the dimensions provided?

$\endgroup$
4
  • 1
    $\begingroup$ X is n observations, p variables; a is p-long vector of coefficients. So the multiplication gives n-long vector. Likewise we obtain the n-long vector from Y and b. The correlation between the two vectors (called canonical pair of variates) is maximized. $\endgroup$
    – ttnphns
    Commented May 18, 2018 at 20:00
  • $\begingroup$ So in the Wikipedia example, when they describe X as a column vector, they mean a column vector in which each component is a p-vector, i.e. an n-by-p matrix? $\endgroup$
    – jds
    Commented May 18, 2018 at 20:12
  • $\begingroup$ Here is a simple visual explanation of CCA stats.stackexchange.com/q/65692/3277. And here is algorithm of it in algebraic notation stats.stackexchange.com/a/77309/3277. Might that do for you? Or do you just need a numeric example of doing CCA? $\endgroup$
    – ttnphns
    Commented May 21, 2018 at 6:38
  • $\begingroup$ This is a good resource $\endgroup$ Commented May 21, 2018 at 18:53

1 Answer 1

5
$\begingroup$

Canonical Correlation

In a uni/multivariate regression, one tries to predict observations (Y) based on a linear combination of predictor variables (Xb). This assumes a linear relationship between the predictor and observed variables.

Y = Xb

In this situation, X can be of any dimensions i.e. can contain any number of predictor variables in its columns, whereas Y is a vector of observations. The correlation value indicates how good the model in X explains observed data in Y. This is also referred as explained variance. b is a vector that maximizes the explained variance.

Canonical correlation asks a similar question in a more general context. It is most useful when both predictor (X) as well as observed data (Y) results from multiple data sources i.e. they are both matrices.

Example

Assume we collected some behavioral responses in an experiment. These responses can be described with 3 different characteristics such as speed, reliability, amplitude, etc. When we do this for N different individuals we would end up with a matrix of 3 by N. Assume now that for each of these N individuals, we have also 10 psychological variables (IQ, anxiety, impulsiveness, etc.) that we collected before the experiment, leading to 10 by N matrix. Note that both Y and X have the same number of observations, but may have different number of data sources. Now we can ask the question of how to understand the relationships between these variables. More precisely, how does psychological variables can be used to predict behavioral parameters? For example, does the data supports the hypothesis that a combination of anxiety and impulsiveness jointly predicts a faster and smaller behavioral responses?

Cannonical Correlation finds the best combination of both datasets in Y and X that maximizes their correlation. It creates a new set of variables, called canonical variables, U and V by linearly combining observations. The number of canonical variables depends on the number of data sources in X and Y. In the previous example, there will be 3 different canonical variables, which is the min(rank(X),rank(Y)).

U = Xa

V = Yb

The first column of U and V contains the first canonical variables. The key properties of U and V is that the correlation between the first canonical variables is the highest correlation you can achieve with this dataset when characterizing linear relationships. The second and third canonical values will be the second and third highest correlation values, respectively.

Canonical correlation can therefore be useful to understand how different set variables can be related to each other.

$\endgroup$
1
  • $\begingroup$ it is absolutely worth looking at the superb explanations here. $\endgroup$
    – bonobo
    Commented May 25, 2018 at 8:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.