# Multiple dim responding variable the relation between CCA and trivial linear regression (PLSR, CCA, PCA, PCR and Linear Regression)

Here is my summary of Multivariate Linear Regression between explain variable $$\textbf{x}$$ and responding variable $$\textbf{y}.$$ I have summaried the relation between PLSR, CCA, PCA, PCR and Linear Regression.

However I still left one relation of the CCA and trivial regression which means that regress each column of $$\textbf{y}$$ respect to the sample matrix of $$\textbf{x}$$ and simply aggregates the coefficients (widely used in NIPLAS). Since I don't know how to implement the regression of CCA, I guess above two things are essentially equivalent?

Summary:

1. $$\textbf{PLSR:} \max\limits_{||\alpha_y|| = |\alpha_x|| = 1} Cov(\alpha_x\cdot\textbf{x},\ \alpha_y\cdot\textbf{y}).$$

PLSR simultaneously considers the variance of $$\textbf{x}$$, variance of $$\textbf{y}$$ and correlation between $$\textbf{x}$$ and $$\textbf{y}$$

Special case $$\dim \textbf{y} = 1: \textbf{PCR}.$$

2. $$\textbf{CCA:} \max\limits_{\alpha_x,\alpha_y}\rho(\alpha_x\cdot\textbf{x},\ \alpha_y\cdot\textbf{y})$$ $$\Leftrightarrow$$ $$\max\limits_{\alpha_x,\alpha_y}Cov(\alpha_x\cdot\textbf{x},\ \alpha_y\cdot\textbf{y})$$ $$s.t.\ Var(\alpha_x\cdot\textbf{x}) = Var(\alpha_y\cdot\textbf{y}) = 1.$$ CCA only consider the correlation between $$\textbf{x}$$ and $$\textbf{y}.$$

Special case $$\dim \textbf{y} = 1: \textbf{Linear regression}$$

3. $$\textbf{PCA:} \max\limits_{||\alpha_x|| = 1} Cov(\alpha_x\cdot\textbf{x},\ \alpha_x\cdot\textbf{x}).$$ PCA only consider the variance of $$\textbf{x}$$.

$$PCR = X \xrightarrow{PCA} Y \xrightarrow{MLR} \textbf{y}.$$