# Finding transformation $\mathbf{A}$ which minimizes $\sum_t \lVert \mathbf{x}(t) - \mathbf{Ay}(t) \rVert^2$

Let's say, I have a bunch of measurement vectors $\mathbf{x}(t)$ and $\mathbf{y}(t)$ for times $\{t_1,t_2...t_n\}$ and would like to compute the matrix A s.t. $$A =\underset{A}{\operatorname{argmin}} \sum_{t} \lVert \mathbf{x}(t) - \mathbf{A}\mathbf{y}(t) \rVert ^ 2$$

Let, $\mathbf{x}(t)$ be the $t^{th}$ column of a the matrix $\mathbf{X}$ and $\mathbf{y}(t)$ be the $t^{th}$ column of a the matrix $\mathbf{Y}$

My initial intuition/approach was that $\mathbf{A}$ should be a projection matrix that projects the columns of $\mathbf{Y}$ to the column space of $\mathbf{X}$, which would minimize the expression above. This gives me $\mathbf{A} = (\mathbf{X}^{\top}\mathbf{X})^{-1}\mathbf{X}^{\top}$.After some thinking, I've come up with a few counter examples to convince myself that the above solution does not result in the global minimum.

However, in this paper , appendix A2, $\mathbf{A} = \mathbf{X}\mathbf{Y}^{\top}(\mathbf{Y}\mathbf{Y}^{\top})^{-1}$ seems to be their solution.

Is there something trivial I am missing? How does one derive the expression from paper  above?

 Parra, Lucas C., et al. "Recipes for the linear analysis of EEG." Neuroimage 28.2 (2005): 326-341.

## 2 Answers

Given matrices $\mathrm A$ and $\mathrm B$, both with $n$ columns, we would like to find matrix $\mathrm X$ such that

$$\| \mathrm X \mathrm A - \mathrm B \|_{\text{F}}^2$$

is minimized. Differentiating the cost function with respect to $\mathrm X$ and finding where the derivative vanishes, we obtain the following linear matrix equation

$$\mathrm X \mathrm A \mathrm A^{\top} = \mathrm B \mathrm A^{\top}$$

If $\mathrm A$ has full row rank, then $\mathrm A \mathrm A^{\top}$ is invertible and the linear matrix equation has the unique solution

$$\hat{\mathrm X} := \mathrm B \mathrm A^{\top} (\mathrm A \mathrm A^{\top})^{-1}$$

Thank you both @Rodrigo de Azevedo and @Stefan Jorgensen. Here is my solution based on solution by @Rodrigo de Azevedo:

Turns out the Frobenius Norm of a matrix $A$ is equal to the square root of the matrix trace of $AA^H$ . If $A$ is a real matrix:

$$\lVert A \rVert_{F} = \sqrt{AA^H} = \sqrt{AA^{\top}}$$ So,
$$\underset{A}{\operatorname{argmin}} \sum_{t} \lVert \mathbf{x}(t) - \mathbf{A}\mathbf{y}(t) \rVert ^ 2 = \underset{A}{\operatorname{argmin}} \lVert \mathbf{X} - \mathbf{A}\mathbf{Y} \rVert _{F}^{2} = \underset{A}{\operatorname{argmin}} \DeclareMathOperator{\Tr}{Tr}[( \mathbf{X} - \mathbf{A}\mathbf{Y})(\mathbf{X} - \mathbf{A}\mathbf{Y})^{\top}]$$

From here we can take the derivative of the above objective function w.r.t. A, we can use the Eq. 119 from the matrix cookbook .

$$\frac{\partial}{\partial \mathbf{A}}\Tr[(\mathbf{AXB}+\mathbf{C})(\mathbf{AXB}+\mathbf{C})^{\top}] = 2\mathbf{A^{\top}}(\mathbf{AXB}+\mathbf{C})\mathbf{B^{\top}}$$

By substituting: $$\mathbf{I \rightarrow A, -A \rightarrow X, Y \rightarrow B, X \rightarrow C}$$ We Get: $$\frac{\partial}{\partial \mathbf{A}} \Tr[( \mathbf{X} - \mathbf{A}\mathbf{Y})(\mathbf{X} - \mathbf{A}\mathbf{Y})^{\top}] = \mathbf{0}$$ $$2 \mathbf{I}^{\top}(\mathbf{I}(\mathbf{-A})\mathbf{Y} + \mathbf{X})\mathbf{Y}^{\top} = \mathbf{0}$$ $$\mathbf{A}= \mathbf{XY}^{\top}(\mathbf{YY}^{\top})^{-1}$$
 http://mathworld.wolfram.com/FrobeniusNorm.html
 http://www2.imm.dtu.dk/pubdb/views/edoc_download.php/3274/pdf/imm3274.pdf

• Sorry about that! Fixed. – Lycan22 Jan 27 '17 at 5:05