1
$\begingroup$

Let's say, I have a bunch of measurement vectors $\mathbf{x}(t)$ and $\mathbf{y}(t)$ for times $\{t_1,t_2...t_n\}$ and would like to compute the matrix A s.t. $$A =\underset{A}{\operatorname{argmin}} \sum_{t} \lVert \mathbf{x}(t) - \mathbf{A}\mathbf{y}(t) \rVert ^ 2$$

Let, $\mathbf{x}(t)$ be the $t^{th}$ column of a the matrix $\mathbf{X}$ and $\mathbf{y}(t)$ be the $t^{th}$ column of a the matrix $\mathbf{Y}$

My initial intuition/approach was that $\mathbf{A}$ should be a projection matrix that projects the columns of $\mathbf{Y}$ to the column space of $\mathbf{X}$, which would minimize the expression above. This gives me $\mathbf{A} = (\mathbf{X}^{\top}\mathbf{X})^{-1}\mathbf{X}^{\top}$.After some thinking, I've come up with a few counter examples to convince myself that the above solution does not result in the global minimum.

However, in this paper [1], appendix A2, $\mathbf{A} = \mathbf{X}\mathbf{Y}^{\top}(\mathbf{Y}\mathbf{Y}^{\top})^{-1}$ seems to be their solution.

Is there something trivial I am missing? How does one derive the expression from paper [1] above?

[1] Parra, Lucas C., et al. "Recipes for the linear analysis of EEG." Neuroimage 28.2 (2005): 326-341.

$\endgroup$
3
$\begingroup$

Given matrices $\mathrm A$ and $\mathrm B$, both with $n$ columns, we would like to find matrix $\mathrm X$ such that

$$\| \mathrm X \mathrm A - \mathrm B \|_{\text{F}}^2$$

is minimized. Differentiating the cost function with respect to $\mathrm X$ and finding where the derivative vanishes, we obtain the following linear matrix equation

$$\mathrm X \mathrm A \mathrm A^{\top} = \mathrm B \mathrm A^{\top}$$

If $\mathrm A$ has full row rank, then $\mathrm A \mathrm A^{\top}$ is invertible and the linear matrix equation has the unique solution

$$\hat{\mathrm X} := \mathrm B \mathrm A^{\top} (\mathrm A \mathrm A^{\top})^{-1}$$

$\endgroup$
3
$\begingroup$

Thank you both @Rodrigo de Azevedo and @Stefan Jorgensen. Here is my solution based on solution by @Rodrigo de Azevedo:

Turns out the Frobenius Norm of a matrix $A$ is equal to the square root of the matrix trace of $AA^H$ [1]. If $A$ is a real matrix:

$$\lVert A \rVert_{F} = \sqrt{AA^H} = \sqrt{AA^{\top}} $$ So,
$$\underset{A}{\operatorname{argmin}} \sum_{t} \lVert \mathbf{x}(t) - \mathbf{A}\mathbf{y}(t) \rVert ^ 2 = \underset{A}{\operatorname{argmin}} \lVert \mathbf{X} - \mathbf{A}\mathbf{Y} \rVert _{F}^{2} = \underset{A}{\operatorname{argmin}} \DeclareMathOperator{\Tr}{Tr}[( \mathbf{X} - \mathbf{A}\mathbf{Y})(\mathbf{X} - \mathbf{A}\mathbf{Y})^{\top}]$$

From here we can take the derivative of the above objective function w.r.t. A, we can use the Eq. 119 from the matrix cookbook [2].

$$\frac{\partial}{\partial \mathbf{A}}\Tr[(\mathbf{AXB}+\mathbf{C})(\mathbf{AXB}+\mathbf{C})^{\top}] = 2\mathbf{A^{\top}}(\mathbf{AXB}+\mathbf{C})\mathbf{B^{\top}}$$

By substituting: $$\mathbf{I \rightarrow A, -A \rightarrow X, Y \rightarrow B, X \rightarrow C}$$ We Get: $$\frac{\partial}{\partial \mathbf{A}} \Tr[( \mathbf{X} - \mathbf{A}\mathbf{Y})(\mathbf{X} - \mathbf{A}\mathbf{Y})^{\top}] = \mathbf{0}$$ $$2 \mathbf{I}^{\top}(\mathbf{I}(\mathbf{-A})\mathbf{Y} + \mathbf{X})\mathbf{Y}^{\top} = \mathbf{0}$$ $$\mathbf{A}= \mathbf{XY}^{\top}(\mathbf{YY}^{\top})^{-1}$$
[1] http://mathworld.wolfram.com/FrobeniusNorm.html
[2] http://www2.imm.dtu.dk/pubdb/views/edoc_download.php/3274/pdf/imm3274.pdf

$\endgroup$
  • $\begingroup$ Sorry about that! Fixed. $\endgroup$ – Lycan22 Jan 27 '17 at 5:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.