# Closed-form solution to least squares with a matrix of parameters?

I'm familiar with the closed-form solution of ordinary least squares which minimizes $$\sum_{n=1}^N(y_n - \mathbf{\beta x_n})^2$$ for scalar $$y_n$$. However, in my situation I am trying to fit some data to a vector output through a matrix transformation, so I am instead looking to minimize the norm of the difference for each sample:

$$\min_B\sum_{n=1}^N\|\mathbf{y_n}-B\mathbf{x_n}\|^2$$

where $$\mathbf{y_n}$$ is now a vector and the parameters are now a matrix $$B$$. I suspect I am just unfamiliar with the name for this as searches of "vector response OLS" and "matrix OLS" keep bringing me the compact matrix formulation of OLS rather than the problem above.

• The key search term is "multivariate." Many people confuse this with "multiple regression," so eliminate "multiple," include "squares" to capture the OLS flavor of your problem, and check out the hits: stats.stackexchange.com/…. In light of this, what is your question? – whuber Feb 20 at 17:04
• The model you're looking for is called "Multivariate Multiple Regression" – Lucas Farias Feb 20 at 17:07

Ah, I just realized that I can run OLS on each output variable separately with the same input data and concatenate the resulting parameter vectors into $$B$$. The cost function is separable into the OLS cost of each output variable:
$$\|\mathbf{y_n}-B\mathbf{x_n}\|^2=\sum_{k=1}^{M}(y_{n,k}-\mathbf{B_k\mathbf{x_n})^2}$$
where $$\mathbf{B_k}\in \mathbb{R}^{1\times D}$$ is the k-th row of $$B$$. OLS minimizes each term, and the total will be minimized because the parameters of each term are independent.