Least angle regression for a set of vectors?

As far as I know, LARS solves the following problem (using the same notation as Efron et al. Least angle regression):

Given a vector y, and a matrix X. Pick some column vectors from X, and express y as a linear combination of these column vectors.

In my problem, I have a set of vectors, which can form a matrix Y. I would like to express each column vector of Y as a linear combination of some column vectors from X. I hope that the fitting quality could be good (L2), and the number of column vectors I use from X is small (L1). Therefore, I am wondering if there is any systematic study (any papers?) on applying LARS to fit a set of vectors, rather than just one vector.

Below is my solution. Comments are highly appreciated.

Suppose Y = (y1 y2 ...) and X = (x1 x2 ...), where yi, xi are vectors. Treat Y as a high-dimensional 1D vector (flatten it) like

$Y' = \left(\begin{array}{c} y1\\ y2\\ ...\end{array}\right)$

Then we define the new basis matrix X' as (the blank part are all zeros)

$X' = \left(\begin{array}{ccc} X & & \\ & X &\\ & & ...\end{array}\right)$

Then we just apply LARS directly to fit Y' using X'. Once we have picked columns from X', we merge them into a set of columns from X, which will be the final "basis vector" set to fit Y.

One thing I feel uncomfortable about the above solution is that, it does not consider the fact that the same set of basis vectors from X will be used to express all column vectors of Y. Only the final step merges the result from LARS. Your help is appreciated. Thanks.

• The implication seems to be that you want to select the same columns of $X$ for all the response vectors, but allow for a different linear combination of the former for each of the latter. Is this your intent? – cardinal Feb 22 '12 at 16:04