UPDATE : Based on the answer, I have tried to apply LASSO to a large MA model having 89 lags. I am trying to find out lambda using cross-validation. I have split the data into training and hold out sample denoted by the variables iTr and iHo respectively. I want to calculate the mean square error between the actual y and the predicted y obtained using the estimates. I am getting wrong values for MSE and unable to understand which estimated coefficients to use with the hold out samples in the lines `[r_hat_lasso, FitInfo] = lasso(X1(iTr,1:end), y(iTr));

[rhatLASSO,stats] = lasso(X1(iTr,2:end),y(iTr),'CV',10);

yLasso = X1(iHo,:)*rLasso; ` . I think I am incorreclty doing the cross validation step and incorrectly generating the predictions. The code is below:

UPDATE : Based on the answer, I have tried to apply LASSO to a large MA model having 89 lags. I am trying to find out lambda using cross-validation. I have split the data into training and hold out sample denoted by the variables iTr and iHo respectively. I want to calculate the mean square prediction error between the the hold out samples in y and the predicted y obtained using the estimates. I am getting wrong values for MSE and unable to understand which estimated coefficients to use with the hold out samples in the lines [r_hat_lasso, FitInfo] = lasso(X1(iTr,1:end), y(iTr));

[rhatLASSO,stats] = lasso(X1(iTr,2:end),y(iTr),'CV',10);

yLasso = X1(iHo,:)*rLasso;

I am getting NaN as the error. Need help with the correct method to use cross validation and calculate the MSE? The code is below:

UPDATE : Based on the answer, I have tried to apply LASSO to a large MA model having 89 lags. I am trying to find out lambda using cross-validation. I have split the data into training and hold out sample denoted by the variables iTr and iHo respectively. I want to calculate the mean square error between the actual y and the predicted y obtained using the estimates. I am getting wrong values for MSE and unable to understand which estimated coefficients to use with the hold out samples in the lines `[r_hat_lasso, FitInfo] = lasso(X1(iTr,1:end), y(iTr));

[rhatLASSO,stats] = lasso(X1(iTr,2:end),y(iTr),'CV',10);

yLasso = X1(iHo,:)*rLasso; ` . I think I am incorreclty doing the cross validation step and incorrectly generating the predictions. The code is below:

clear all
clc
    %Generate input
    N=200;
   x=(randn(1,N)*100);
    L = 90;
    Num_lags = 1:89;
    r = 1+randn(L,1);
    %Data preparation into regressors    
    r(rand(L,1)<.7)=0; % 70 of the coefficients are zero
    X1 = lagmatrix(x, [0 Num_lags]);

     y=X1*r ;
 
% %Estimation
 iTr = rand(N,1)<0.5; %training
 iHo = ~iTr;  % holdout
 

% %LASSO
 [r_hat_lasso, FitInfo] = lasso(X1(iTr,1:end), y(iTr));
 [rhatLASSO,stats] = lasso(X1(iTr,2:end),y(iTr),'CV',10);
 
% %Picking the hyper parameter, lambda
lassoPlot(rhatLASSO,stats,'PlotType','CV');
rLasso = [stats.Intercept(stats.Index1SE);rhatLASSO(:,stats.Index1SE)];

 stats.Index1SE
% 
% %ans =
% 
%  %   87
%  %Evaluate predictions on holdout  samples
  yLasso = X1(iHo,:)*rLasso;

%  %Assess prediction error
 fprintf('---MSE in holdout sample---\n');

  fprintf('MSE LASSO:  %f\n',mean((y(iHo)-yLasso).^2));

OUTPUT : MSE LASSO: NaN

My observation y is obtained from the model, $y(n) = \sum_{i=0}^{p-1} r(i) x(n-i) + v(n)$ where r is the sparse channel coefficients, x is the one dimensional input and v is additive White Gaussian noise of zero mean. y = filter(.) command is used to model the above equation and thus creating an FIR filter or a moving average (MA) model. The order of the MA model is p=3.

So, $y = [y(1),y(2),....,y(100)]$ is a vector of 100 elements. I am generating noise of variance 0.1 I want to estimate the sparse channel coefficients using LASSO. As there are p channel coefficients, I should get p estimates.

According to the equation of LASSO, ||rx - y||_2^2 + lambda * ||r||_1 I am estimating the sparse coefficients, r. As the true coefficient array contains p elements, I should get p estimated elements. I am not quite sure if this is the way to do. I have not found any example on LASSO being applied to univariate time series model such as ARMA. I don't know how to estimate the sparse coefficients using the appropriate algorithm and need help.

I don't know if the estimation is correct or not in estimating the sparse coefficients using the appropriate algorithm and need help.

Please find below the code for LASSO using Matlab function. As a toy example I am just assuming model order to be of lag 3 but I know that LASSO can be applied efficiently to a large model. I can test for larger order MA model having lag > 3.

% Code for LASSO estimation technique for 
%MA system, L = 3 is the order,  
     
%Generate input
 x = -5:.1:5;
 
r = [1    0.0   0.0];% L elements of the channel coefficients     
%Data preparation into regressors    
X1 = [ ones(length(x),1) x' x']; %first column treated as all ones since    x_1=1
   
y = filter(r,1,x); % Generate the MA model
[r_hat_lasso, FitInfo] = lasso(X1, y, 'alpha', 1, 'Lambda', 1, 'Standardize', 1);

OUTPUT :

The estimates returned are r_hat_lasso = 0, 0.657002829714982, 0

Question : This differs very much from the actual r. Is my understandin wrong?

My observation y is obtained from the model, $y(n) = \sum_{i=0}^{p-1} r(i) x(n-i) + v(n)$ where r is the sparse channel coefficients, x is the one dimensional input and v is additive White Gaussian noise of zero mean. y = filter(.) command is used to model the above equation and thus creating an FIR filter or a moving average (MA) model. The order of the MA model is p=3.

So, $y = [y(1),y(2),....,y(100)]$ is a vector of 100 elements. I am generating noise of variance 0.1 I want to estimate the sparse channel coefficients using LASSO. As there are p channel coefficients, I should get p estimates.

According to the equation of LASSO, ||rx - y||_2^2 + lambda * ||r||_1 I am estimating the sparse coefficients, r. As the true coefficient array contains p elements, I should get p estimated elements. I am not quite sure if this is the way to do. I have not found any example on LASSO being applied to univariate time series model such as ARMA. I don't know how to estimate the sparse coefficients using the appropriate algorithm and need help.

The first part of the Equation : $||rx - y||_2^2$ is a least squares formulation which I can solve using Least Squares Approach. In order to implement LS, I have to arrange the input in terms of regressors. However, if the coefficients, $\mathbf{r}$ are sparse then I should use LASSO approach. I have tried using Matlab's LASSO function. For LASSO, I rearranged the input data $x$ in terms of regressors, but I don't know if this the correct approach.

I need help. Is there an approach to include the sparsity term in the LS?

Please find below the code for LASSO using Matlab function. As a toy example I am just assuming model order to be of lag 3 but I know that LASSO can be applied efficiently to a large model. I can test for larger order MA model having lag > 3.

% Code for LASSO estimation technique for 
%MA system, L = 3 is the order,  
     
%Generate input
 x = -5:.1:5;
 
r = [1    0.0   0.0];% L elements of the channel coefficients     
%Data preparation into regressors    
X1 = [ ones(length(x),1) x' x']; %first column treated as all ones since    x_1=1
   
y = filter(r,1,x); % Generate the MA model
[r_hat_lasso, FitInfo] = lasso(X1, y, 'alpha', 1, 'Lambda', 1, 'Standardize', 1);

OUTPUT :

The estimates returned are r_hat_lasso = 0, 0.657002829714982, 0

Question : This differs very much from the actual r. Is my understandin wrong?