1
$\begingroup$

I have a time-series dataset and I am a bit lost how to perform the analysis.

I have an dedependent variable and about 2000 independent variables for some entity over a time period of about 2500 daily observations (like daily consume behavior of an individual over time for 2000 products). In fact, those 2000 variables loosely "belong together". To be precise: Each of the 2000 variables can be assigned to one of three "main categories" (e.g., 600 variables belong to main category 1, 1200 belong to main category 2 and 200 to main category 3). Also, the variables belonging to one category are in most cases moderately to highly correlated.

Obviously, it won't make much sense to run a regression with 2000 independent variables. It is completely fine for my purpose to obtain just one coefficient for each "main category variable" (so three coefficients instead of 2000) in the end. However, I am unaware of techniques to "condense" my variables in three main variables before performing the final regressions. I cannot simply eliminate single variables from the setup to reduce the regressors and, for instance, choose only a subset of "most useful" variables.

Any ideas how to handle this problem?

Improve this question
$\endgroup$
2
  • $\begingroup$ Besides the two answers, another option is Partial Least Squares. $\endgroup$
    – Richard Hardy
    Oct 5, 2017 at 5:20
  • 1
    $\begingroup$ What do you want to do with the model? How does the time series aspect come into play? The PCA answer is pretty straightforward to apply if you can ignore the time dimension and treat your 2500 observations as independent. If however, you want to predict the time-series into the future, this won't help. Hence, what do you want to get out of your model? $\endgroup$
    – David Ernst
    Oct 5, 2017 at 9:24

2 Answers 2

Reset to default
2
$\begingroup$

You can first do the Principal Component Analysis of your independent variables and then only do the do the regression of your dependent variable w.r.t. the PCA coefficients. An alternative to that is to do Lasso regression, i.e. use the regularisation that forces to use the smallest possible number of variables in the regression.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ Maybe it would be good to elaborate how to apply PCA to time series data. $\endgroup$
    – Michael M
    Oct 5, 2017 at 6:46
0
$\begingroup$

You have several options for variable selection depending on what you want to do. But most of them are regularization methods.

Ridge Regression: uses regularization for variable selection to eliminate variables by having variables converge to zero the fastest get eliminated

LASSO adds a penalty strength (lambda) to variables that approach zero. Coefficients are set to zero as the lambda parameter increases.

This post would be useful to determine which technique to use When should I use lasso vs ridge?

If you have high amounts of data where there are more features than samples and large amount of correlated use Elastic Nets. Elastic net has a more ridged ridge trace than LASSO.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Ridge regression doesn't do variable selection -- it penalizes the L2 norm of the regression vector, which means you'll have small values (but the minima of the loss lie in dense regions of the parameter space). If you want few components you need an L1 approach like LASSO $\endgroup$
    – bibliolytic
    Oct 5, 2017 at 13:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.