8
$\begingroup$

I'm looking for some advice on selecting an appropriate forecasting model for panel data. I'm just starting out in the field and would appreciate any hints or rules of thumb to help make such decisions. Here's my particular case:

I have quarterly data on the incomes of 12 competing companies. Each company has three main sources of income: A,B,C (same for all of them). For each source and company, I have data starting from about 2010, so about 30 time points. The data is non-stationary. I think this kind of structure is called 'panel data'. I want to make forecasts for each company and source one time period ahead.

Additionally, it is predicted that the incomes depend on a set of market parameters. A set of 50 time series ("drivers", mostly various stock exchange indices etc.) is proposed, this data is freely available. Each of these 50 series falls into one of 10 categories

My questions are the following:

  1. The proposed set of potential drivers is too large. Before running any regression for the incomes, I want to reduce it to a more manageable amount. Is this reasonable? If yes, which method should I choose? I read about various dimension reduction procedures, but I don't really have any experience which would allow me to make a decision. Ideally I'd like something with a ready made R library, for example DFA - does this make sense in this context (nonstationarity, potential other irregularities).

  2. Suppose I managed to separate out several drivers using some dimension reduction technique. How would I structure my model now? Should I consider three separate models for A, B, C? It seems like I would lose something in doing that because the income sources are likely correlated. I thought about using a dynamic panel model (plm supports this). But the data isn't stationary, perhaps there are models that don't require this? Another thing is seasonality, since the data is quarterly it would make sense to account for this somehow - nowcasting?). I have zero knowledge about (dynamic) panel data models so I'd be grateful for any pointers.

  3. Does it make sense to separate the procedure in two steps (dimension reduction and fitting a panel model)? Does it potentially introduce many errors? Can and should the two steps be combined somehow? Are there perhaps more natural approaches to modelling such financial panel data? With so few data points I don't think it's sensible to model each company/source separately, I read that panel analysis has the advantage of not requiring many time points.

Edit: Here's what the data is structured like for clarity (not the real data, just generated random numbers in Excel). On top of this I have access to an external set of ~50 time series which would serve as potential predictors which can be classified into ~10 categories.

Data

Edit 2: In response to DJohnson's wonderful answer, I have a few more questions:

  1. MANOVA and CCA approaches are suggested once the relevant predictors are uncovered. I'm curious on how to modify these procedures to accommodate time series data. In other words, what is the procedure to go from an "ordinary" model to a time-sensitive one? How would the equations for the two proposed models change? Besides, I don't quite get why nominally ordered time would be appropriate.

  2. Going back to dimensionality reduction and predictor selection: I forgot to mention something relevant in my original question, I edited it in now, namely that each of the potential 50 predictors falls into one of 10 categories (one category would be for example trading volumes for various products). Series from different categories are still highly correlated, but perhaps there is a specific approach best suited for dealing with data characterized by such knowledge? This presentation mentions something called 'constrained factor models' which could perhaps be more suitable than PCA Lasso or Relaimpo? I should mention that I'm also interested in the model being generally well interpretable.

Improve this question
$\endgroup$
7
+50
$\begingroup$

Excellent question(s). Benny_dref's suggestions are useful. The first sentence of the OPs query is concerned with "selecting an appropriate forecasting model for panel data." With several DVs, more than a few predictors and a wide variety of possible methods and models to pursue, one has two broad avenues wrt model selection. The first is theoretical -- Wooldridge (referenced below) and the link to the DPM method(s) in the body of the OPs question provide guidance along those lines. The second avenue is empirical, e.g., which model has the most predictive power? explains the most variance? minimizes error? and/or minimizes AIC/BIC metrics? In other words, there are multiple possible criteria for model selection. You can pick one or triangulate across several metrics in distilling the information down to make that decision.

The (linked) DPM deck in the body of the OPs query is exhaustive in covering theoretical issues wrt small T panel data, GMMs and instrumental variables (IVs). Note, however, that their approach is confirmatory and presumes that many, if not all, of the OPs more applied and practical concerns have been addressed and answered. Also, its high level of technical rigor can take an inexperienced modeller down a bewildered rabbit hole. My recommendation would be to step back from immediate pursuit of their prescriptions by building up to them using less demanding, more easily digested and exploratory approaches, methods and models. For instance, instead of GMMs, why not build up to their use with simpler, more tractable, easily implemented and understood OLS models? Or consider IVs: econometricians prefer IV solutions to issues wrt endogeneity but noneconometricians question whether or not they create as many problems as they solve. Of course, these problems begin with simply identifying an appropriate IV(s). Why not put these theoretical, last step issues aside until you know more about the behavior of the data? This also raises the possibility of stepping outside a purely econometric framework to consider methods and models proposed in other disciplines.

As the OP notes in question 1), there are some exploratory issues to resolve wrt variable selection. PCA is widely employed but has the drawback of blurring the specificity inherent in the individual predictors. Why not use the Lasso for variable selection? Another approach to variable selection could be relative variable importance. Ulrike Groemping's RELAIMPO is one such (https://cran.r-project.org/web/packages/relaimpo/relaimpo.pdf). RELAIMPO is an R module that can be easily implemented within the framework of OLS regression and helpful in terms of generating qualitative insights wrt panel data. In addition, Groemping exhaustively reviews the many approaches to variable importance that have been proposed over the years.

Wrt 2), the three (A,B,C) correlated dependent variables (DVs) could be simultaneously modeled with canonical correlation or MANOVA-type approaches. This would be informative as it would tell you how the importance of a predictor varies as a function of the DV. Assuming the DVs are non-negative, a useful normalizing transformation different from first-differencing would be the natural log. There are many advantages to natural logs including the fact that the resulting parameters are expressed as elasticities and they minimize retransformation bias when converting predictions back into their original units. If the DVs contain negative values Lee Cooper suggests log-centering (Market Share Analysis, http://www.anderson.ucla.edu/faculty/lee.cooper/MCI_Book/BOOKI2010.pdf). Cooper's book is a goldmine of applied ideas and suggestions wrt exploring panel data structures (his term is pooled time series models).

Wrt 3), the short answer is yes, a multi-step, exploratory process makes sense, until you know more...much more.

Within the context of econometric literature, Wooldridge's Econometric Analysis of Cross Section and Panel Data is the "go-to" resource. His book has the disadvantage of being almost purely theoretical with little in the way of practical, applied guidance. A good complement to Wooldridge is Badi Baltagi's Econometric Analysis of Panel Data which considers the inevitably messy nature of real world data.

M. Hashem Pesaran's papers cover many panel data model issues not otherwise discussed in the econometric literature such as weak cross-sectional dependence and nonstationarity. I would recommend a google search using 'filetype:pdf hashem pesaran' to uncover his published papers.

Linear models are not the only game in town. Weiss' reviews nonlinear mixed effects models in his lecture notes (http://www.unc.edu/courses/2008fall/ecol/563/001/docs/lectures/lecture27.htm). Similarly, Rob Hyndman's suggestion of 'boosted additive quantile regression' is another excellent example of a rigorous methodology for exploring complex, nonlinear panel data structures (http://ieeexplore.ieee.org/document/7423794/). An additional advantage to BAQR is that, being nonparametric and distribution free, it controls for extreme values (aka outliers) much more effectively than a natural log transformation which, in the presence of truly extreme valued data, does not capture heavy-tails.

Econometric panel data models are only one type in the broader class of event data modelling. There is a large literature on the closely related topics of growth and hierarchical models, e.g., Gelman and Hill's Data Analysis Using Regression and Multilevel/Hierarchical Models, Singer and Willett's Applied Longitudinal Data Analysis or Raudenbush and Bryk's Hierarchical Linear Models are all excellent references covering a broad class of noneconometric models and methods including marketing science, education, health, the environment, and more.

Improve this answer
$\endgroup$
7
  • 1
    $\begingroup$ Thank you very much for your detailed answer. I suppose the first thing I'll do is try the MANOVA (or MANCOVA? would that be better?) and CCA approaches. I understand that the two vectors to analyze would be (A,B,C) and (50 predictors). What exactly should I look for in these models to guide my further approach? $\endgroup$ – Spine Feast Jan 23 '18 at 8:32
  • 1
    $\begingroup$ Also: you suggested to MANOVA and CCA. But I don't know how to adapt these methods to time series data. Surely one can't just ignore the time ordering? $\endgroup$ – Spine Feast Jan 23 '18 at 10:42
  • 1
    $\begingroup$ I'm agnostic wrt MANOVA vs MANCOVA. Assuming you are leveraging an empirical (not theoretical) exploratory approach, time can be specified as discrete or continuous. As part of your exploration, why not try several? And identify which has the most predictive power. Nominally scaled, discrete time would introduce an anova-type factor in MANOVA (MANCOVA) and a set of 0,1 dummy variables in CCA. Continuous time would be the same for both frameworks. These could include linear or deterministic time as well as various polynomial functions (quadratic, cubed, etc.). $\endgroup$ – Mike Hunter Jan 23 '18 at 13:01
  • $\begingroup$ 1) It may be that only one or two years in your series are above (below) the grand mean. If so, treating time as noncontinuous would identify them. 2) You appear to want to choose a specific approach to dimension reduction. If so, then that should be driven by theory which typically constrains the set of possible methodological options. Do you think you know enough to make that decision? If not, then an empirical and exploratory approach would widen the methodological aperture and 'constrained factor models' would be just another possibility along with PCA, Lasso and Relaimpo. $\endgroup$ – Mike Hunter Jan 24 '18 at 12:08
  • 1
    $\begingroup$ 'Well-established theory' exists...within disciplines. Wrt econometrics, Wooldridge's book offers the 'canonical' theory about panel data models but he doesn't address issues related to dimension reduction. It is in this sense that 'theory' is circumscribing since you are reduced to tinkering with, advancing or contributing within an extant body of literature. Consider 2) an opportunity to learn a whole lot about differing methodologies and approaches to analysis and modeling. Trust me, as your experience and bookshelf grows, this learning will pay future dividends. $\endgroup$ – Mike Hunter Jan 24 '18 at 21:49
2
$\begingroup$

Without seeing data this is difficult to answer but because the data is quarterly, it appears you have time-series data with categorical dimensions.

If you're trying to build a regression and use the outputs for forecasting, building a multivariate model with only 30 observations is going to be difficult, and your model-fit will reflect that (either adjusted R2 or F-Stat).

If you have to brute force this thing, I'd try a few of these steps:

  1. Run a correlation of all metrics to one-another to determine multicollinearity, this way you can see what metrics move together and can execute a form of 'manual' dimension reduction, as some metrics can be a proxy for another
  2. Select no more than three x-variables and run a single multivariate model for one of the companies,

OR

  1. Pool your data and run a mixed-model (using company name as a categorical variable), which will greatly increase the amount of observations you have and therefore strengthen the model and it's predictive capacity

OR

  1. Run an ARIMA on each companies quarterly data, see what you get
Improve this answer
$\endgroup$
16
  • 1
    $\begingroup$ If you want to shorten the list of external regressors, combine all timeseries data in one pool, add all 50 external regressors, and run a random forest on these set. Stationarize the timeseries prior to using the RF. When done, study the importance of variables and pick top most important. $\endgroup$ – Alexey Burnakov Jan 22 '18 at 8:57
  • 1
    $\begingroup$ @SpineFeast, just bind all timseries in one series. It might me that for original timseries the regressors should be different, but you don't have enough data to test that. Yes, make all inputs stationary: timeseries lags, seasonal dummy variables, external regressors. $\endgroup$ – Alexey Burnakov Jan 22 '18 at 9:49
  • 1
    $\begingroup$ @SpineFeast, pooling increase data volume, thus decreasing variance of statistic. Not more than that, and sample specifics will vanish. However, as already mentioned by other member, you can include the timeseries id as a dummy binary variable in order to allow the model to fit the sample specifics, while having enough data. Several pools might work as well. $\endgroup$ – Alexey Burnakov Jan 22 '18 at 10:07
  • 1
    $\begingroup$ @SpineFeast, try all you can. I liked the "Pool your data and run a mixed-model (using company name as a categorical variable), which will greatly increase the amount of observations you have and therefore strengthen the model and it's predictive capacity" advice, I would do that ( I did that). $\endgroup$ – Alexey Burnakov Jan 22 '18 at 11:06
  • 1
    $\begingroup$ @SpineFeast, I would use a desing where COMPAY would interact with SOURCE, like company_1_source_a == 1. So multiply company number by source number and take minus 1 to get the number of dummy vars. Please note that if you build a model that does not include interactions (such as a multivariate linear model without interactions) you most likely wont infere into the each timeseries specific. On the contrary, RF (or a decision tree for simplicity) can handle this, but it's not guaranteed as well. $\endgroup$ – Alexey Burnakov Jan 22 '18 at 12:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.