Forecasting with mixed models I need to forecast sales for a data set where I have the amount sold per item and week. There are also categorical variables that are supposed to be integrated into the model as random effects. What approaches could I use for forecasting in this scenario?
I do not have information on where the products are in their product life cycle or any information beyond the two years of data in the data set (meaning I have 104 measurements per item).
More on the data: One variable is the number of items sold in the respective week. The other variables do not change over time: price for the item and many categorical variables describing the context of the item like product group of the item, marketing scheme for the item etc.
 A: I understand that you have weekly product sales over 2 years with up to 30 categorical grouping variables, and you want to predict future sales using a mixed effects model.
You can try to use a mixed effects model for prediction/forecasting, although I personally would advise caution.
You would include the weekly sales as the outcome/response, and any independent variables, other than those you want to model as random effects, as predictors (fixed effects). It is unclear from the question how many fixed effects you have - only price has been mentioned. Presumably some of the categorical variables are also to be fixed effects. Note that it very rarely makes sense to include a categorical variable as both fixed and random, except when fitting random slopes - here I am talking about random intercepts.
As for the random structure, you would include a random intercept for the product ID. You mention that some of the other categorical variables are nested. So that means that some of them are crossed. For example you might have a store ID. Sales in one store are likely to be similar to sales in the same store than to sales in other stores. If all stores sell the same products then these two factors are crossed. On the other hand, a particular product might "belong" to a particular product category. In this case product ID and product category are nested. Provided that the variables are all coded uniquely, then in most software, such as lme4 in R, this will be automatically handled.
You also don't mention how time would enter the model. If there is some overall trend, (linear or nonlinear) this can be accounted for by including time as a fixed effect, and possibly a random effect too (with nonlinear terms or splines for nonlinearity).
Having fitted the model, you will then be able to use it predict sales for new data, using whatever predict() function is available in your software of choice.
However I foresee a few difficulties. First, you mention around 30 categorical variables but you don't say how many are to be used for random effects (intercepts) With that many potential grouping variables, you may find that the model is too complicated to be estimated, either due to numerical/computation problems, or due to the computational burden (ie time taken) of fitting such a model, and this will also be exacerbated by the sample size. If the number of products is insufficient then the random effects may not be identifiable, but if it is too large, it may not be feasible to run it at all. Including time in the model will only worsen this problem. Of course you might be lucky and it does converge, within a reasonable time. But if not probably your only option will be to simplify the random structure by first removing random slopes (if any), and then by removing random intercepts for the variables at the top of the nesting structure, and fitting them as fixed effects instead, and repeating this by working down the hierarchy (or hierarchies, as I assume there will be several).
Second, I would expect that this dataset would exhibit autocorellation, have seasonal effects and may not have constant variance over time (heteroscedasticity). These are problems that are typically solved by using time series methods, eg ARIMA and ARCH models. Some mixed model software can handle some of these issues, such as autocorrelation in nlme however nlme cannot easily handle crossed random effects - and you would also need to specify the nesting structure explicitly. Seasonality could be addressed by using dummy variables (again adding to the model complexity). Heteroscedasticity may not be a big problem since you are interested in prediction rather than inference.
Good luck !!
