# Large dataset of many short time series: what model to use for forecasting a new time series not in the data?

## Problem statement

Consider this hypothetical but hopefully practical example:

• You have a dataset consisting of home electricity usage for 1,000 homes in a city.
• For each home, you have a time series measured hourly.
• Each time series has a different start/stop date and time. Some are as short as 12 hours, some are as long as 336 hours (a full 2 weeks).
• You are now given a time series for a new that isn't already in the data, and you need to forecast the time series out by several hours. This time series might be as short as 12 hours.
• Assume, for simplicity, that the data is "clean" -- no missing data and no obvious outliers.

Hopefully you can see that this problem is analogous to many problems that appear in social and natural science.

What kind of model do you build to accomplish this task? I would expect that there is a lot of "common information" shared across all of the time series that can be used to make better forecasts when given a new time series, especially with respect to cyclical trends throughout the day and week.

My experience with time series is limited to 2 undergrad econometrics courses, the contents of FPP2, and a handful of ad-hoc problem solving I've done at work, which has had mixed results.

## The question

Is there a recommended best technique for this problem?

I can think of a few possible approaches:

1. Use some kind of time series decomposition method to learn seasonal/cyclical components from the dataset. Then see if there is a linear trend in the new time series, then apply seasonal component from the training data on top of the linear trend from the new data.
2. A regression model trained on hour-of-day and day-of-week converted to polar + any other relevant exogenous features that might be available - this could be linear regression, XGBoost, whatever. Facebook's "Prophet" forecasting model seems to be a sophisticated variation on this idea.
3. Some kind of Bayesian hierarchical time series model to benefit from partial pooling across homes.
4. Something else entirely.

## Related questions on Stackexchange

Note that there are several related questions on this site, but we do not appear to have any authoritative thread on this topic. So this question is not only for my benefit, but hopefully for the benefit of the community.