Are short time series worth modelling?

Question

Here is some context. I am interested in determining how two environmental variables (temperature, nutrient levels) impact the mean value of a response variable over a 11 year period. Within each year, there is data from over 100k locations.

The goal is to determine whether, over the 11 year period, the mean value of the response variables has responded to changes in environmental variables (e.g. warmer temperature + more nutrients would = greater response).

Unfortunately, since the response is the mean value (without looking at the mean, just regular inter-annual variation will swamp the signal), the regression will be 11 data points (1 mean value per year), with 2 explanatory variables. To me even a linear positive regression will be hard to consider as meaningful given that the dataset is so small (does not even meet the nominal 40 points/variable unless the relationship is super strong).

Am I right to make this assumption? Can anyone offer any other thoughts/perspectives that I may be missing?

PS: Some caveats: There is no way to get more data without waiting additional years. So the data that is available is what we really have to work with.

Answer 1

The small number of data points limits what kinds of models you may fit on your data. However it does not necessarily mean that it would make no sense to start modelling. With few data you will only be able to detect associations if the effects are strong and the scatter is weak.

It's an other question what kind of model suits your data. You used the word 'regression' in the title. The model should to some extent reflect what you know about the phenomenon. This seems to be an ecological setting, so the previous year may be influential as well.

Answer 2

Modeling the data fundamentally (especially for time series) assumes that you have collected data at a sufficient enough frequency to capture the phenomena of interest. Simplest example is for a sine wave - if you are collecting data at a frequency of n*pi where n is an integer then you will not see anything but zeros and miss the sinusoidal pattern altogether. There are articles on sampling theory which discuss how often should data be collected.

Answer 3

I've seen ecological datasets with fewer than 11 points, so I would say if you are very careful, you can draw some limited conclusions with your limited data.

You could also do a power analysis to determine how small an effect you could detect, given the parameters of your experimental design.

You also might not need to throw out the extra variation per year if you do some careful analysis

Answer 4

I am not sure I understand this bit: "Unfortunately, since the response is the mean value (without looking at the mean, just regular inter-annual variation will swamp the signal)"

With careful modelling, it seems to me you could gain a lot by modelling this as panel data. Depending on the spatial scope of your data, there may be large differences in the temperatures that your data points were exposed to within any given year. Averaging all these variations seems costly.

Answer 5

I would say that the validity of the test has less to do with the number of data points and more to do with the validity of the assumption that you have the correct model.

For example, the regression analysis that is used to generate a standard curve may be based on only 3 standards (low, med, and high) but the result is highly valid since there is strong evidence that the response is linear between the points.

On the other hand, even a regression with 1000s of data points will be flawed if the wrong model is applied to the data.

In the first case any variation between the model predictions and the actual data is due to random error. In the second case some of the variation between the model predictions and the actual data is due to bias from choosing the wrong model.

Answer 6

The required number of observations to Identify a model depends on the ratio of signal to noise in the data and the form of the model. If I am given the numbers ,1,2,3,4,5 , I will predict 6,7,8,.... Box-Jenkins model identification is an approach to determine the underlying General Term much like the test for "numerical intelligence" that we give to children. If the signal is strong then we need fewer observations and vice-versa. If the observed frequency suggests a possible "seasonal structure" then we need repetitions of this phenomena e.g. at least 3 seasons ( preferably more ) as a rule of thumb to extract (identify this from the basic descriptive statistics (the acf/pacf ).

Answer 7

Maybe you can try to handle your time series as a linear equation system and solve it by Gauss elimination. Of course in that case you constraint yourself to the available data but this is the only price you have to pay.