3
$\begingroup$

I believe that this is a statistics rather than a programming question, though I am tied to an R implementation and hope for a reply in kind.

I have data that constitutes several time series. I expect to be studying this data for some time to come, and would like to apply a wide range of time series methods, including ARIMA models, vector autoregression, fractionally integrated series, and others.

My data is of daily frequency and has, in essence, four observations a day: two at fixed times at the beginning and end of the day, respectively, and a maximum and minimum value between those points, with no associated time for each. Although it does not involve any additional measurements, the interval between the end-of-day and the beginning-of-day is also meaningful, with the change morning to night and the change night to morning measured by values of the same two series, offset by one.

In implementing this as an R time series that various tools can use, I face two problems, both with the extremal values. The first is that these two points are only partially ordered in time. They fall in the daytime interval between the beginning and end of the day, but there is no information in the data about which occurs first.

The second, related problem is in representing the frequency. The start and end of the day happen at specific identifiable moments in time, always the same. I have not been able to find any time series representation of the occurrence of the maximum and minimum that R time series routines will accept, and that accurately reflect the temporal process as I understand it. This is a different problem than the first, because that problem deals with representing data points without knowing the order in which they occur, while this one deals with combining data that occur at fixed times with data where the time of occurrence is known only down to an interval.

Am I just out of luck here? Do I have to reinvent time series from the bottom up before I can use any of the standard tools? Or switch to some non-parametric time series representation about which I know nearly nothing? Or is there a common work-around, or an alternative time series implementation that can handle problems of this sort and speak to other R time series routines in a way they will understand?

$\endgroup$
2
$\begingroup$

This kind of issue occurs for many temperature and other weather data sets. For example, most temperature data sets contain temperatures at fixed points in each day, but also contain the daily maximum and minimum temperature, without specification of the time that these occurred. For these kinds of data sets, it is best to treat the data as parallel time-series, with the time variable specified only down to the day, but no further. This will essentially give you four parallel time-series, with each time-series having one value each day, and the four series representing the start-of-day values, end-of-day values, minimum daily values, and maximum daily values. These four series can be treated as separate objects, or you can put them in a single data frame with a factor variable describing the type of measurement.

To code your data in this structure in R you would use an integer time-index and a factor variable describing the type of measurement. I will show an example using temperature data. Denoting the time-index and measurement variable by Time and Measure would give you a data frame with output and structure like this:

DATA

           Time          Temp       Measure
1             1          12.4         Start
2             1          21.3           End
3             1          10.7           Min
4             1          25.6           Max
5             2          10.2         Start
6             2          20.4           End
7             2           8.0           Min
8             2          25.2           Max
9             3          11.1         Start
10            3          19.9           End
11            3          10.1           Min
12            3          24.7           Max
13            4          12.3         Start
14            4          20.8           End
...

str(DATA)

'data.frame':   23,376 obs. of  3 variables:
 $ Time   : int  1 1 1 1 2 2 2 2 3 3 ...
 $ Temp   : num  12.4 21.3 10.7 25.6 10.2 20.4 ...
 $ Measure: Factor w/ 4 levels "Start", "End",..: 1 2 3 4 1 2 3 4 1 2  ...

If you would like to isolate a particular time-series of values, this can be done with a simple subset query:

dplyr::filter(DATA, Measure == 'Start')

           Time          Temp       Measure
1             1          12.4         Start
2             2          10.2         Start
3             3          11.1         Start
4             4          12.3         Start
5             5          11.7         Start
6             6          11.9         Start
...

For modelling this type of data, it is usually preferable to model a single time-series, using a single measure. It is possible to model the four time-series together, using the Measure factor variable as a variable in your model (with accompanying interaction terms where needed), but this is not usually necessary.

$\endgroup$
  • $\begingroup$ Sorry to be so slow in responding. There are still some problems with my implementation of this idea as applied to my data, but this answer definitely pointed me in a productive direction. $\endgroup$ – andrewH Jan 25 at 6:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.