# Suggest a model for this dataset

I have a time series data set (the Old Faithful geyser data available here: http://www.gatsby.ucl.ac.uk/teaching/courses/ml1-2012/geyser.txt). Plotting the eruption duration on the x axis and the waiting time between eruptions on the y axis, the data looks like this:

I'm also told to plot the x values against delayed x values (for different delay lengths n):

and to plot the x values against delayed y values (for different delay lengths n):

Based on these graphs, I need to recommend one of the following models:

(i) multivariate normal (ii) Gaussian mixture model (iii) Markov chain (iv) hidden Markov model (HMM) (v) observed stochastic linear dynamical system (vi) linear Gaussian state-space model

So, I want to argue for an HMM. I think the latent variable should be able to occupy one of four states corresponding to the different cluster means we see in the delay graphs - the reason being that this would allow us to explain the long term trend in the time series. I think if each latent state corresponded to a cluster mean then the emission probability would just be a Gaussian centred on that cluster mean as this could generate the observed data.

I have two questions:

(1) What do you think of my above explanation? Have I chosen the right model? Is my reasoning correct?

(2) How can I argue against the other models? Basically I need one reason to say why each of the other models are bad. My thoughts so far are that:

(i) a multivariate normal could explain the clusters in original dataset but wouldn't be able to explain the clusters in the delay data

(ii) same argument above to get rid of Gaussian mixture model

Is that right? How to argue against the other three?

• Look at your first plot. Roughly, values may be classified as Small and Large. Notice that a Small value is never followed by another Small value. That is Markovian behavoir. Aug 11 '19 at 0:20
• @BruceET Do you mean that a small y value (small waiting time) is never followed by a small x value (small eruption time)? Why is this Markovian? Each point on the graph has coordinates (x_t,y_t); wouldn't Markovian behaviour require the values of x_t & y_t to depend on (x_{t-1},y_{t-1})? But we can't see that from the graph since the points aren't indexed in any way.... Aug 13 '19 at 14:32
• Sorry. Misread your plot. But if you make a plot of waiting times vs last waiting times, what I said is true. (Similarly for length of current eruption vs length of last.) So it is possible to use immediate past behavior to predict next eruption. Aug 13 '19 at 17:17
• @BruceET ok, this would correspond to the n=1 graph in my second and third figures. We see that for eruption times, small eruptions are never followed by small eruptions since the bottom left quadrant is empty. Your argument is that this means current eruption does depend (in some latent way) on previous eruption time since it has "knowledge" of whether it was short or not? As for the waiting times, we have top left and bottom right quadrants empty which seems to suggest short (resp long) wait always followed by short (resp long) wait; this doesn't make sense since we would get stuck in cycle? Aug 13 '19 at 18:41

I have looked at data on durations of eruptions of Old Faithful geyser collected during the summers of 1978 and 1979. (There have been several earthquakes nearby since then, so the underground 'plumbing' of the geyser, and hence behavior of eruptions may be different now.)

Although 222 eruptions were observed, only 205 of them were adjacent pairs, and we focus on those. (Data were not collected at night.)

A histogram of durations is bi-modal. Somewhat arbitrarily, eruptions shorter than 3 min. in length are called Short (0) and others Long (1). We can model eruption lengths $$X_i$$ as a 2-state homogeneous Markov chain.

Based on counts, one can estimate: $$p_{01} = \alpha = P(X_{n} = 1\,|\, X_{n-1} = 0) = 1$$ and $$p_{10} = \beta = P(X_{n} = 0\,|\, X_{n-1} = 1) = 0.44.$$

The stationary distribution (hence the limiting distribution) of this ergodic chain can be shown to be $$\sigma = \left(\frac{\beta}{\alpha+\beta}, \frac{\alpha}{\alpha+\beta}\right) = (0.3056, 0.6944).$$

This is in good agreement with the 69.8% of long eruptions in the original data.

Notes: [a] I have heard theories (based on two underground reservoirs of hot water, with either one or both emptied in any eruption), but I have not seen them confirmed.

[b] I have seen regression analyses that attempt to predict the waiting time for the next eruption, based on the duration and waiting time for the preceding eruption. Predictions of waiting times between eruptions based on regression have been useful for park visitors, trying to view eruptions. However, to the extent that these regressions depend on independent sequences of events for the predictor variables, they may not be exactly correct---we have successfully modeled durations of eruptions as Markovian, not independent. Waiting times also seem not to be independent.

[c] The dataset I used is a classic one, with data collected by park rangers, reported in Weisberg, S. (1985), Applied linear regression, Wiley. Various websites have undertaken to report data on contemporary eruptions of Old Faithful geyser. I do not know what data you are using.

• Thanks for the detailed reply. It's definitely helping with my understanding of this problem. However, I am only allowed to use the graphs that I've plotted above to reach my conclusion. Your argument advocated for a first order Markov chain but I'm not sure how I can reach that conclusion given just those graphs. I'm working on an old assignment (Q2 here: gatsby.ucl.ac.uk/teaching/courses/ml1-2011/asst2.pdf) and the data is linked in the question. Any suggestions? Aug 15 '19 at 12:43

you have 272 values that are not taken at fixed time intervals THUS you data set is not a classic time series data set. I took your 272 values and plotted them ( as you did ) and obtained . If you segment your data based upon time between (x) .. set 1 where X <64 AND SET 2 WHERE X>64 there is no relationship. Globally there is a relationship but this is a classic case of a spurious relationship due to a concommitant effect and realized by pooling all the data.

I don't think , I was much of help other than identifying a few spurious data points i.e. outliers.

• Thanks for your reply. I must apologise for uploading the wrong link; I'm really sorry about that. The Kaggle link I posted was from a page I was reading to try and get some ideas. I have replaced the link with one to the actual dataset; as you can see there are in fact 295 values. I suppose since they are consecutive measurements, we can still observe that they are not taken at fixed time intervals but I'm not sure why this is significant? In any case, if you plot the new data you get graphs like those I posted above - any recommendations for a suitable model? Thanks & sorry again Aug 10 '19 at 19:56
• autobox.com/pdfs/regvsbox-old.pdf will help explain time series to you. To use lag structures they have to be taken at FIXED intervals . I will look at you corrected data set of 295 values. Aug 10 '19 at 20:46
• I would suggest closely examining the items where eruptions were less than 31 and try to identify a factor that might have been common AND NOT PRESENT when eruptions were >31 Aug 10 '19 at 21:02
• Eruption length is on x axis so do you mean 3.1 rather than 31? To find said factor, do we not first need to agree on which of those models we are going to use? Aug 11 '19 at 16:57