# How to determine whether a data set is continuously growing?

I sampled some data at certain sampling rate (e.g., 1 sample per minute). I want to know whether the values in this data set are continuously growing or they are just fluctuating around certain value. How can I do that?

In addition, I would like to predict the future values to be sampled. Based on the prediction (whether it will grow along with certain confidence level), I can decide whether to continue the experiment to sample more data or simply stop the experiment.

Update: I think this is exactly what I need: Trend Esitmation

Note: I know very little (if not nothing) about statistics, so bear with me and do correct me if you find me using wrong terminology.

-

One way to do this would be to regress on time. What software are you using?

This is a very simple approach that doesn't allow for effects other than a continuous effect of time. In R you could do something like this.

time <- 1:10
values <- c(2,4,5,7,5,6,8,10,11,10)
m1 <- lm(values~time)
summary(m1)


but you will probably have more values (and times) and you may be reading from some other file, which would change things.

summary(m1) yields

Call: lm(formula = values ~ time)

This just says what m1 is.

Residuals: Min 1Q Median 3Q Max -1.3455 -0.8455 0.1091 0.8136 1.5636

This gives some info about the residuals. You can get more with plot(m1). Ordinary least square regression assumes some things, including that the residuals are normally distributed with constant variance.

Coefficients:

Estimate Std. Error t value Pr(>|t|)

(Intercept) 1.8000 0.7412 2.428 0.0413 *

time 0.9091 0.1195 7.610 6.25e-05 *

this tells you the model that is estimated by m1: value = 1.8 + .91*time. It also give the standard errors of the coefficients (0.74 and 0.12), their t-values and their statistical significance.

Residual standard error: 1.085 on 8 degrees of freedom Multiple R-squared: 0.8786, Adjusted R-squared: 0.8635 F-statistic: 57.92 on 1 and 8 DF, p-value: 6.245e-05

If you want to do more complex things, let us know more about your problem.

-
I'm using R as well, but I'm open to any software. Problem is I don't understand at all the output of summary(m1). Can you explain that to me, or you can point me to some references? –  dacongy Aug 17 '12 at 16:59
Well, the easiest way to get some help is ?lm while in R. But that's not necessarily the easiest help to read. :-). I will edit my answer. –  Peter Flom Aug 17 '12 at 17:05
I would also like to suggest using plot(time, values, type="b") to visualize the temporal trend of the outcome. –  Penguin_Knight Aug 17 '12 at 19:05
There is only one parameter associated with time in the model I showed. You don't need a parameter to know if the data fluctuates - it does. I think what you want is to determine if the trend is "big" compared to the fluctuation. –  Peter Flom Aug 21 '12 at 20:53
@dacongy Correlation will tell you how consistently the values change over time. The slope/coefficient/"parameter" for time tells you how much they change per unit of time. The t-statistic and p-value associated with that slope tells you something about how statistically significant it is--how rare such a slope would be if it were the result of chance alone (and if the slope in the larger population were zero). –  rolando2 Aug 22 '12 at 3:10