# Transform time-dependent data

I have pollution data (quantitative) plotted against time (categorical), the hours of the day. Via ANOVA testing I've found significance at many of the hours, however, the relationship is definitely not linear. For example, it tends to rise in the morning, lower in the afternoon, and rise again in the evening, as you can see in the picture: How can I transform this data set in order to build a linear model (in R) ?

I've chopped the data up into 6 groups of 4 hours to find average means of each group, but I'd like to be able to say, with a level of significance, that time X is the most polluted, or least polluted. How can I use R to make such a statement?

I've plotted the residuals of the model below -- since it's not normally distributed -- there is something that the linear model is not capturing. I've put my data in a time series object - but I don't know what it means:

myts = ts(df\$value, start=c(0), end=c(24), frequency=24)
plot(stl(log(myts), s.window="periodic")) I now have a model, but not a very good one. It has an adjusted r-squared of .18.

plot(value~(hour), data=df, ylim=c(0, 300))
my.lm = lm(value ~
sin(100*hour) +
sin(50*hour) +
cos(100*hour) +
cos(50*hour) +
day +
month,
data=df)
lines(predict(my.lm), col=3, lwd=10)


Any thoughts on what I could do better?  • As 24:00 = 00:00 you should be thinking about a model periodic in time of day. Using sines and cosines as predictors would be a linear model. If your original data are recorded more finely than hour of day, use the precise times. Blocking into hour groups would just degrade your data, as to a good first approximation pollution will vary continuously with time of day. If you have reason to suppose that emissions from some source start and/or stop abruptly, that might need to be built in. Analysis of variance treating hours as categorical predictors is a poor model here and quite unphysical. Nov 20 '14 at 11:10
• +1 to @NickCox's comment. An alternative would be periodic splines, e.g., B-splines. I personally don't like how sines and cosines fit global effects. Nov 20 '14 at 11:25
• Here, as so often, there is a mismatch between your graphics and your analysis. Box plots show medians; ANOVA uses means. It's entirely likely that they tell consistent stories here, but watch out. More importantly, it is far from obvious that means are the best summaries here or that data should be analysed as they come rather than by using a transformed scale or a non-identity link function. Nov 20 '14 at 15:32
• That's B-splines, on which @Stephan Kolassa may have some suggestions. Make sure you implement a periodic flavour, or in such a way that the fit is periodic in practice. On sines and cosines, www.stata-journal.com/sjpdf.html?articlenum=st0116 may help. The equivalent R code shouldn't be difficult (but is off-topic here). Nov 21 '14 at 7:40
• To repeat, ANOVA as you have applied it makes very little sense here as it is not focused on capturing the obvious and natural periodic structure in the data. (I'd reject any paper based on such analysis as fatally flawed.) Impossible negative values should be recoded to missings. I'd recommend a generalized linear model with some skewed probability distribution family and log link and periodic predictors. Nov 21 '14 at 7:44