# How to compare loess models from related datasets?

I'm comparing matched data from a series of patients, looking at the effect of mental stress on cardiac conduction. I'm using the loess function to visually describe my data, and have 2 curves: one for with and one for without mental stress. I'd like to compare these curves.

Is it possible to compare two loess models from different (thought related) datasets. Alternatively, could I code +/- mental stress with 1/0 and use that to get a statistical relationship between the two models?

Any pointers would be super-appreciated.

• Comparing curve just to get a nice p-value for publication is a can of worms with so many wormholes that you better repost this on stackexchange - this has nothing to do with R. And if you do, tell them what your horizontal curve axis means. Because of serial correlation, it is absolutely important to know if this is within subject or between subjects. – Dieter Menne Jun 27 '12 at 11:54
• Thanks Dieter. The horizontal curve is within subject. Thanks for your pointers! – Malcolm Jun 27 '12 at 12:24
• Could you give us a bit more detail about what's on the y axis (I'm assuming that the x axis is time)? And for each patient do you have a series of measurements with and without mental stress, or do some have it and some not? – Andrew Jul 10 '12 at 16:44
• Is your question related to this Is it possible to fit a data curve to another data curve? (Deleting the answer added, that should have been a comment.) – Noble P. Abraham Sep 7 '12 at 15:32
• Not loess but gam, which is similar. Calculate the CI of the difference. fromthebottomoftheheap.net/2017/10/10/difference-splines-i – Lennard Jun 7 '18 at 12:30

## 4 Answers

Unfortunately loess fits are not comparable. A loess (or lowess) curve is not like one based on a linear or quadratic or cubic equation. I know of no statistial package that will provide an equation that defines a loess curve, nor a fit statistic such as R-squared for it. Such a curve is completely opportunistic and therefore unique to each dataset. A loess fit is atheoretical; one would not want to use it to try to formally replicate a pattern across datasets. You might say it's "for exploratory purposes only," and even there one must use care, as you can see if you read through threads such as this one.

Is the variance between the data and the curve bigger than the variance between the two curves, averaged over patients? If you want a p-value I suppose you could assume normality of errors and use the F-test on ratio of variances. The loess fit isn't a formal fit, so this is not a formal test. Could you use a formal fit? Have a look at the graphs. Do they look the same across patients?

predict.loess in R has an se parameter which if TRUE returns standard errors for all predicted points. These are the usual standard errors of residuals. One can, as well, simply take the absolute value of difference between predicted values and originals and so get a sequence of absolute errors. These are often more robust than standard errors.

In any case, with these in hand you are set up for doing cross validation or jackknifing to get some notion of which of a set of parameters describe a dataset better.

Note that in time series applications, these are dependent data so cross validating or jackknifing points is not appropriate: You need to pick random subsequences of the original parent in some principled way and cross validate over those. The lengths or windows of these should tie to some phenomenon horizon in the original dataset or problem. Failing that, could look at using the stationary bootstrap of Politis and Romano as implemented in the tsbootstrap function of the tseries package. That gives a bit more flexibility for dependent sequences, but the analyst still needs to specify a mean block length.

In general when comparing two different fitting curves the way to go is a performance measure. It does not allow you to draw strong conclusions, but if you have two options you can calculate the R squared for example and at least have an idea of how good your fit can be.