@Ronald 's answer is the best and it's widely applicable to many similar problems (for example, is there a statistically significant difference between men and women in the relationship between weight and age?). However, I'll add another solution which, while not as quantitative (it doesn't provide a p-value), gives a nice graphical display of the difference.
EDIT: according to this question, it looks like predict.lm, the function used by ggplot2 to compute the confidence intervals, doesn't compute simultaneous confidence bands around the regression curve, but only pointwise confidence bands. These last bands are not the right ones to assess if two fitted linear models are statistically different, or said in another way, whether they could be compatible with the same true model or not. Thus, they are not the right curves to answer your question. Since apparently there's no R builtin to get simultaneous confidence bands (strange!), I wrote my own function. Here it is:
simultaneous_CBs <- function(linear_model, newdata, level = 0.95){
# Working-Hotelling 1 – α confidence bands for the model linear_model
# at points newdata with α = 1 - level
# summary of regression model
lm_summary <- summary(linear_model)
# degrees of freedom
p <- lm_summary$df[1]
# residual degrees of freedom
nmp <-lm_summary$df[2]
# F-distribution
Fvalue <- qf(level,p,nmp)
# multiplier
W <- sqrt(p*Fvalue)
# confidence intervals for the mean response at the new points
CI <- predict(linear_model, newdata, se.fit = TRUE, interval = "confidence",
level = level)
# mean value at new points
Y_h <- CI$fit[,1]
# Working-Hotelling 1 – α confidence bands
LB <- Y_h - W*CI$se.fit
UB <- Y_h + W*CI$se.fit
sim_CB <- data.frame(LowerBound = LB, Mean = Y_h, UpperBound = UB)
}
library(dplyr)
# sample datasets
setosa <- iris %>% filter(Species == "setosa") %>% select(Sepal.Length, Sepal.Width, Species)
virginica <- iris %>% filter(Species == "virginica") %>% select(Sepal.Length, Sepal.Width, Species)
# compute simultaneous confidence bands
# 1. compute linear models
Model <- as.formula(Sepal.Width ~ poly(Sepal.Length,2))
fit1 <- lm(Model, data = setosa)
fit2 <- lm(Model, data = virginica)
# 2. compute new prediction points
npoints <- 100
newdata1 <- with(setosa, data.frame(Sepal.Length =
seq(min(Sepal.Length), max(Sepal.Length), len = npoints )))
newdata2 <- with(virginica, data.frame(Sepal.Length =
seq(min(Sepal.Length), max(Sepal.Length), len = npoints)))
# 3. simultaneous confidence bands
mylevel = 0.95
cc1 <- simultaneous_CBs(fit1, newdata1, level = mylevel)
cc1 <- cc1 %>% mutate(Species = "setosa", Sepal.Length = newdata1$Sepal.Length)
cc2 <- simultaneous_CBs(fit2, newdata2, level = mylevel)
cc2 <- cc2 %>% mutate(Species = "virginica", Sepal.Length = newdata2$Sepal.Length)
# combine datasets
mydata <- rbind(setosa, virginica)
mycc <- rbind(cc1, cc2)
mycc <- mycc %>% rename(Sepal.Width = Mean)
# plot both simultaneous confidence bands and pointwise confidence
# bands, to show the difference
library(ggplot2)
# prepare a plot using dataframe mydata, mapping sepal Length to x,
# sepal width to y, and grouping the data by species
p <- ggplot(data = mydata, aes(x = Sepal.Length, y = Sepal.Width, color = Species)) +
# add data points
geom_point() +
# add quadratic regression with orthogonal polynomials and 95% pointwise
# confidence intervals
geom_smooth(method ="lm", formula = y ~ poly(x,2)) +
# add 95% simultaneous confidence bands
geom_ribbon(data = mycc, aes(x = Sepal.Length, color = NULL, fill = Species, ymin = LowerBound, ymax = UpperBound),alpha = 0.5)
print(p)
The inner bands are those computed by default by geom_smooth: these are pointwise 95% confidence bands around the regression curves. The outer, semitransparent bands (thanks for the graphics tip, @Roland ) are instead the simultaneous 95% confidence bands. As you can see, they're larger than the pointwise bands, as expected. The fact that the simultaneous confidence bands from the two curves don't overlap can be taken as an indication of the fact that the difference between the two models is statistically significant.
Of course, for a hypothesis test with a valid p-value, @Roland approach must be followed, but this graphical approach can be viewed as exploratory data analysis. Also, the plot can give us some additional ideas. It's clear that the models for the two data set are statistically different. But it also looks like two degree 1 models would fit the data nearly as well as the two quadratic models. We can easily test this hypothesis:
fit_deg1 <- lm(data = mydata, Sepal.Width ~ Species*poly(Sepal.Length,1))
fit_deg2 <- lm(data = mydata, Sepal.Width ~ Species*poly(Sepal.Length,2))
anova(fit_deg1, fit_deg2)
# Analysis of Variance Table
# Model 1: Sepal.Width ~ Species * poly(Sepal.Length, 1)
# Model 2: Sepal.Width ~ Species * poly(Sepal.Length, 2)
# Res.Df RSS Df Sum of Sq F Pr(>F)
# 1 96 7.1895
# 2 94 7.1143 2 0.075221 0.4969 0.61
The difference between the degree 1 model and the degree 2 model is not significant, thus we may as well use two linear regressions for each data set.
The models are significantly different even though they overlap. Am I right to assume so ?
