This question is about interpreting the results from non-linear regression models, especially when using regression splines. The numerical output is not very informative when interpreting the effects, so we have to rely on diagrams.
I did a generalized additive regression model with a binary dependent variable, logit link, and some independent variables. The dependent variable is current risk drinking at adult age, and the independent variable of interest for this question is the age when the subjects were drunk for the first time. This is based on cross-sectional data with retrospective report of age of alcohol use initiation, and that of course entails lots of problems, but that's not the issue here. My question regards the interpretation of the following diagram:
What I want to know is if my interpretation is correct. First of all, we can see that the confidence intervals are narrow at the ages 13-18 or so, and very wide at early and late ages.
- Is it valid to conclude that the wide confidence intervals in the lower and higher ranges is in part caused by that there are few observations in those age ranges? Would the curve look different in this respect if the number of observations were more evenly distributed?
Edit: oops, couldn't help myself, I had to try it out:
The model that this diagram is based on contains the same number of observations, but weighted so that each age group (age is recorded as an integer with the range 6-30) contributes equally.
This answers my own question, the confidence intervals are in general much more narrow here, but not in the age range 13-18, where there is less data in this diagram. I'll keep it as it provides a good extension to my next question below:
- When considering the first graph, what conclusions can we draw (pretending that there is no issues with recall bias and selection bias etc)? I would say that this diagram shows that early age of initiation is associated with greater chance of risk drinking in adulthood, and that the risk is greatest at age 12, while below that, the confidence intervals are too wide to draw any firm conclusions. The confidence intervals are so narrow in the age range 13-18, so it's safe to say that delay of initiation of alcohol use at least to the age of 18 seems protective against risk drinking in adulthood (again, pretending there are no problems with bias etc). Above the age of 18, the trend seems to point upward, but the confidence intervals are much to wide to draw any conclusions.
Is this an acceptable interpretation of the diagram? When using smoothers to illustrate non-linear effects, there are no numbers we can report that are meaningful so I'm trying to understand how to report these kinds of results to a reader.
So the extension of this question based the second diagram above. If that diagram was the result of my study, would it be acceptable to say that based on the confidence intervals, it seems like there are three spikes were age of initiation of alcohol use seems to confer an increased risk of later risk drinking behavior: ages 10-13, ages 21-22, and ages 28 and above. This is not realistic of course, but with this diagram and the interpretation above, we could then go on to speculate about why these age ranges seem so strongly associated with risk drinking. To me, this diagram seems to say that the evidence of these peaks and valleys is very strong based on the confidence intervals.
It is correct to rely heavily on the confidence intervals rather than p-values when interpreting these kinds of non-linear effects?