# In hierarchical regression, the first step is not significant but later steps are

I ran a hierarchical regression comparing different models, in order of complexity. The first step was not significant (the simplest model vs a more complex model). However the next two steps were significant (a more complex model vs an even more complex model). Since the first step was not significant, I think I should stop here and select the simplest model (Model 1).

Below are the results from the hierarchical regression for model comparison

   Res.Df   RSS        Df   Sum of Sq     F      Pr(>F)

1  45768    311819838

2  45766    311802160  2    17678         1.2981 0.2730717   (n.s.)

3  45764    311696461  2    105699        7.7612 0.0004265 ***p<0.01

4  45762    311613774  2    82687         6.0715 0.0023096 ***p<0.001


I also looked at the AIC and BIC values. The AIC follows the pattern for the p-values: Model 2 is not better than Model 1, but Models 3 and 4 are better (the change in AIC for each step is 1.4, -11.5, -8.1). The BIC suggests that Model 1 is the best (the change in BIC for each step is 18.9, 5.9, 9.3). According to the BIC, I also think I will select Model 1.

Can someone please explain to me why the more complex models show up as significantly better when the earlier model was not?

For reference, the models being tested are segmented regression models as follows

• Model 1: A linear model (one slope); task_score ~ age
• Model 2: A model with one breakpoint in a predictor variable (one psi estimate to mark the breakpoint) task_score~ age + psi1.age
• Model 3: A model with two breakpoints (two psi estimates) task_score ~ age + psi1.age + psi2.age
• Model 4: A model with three breakpoints (three psi estimates) task_score~ age + psi1.age + psi2.age + psi3.age

The hierarchical regression measured

• Step 1: Model 1 vs Model 2
• Step 2: Model 2 vs Model 3
• Step 3: Model 3 vs Model 4

Note on segmented regression: A segmented regression (also called piecewise or changepoint regression) is a linear regression with an abrupt change in the x~y relationship, i.e., where the line is allowed to 'bend' at a given point (called a breakpoint). Examples of 1,2, and 3 breakpoint models are below.

• How do you define the breakpoints and why are they hierarchical? Commented Jun 24, 2020 at 15:57
• A breakpoint is a point in which the x~y relationship is allowed to change. You can think of it as a 'shift' or 'bend' in the regression line, or multiple lines joined together by breakpoints. I have added a definition and some images to my original post. Commented Jun 24, 2020 at 18:35
• They are hierarchical because each breakpoint creates an additional bend in the line, and requires an additional slope to be estimated. Commented Jun 24, 2020 at 18:48