Do you see trends in my residual plots? These residuals plot show the standardized residuals against fitted values, origin period, calendar period, and development period. The patterns in any direction can be result of trends. There seems to be a trend on my bottom left plot, do you agree?

1 This is a research project. I am asking your inputs to confirm my assumptions.

  • $\begingroup$ How much data do you have? Are these time-series? $\endgroup$ – gung - Reinstate Monica Jan 23 '15 at 13:50
  • $\begingroup$ I have 100 data points. Yes, they are time series. I appreciate your prompt reply. $\endgroup$ – Jenny Jan 23 '15 at 13:53
  • $\begingroup$ Have you already modeled the error structure (eg, AR & MA components), & is it stationary? $\endgroup$ – gung - Reinstate Monica Jan 23 '15 at 13:59
  • 2
    $\begingroup$ Additionally to @gung's comment please show a plot of your residuals versus the observed series. $\endgroup$ – IrishStat Jan 23 '15 at 14:10
  • $\begingroup$ @IrishStat there's a reason why people tend to avoid plotting residuals vs observed in regression-type models, though. It might be worth pointing out to the OP the caution needed in interpreting such plots. $\endgroup$ – Glen_b Jan 23 '15 at 15:39

I agree, at first glance it looks like a change in calendar year trend at 1992. (Indeed, you can't see much trend in the other two directions, because the chain ladder has parameters that broadly pick up the trend changes in those directions, so that's the one plot you tend to look for changing trends in).

Consider just the left half of the calendar period plot (up to 1992):

enter image description here

Ignore the red line if you can, and just consider where you think the data seem to be heading (keeping in mind to get the overall "gist" -- try not to let one point overly influence your impression. I find it helps to cover points and see if my impression changes in their absence; if it does, that points overly influencing my impression).

If the trend was similar after that, where would you think it should be for 1993-96?

Now look at the rest (noting the multiple almost-coincident points at the bottom right):

enter image description here

and (again, ignoring the red line) try to "backcast" the first half of the diagram.

If you're like me, you'd probably tend to draw those trends in somewhat different places, so it would make us concerned because the model simply isn't able to capture changes in that direction (though it's rather profligate with parameters in the other directions).

I think there are a variety of plausible ways to see the plot, but it does seem as if there's some kind of change there in the middle. Of course, there's not a lot of data, so we have to be fairly cautious about over-interpreting our impression there. We have a tendency to see more pattern than is really there.

[If you could leave out old payments with the chain ladder, it would be tempting to see if it made a difference to only look at the more recent years, but because it works with predicting cumulated data from previous cumulatives, you can't; the old payments are unavoidably there. You might look at patterns in one-year-ahead prediction errors, but they wouldn't be all that different from what you see here.]

It looks perhaps more like a level drop immediately after 1992, but the loess line tends to smooth over those kind of effects. (I should clarify -- the strength of the indication is not very strong ... but the consequences may be quite substantial.)

In short: there's at least some indication of a change in trend in the plot there, but it's not especially strong, and it's not completely clear that it's quite like the red line suggests.

[The Chain Ladder model has no mechanism for dealing with such a trend change, though they're common -- due to changes in economic inflation, in social trends or judicial trends, in effects peculiar to the specific line of business (such as superimposed inflation), or sometimes to effects at the company level.]

However, the fit to the most recent years of data isn't bad, so it might not be too problematic to use that forecast for the next year (but you'd have to worry about further trend changes - it's no unusual to see a short term trend change followed by a return to approximately the slope of the original trend, for example).


The data looks sort of familiar for some reason. Is that one of the built in data sets in the package, or from something else?

| cite | improve this answer | |
  • $\begingroup$ That would be where I've seen it -- I think I had a research student who used it for an example a few years ago. When you say 'project' ... what are we talking about? I don't think it would be suitable for me to review something for coursework. $\endgroup$ – Glen_b Jan 24 '15 at 1:46
  • $\begingroup$ If you have specific questions, of course they can be posted (though if it's something for coursework you should normally indicate that in the question - in that case broad guidance of the sort you asked for here is reasonable). $\endgroup$ – Glen_b Jan 24 '15 at 1:53

I think it is hard to judge a trend just from the picture. I suggest you do some white noise testing on your residuals in order to be certain that they are not random. For example, you could try using our R package hwwntest, which includes different types of white noise tests.

| cite | improve this answer | |
  • $\begingroup$ After briefly reviewing hwwntest am I right that your are assuming that the series being analyzed is free of Pulses, Level/Step shifts, Seasonal Pulses,LOcal Time Trends AND has constant variance ? If so then perhaps you could add a disclaimer so as the user would be forewarned. $\endgroup$ – IrishStat Jan 23 '15 at 22:09
  • $\begingroup$ Yes, the white noise hypothesis is such - an iid process with constant and finite variance. We are talking about residuals (which also may well be normalised/studentised) in this question. Thus, if the model is correct, the residuals should be just random noise, otherwise the model is missing something, right? $\endgroup$ – Delyan Savchev Jan 24 '15 at 23:21
  • $\begingroup$ correct if they are not white noise residuals the next step is to find out what exactly is causing the test to reject white noise and that is where intelligent software can be useful to sort whether there are deterministic components and/or ARIMA structure and/or non constancy in the parameters and/or non-constancy of the variance present in the current residuals. $\endgroup$ – IrishStat Jan 25 '15 at 0:01
  • $\begingroup$ I Agree. In the case of the forum question asked, I understand that the researcher is looking for a certain white noise departure for the residuals, be it a trend or something else. Thus, if you look closely at hwwntest or the paper therein, you could see that for one test - genwwn - it provides a theoretical statistical power function, which could be evaluated with respect to a specific alternative hypothesis from an ARMA class model. Of course, all those depend on the research problem and approach. $\endgroup$ – Delyan Savchev Jan 25 '15 at 12:19
  • $\begingroup$ Does it also suggest the presence of specific deterministic structure such as in the data under review ; 2 level shifts and 5 weekly dummies reflecting significant activity for 5 of the 7 days. $\endgroup$ – IrishStat Jan 25 '15 at 12:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.