2
$\begingroup$

I have a dataset that produces a line as shown in the picture below. I would like to statistically test when the line is moving upwards, downwards or sideways (not moving significantly). What would be best practice in order to do so?

I'm thankful for any help!

enter image description here

Improve this question
$\endgroup$
5
  • $\begingroup$ Do you mean testing at each point to break it into segments of “basically constant”, “increasing”, and “decreasing”? $\endgroup$
    – Dave
    Commented Feb 15, 2020 at 15:37
  • 2
    $\begingroup$ Beware: those three possibilities are not exhaustive unless you expand the notion of "sideways" in ways people wouldn't tend to expect. $\endgroup$
    – Glen_b
    Commented Feb 15, 2020 at 23:41
  • 1
    $\begingroup$ Do you really need a statistical test? Why is a difference or growth rate transformation not enough to answer your question? 🤔 $\endgroup$
    – Graeme Walsh
    Commented Feb 16, 2020 at 13:59
  • $\begingroup$ The reason I would like to test it statistically is because I want to automate the process. What I mean by sideways is the point to point changes being relatively flat (not changing significantly) as could be seen in the graph between the period ~ 50 - 120. Furthermore, I will use different datasets so then I would have to reanalyze what to be considered a significant increase/decrease for each set of data! :) $\endgroup$
    – Vichtor
    Commented Feb 16, 2020 at 14:29
  • 1
    $\begingroup$ Best practice would probably be to NOT do this, at least, not until you have made your groups a) More precisely defined and b) Exhaustive. $\endgroup$
    – Peter Flom
    Commented Feb 16, 2020 at 15:13

3 Answers 3

Reset to default
3
$\begingroup$

What do you exactly mean by upwards, downwards or sideways? You could fit a linear regression model on the data. You will get a coefficient for the slope and a confidence interval. If the coefficient is not significant then you could say that there is no trend.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Sorry for being unclear. What I meant was analyzing each point and compare it to its previous value in order to determine if the increase or decrease is signifcant, else it would be "flat" and therefore move "sideways". I believe you provide a good solution that defiently would work if I was to get the trend for the whole graph, which in this case would shows high correlation and strong trend. $\endgroup$
    – Vichtor
    Commented Feb 15, 2020 at 17:29
2
$\begingroup$

Linear regression against time ASSUMES a model form and uncorrelated residuals to actually test the significance of estimated parameters. Your series might be adequately described with a local time trend (NOT GLOBAL) and a few level shifts and possible pulses and a possible memory component (arima) but only your data knows for sure. Post your actual data and I will try and help further .

How to make this data stationary might help you better understand how data like this gets objectively studied.

Improve this answer
$\endgroup$
1
$\begingroup$

If you want to know the direction point to point, I believe that you can evaluate easily the slope: $\frac{Y_{t+1}-Y_{t}}{x_{t+1}-x_t}$. If this result is positive, it is going upwards. On the other hand, if it is negative it is going downward. If it is zero, it is going sideway.

If your question is about to know the average direction, you should estimate a non parametric function to learn the average direction. Andrew Lo has done this in the past to play with finance. See the paper.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ Thank you! I will defiently spend time and read the paper! The problem I find using the slope approach point to point is that minor increases/decreases also return positive/negative slopes. Also, when comparing different sets of data, I would have to reevaluate what would be considered a "noticable" change. $\endgroup$
    – Vichtor
    Commented Feb 15, 2020 at 17:19

Not the answer you're looking for? Browse other questions tagged or ask your own question.