0
$\begingroup$

This may be a silly question but I am not really a math/statistics person and we are working on this assignment at work. So my job is to figure out what is the trend of the data from the last 30 days. So basically I get around 30 values, each is an average value from one day. And using this data I draw a linear graph. Now the difficult part for me, how do I know that mathematically that on average the trend is upward or downward? I can see it from the graph but how do I figure this out matematically?

enter image description here

Improve this question
$\endgroup$
5
  • $\begingroup$ To address your specific question: you regress your variable of interest against time and check if the slope is positive or negative. But that's frequently not the best way to analyse time series data, especially if you have unusual trends like those in the plot above. $\endgroup$
    – mkt
    Commented Aug 30, 2017 at 14:36
  • 5
    $\begingroup$ (1) The plot is extremely deceptive because it compresses most of the month in the left half and uses the right half to show just five days. (2) It doesn't look like a good idea to describe this behavior as a "trend" when there basically was no change for 29 days and then a sudden drop in the last day. Why not describe exactly what happened? $\endgroup$
    – whuber ♦
    Commented Aug 30, 2017 at 17:23
  • $\begingroup$ whuber's observations are spot on. The first thing I would want to know is whether the collapse of the series in the last day is a real value or an error in recording and, thus, should be dropped. If the former and assuming you have additional information or features about this data, then consider decomposing the series into two components: 1) an analyzable trend component for the first 29 days where a simple regression model or spearman correlation would help define its directionality and 2), a second model predicting the probability of collapse given the previous history. $\endgroup$
    – user78229
    Commented Aug 30, 2017 at 17:49
  • $\begingroup$ Wrt the second model and assuming the collapse is real, there could be something cyclical going on. In that case, having more data going back more a single month would not only be useful but necessary as a single month won't be the least bit informative. $\endgroup$
    – user78229
    Commented Aug 30, 2017 at 17:50
  • 2
    $\begingroup$ In agreement with others: the simple but crucial question that must be answered first is what happened at the end of the series. If there isn't independent evidence explaining that, then it's hard to know how to describe the data except by showing the graph and declaring a mystery. Fitting a complicated model to 30 values would be a real stretch here: either it quantifies the obvious, or it is dubious. $\endgroup$
    – Nick Cox
    Commented Aug 30, 2017 at 18:01

2 Answers 2

Reset to default
-1
$\begingroup$

Fit a smoother and obtain an estimate of the derivative. If this is negative over a certain range (30 days in your case), the trend is downward.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks a lot for quick response! Is there any nice and easy equation to do that? $\endgroup$
    – lip3k
    Commented Aug 30, 2017 at 14:38
-1
$\begingroup$

Look at Trend shifts in timeseries and pick up some pointers. Detecting trend is not accomplished by specifying a model or a derivative but by allowing the data to suggest the underlying model and then drawing inference. Auto-correlation , level shifts , error variance heterogeneity , outliers etc can easily thwart any simple strategy.

Improve this answer
$\endgroup$
7
  • 3
    $\begingroup$ The question asks about the trend in the data. Why do all that modeling work to comply with such a simple, straightforward request? $\endgroup$
    – whuber ♦
    Commented Aug 30, 2017 at 17:25
  • $\begingroup$ To quote a famous statstician , you may even know him ! "For every simple/straightforward question there are a thousand wrong answers" . Simple questions regarding trends often involve mean shifts to be questioned. In my opinion a "trend" has to be viewed base upon possible complicating/additional autoregressive structure. $\endgroup$
    – IrishStat
    Commented Aug 30, 2017 at 18:12
  • 2
    $\begingroup$ Sometimes a description is just that: it's not a model and it's not a formal test. That's what summary statistics are all about, for instance: they provide a succinct sense of important data characteristics. Many "trends" that people commonly consider are of the same sort: did the values go up or down during the period? By how much? There is nothing in the question that suggests that any more than this is desired--although it might be. That's why I encourage you to use your commenting privileges to seek such clarification. $\endgroup$
    – whuber ♦
    Commented Aug 30, 2017 at 21:14
  • 1
    $\begingroup$ I'd welcome explicit comment on how well fitting what are quite possibly very elaborate models works in the clearly stated context of having 30 or so values. I'm afraid this looks like a reflex response not well adapted to the specific question here. Why the OP is not analysing data within the largest relevant context is also a key question and could be part of any answer. $\endgroup$
    – Nick Cox
    Commented Aug 31, 2017 at 9:52
  • 1
    $\begingroup$ It is not necessarily the #of values but the ratio of signal to noise that is important. 30 values with a strong signal is often sufficient to identify a useful model. The series 1,9,1,9,1,9,1,9,5,9 suggests an anomaly at time period #9 . If a longer series was observed say 30 periods and periods 9,19 and 29 contained a "1" then no anomaly is detectable. I often try to answer the "true question" that was in the mind of the op but perhaps that should be confirmed as @whuber critically pointed out. In this the op is looking for guidance to approach and not a rule that will often be insufficient $\endgroup$
    – IrishStat
    Commented Aug 31, 2017 at 11:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.