3
$\begingroup$

For purposes of a non-technical analysis of a simple time-series, what is a quick (but effective) technique / approach to understand (or even 'conclude' with some confidence level) that a trend has shifted (versus that the most recent 'unexpected' observations are due to 'noise' or driven by the unexpected error term E)?

One such technique could be to look at moving averages - if so, what would be sensible 'ranges' for the data below (and why)?

PS. This is primarily meant for hands-on business oriented questions, not an academic exercise.

$\endgroup$
  • $\begingroup$ Could you explain what you mean by "the data below"? $\endgroup$ – whuber Nov 5 '16 at 20:26
  • $\begingroup$ i tried to provide (paste in) the data but character limit of 550char doesn't let me do this at this point. is there an alternative way or method? $\endgroup$ – apprenticeQ88 Nov 5 '16 at 22:24
  • $\begingroup$ 5.30 5.25 5.16 5.17 5.08 4.99 4.99 4.91 4.84 4.65 4.47 4.38 4.28 4.15 4.05 3.96 3.81 3.73 3.67 3.59 3.52 3.38 3.30 3.22 3.18 3.10 3.04 3.01 2.93 2.93 2.89 2.86 2.78 2.66 2.55 2.43 2.35 2.27 2.23 2.17 2.12 2.09 2.03 2.04 1.93 1.84 1.77 1.70 1.66 1.66 1.58 1.57 $\endgroup$ – apprenticeQ88 Nov 5 '16 at 22:40
  • $\begingroup$ closing price at end of day. $\endgroup$ – apprenticeQ88 Nov 5 '16 at 22:41
  • $\begingroup$ There is no (practical) character limit--you were probably trying to paste the data as a comment. It is better to include them within the question itself. $\endgroup$ – whuber Nov 5 '16 at 23:10
2
$\begingroup$

To examine/test for an exceptional value , one needs to have an expectation for that value . Essentially one needs to compute the probability of what was observed before the observation took place. That having been said the general rule is to build/identify a reasonably sufficient model using (if existing)user-suggested causal/support/exogenous/helping variables (series) and whatever autoprojective scheme (if the data is temporal or spatial) that can be formed. This model could also include level shifts and anomaly adjustments and in the case of time series or spatial data time trends and seasonal pulses.

Following Bacon :To do science is to search for repeated patterns. To detect anomalies is to identify values that do not follow repeated patterns. For whoever knows the ways of Nature will more easily notice her deviations and, on the other hand, whoever knows her deviations will more accurately describe her ways.One learns the rules by observing when the current rules fail.

Restated perhaps more simply ....

Can you tell me the probability that a single data point (e.g. the latest reading) came from the distribution represented by all the previous data points?

Now if an anomaly is detected it may confirm an emerging trend change or it could be a simple plulse , or an indication that a new level was emerging or possibly even a symptom of a change in error variance or an emerging change in parameters.

Simple straightforward questions like your sometimes require serious analytical often out of the reach of uninformed users as "fools walk where angels fear to tread" . If you wish to answer this question for a serious business problem get serious help.

EDITED AFTER DATA WAS POSTED:

I took your data (52 observations) and analyzed with the help of AUTOBOX ( a time series package that I have helped to develop) and obtained enter image description here . The ACF of the original data strongly suggests non-stationarity ( a symptom ) enter image description here and AUTOBOX suggested a cause enter image description here . THe interesting thing here is that the dominant structure initially masks the seasonal structure but a comprehensive model identification scheme yielded a reasonable ARIMA structure and two anomalies ( curiously one at time period 51 ). enter image description here . The residial ACF suggests model sufficiency enter image description here with a reenter image description heresidual plot here .

The plot of the actual and the outlier adjusted series is here enter image description here . This analysis suggests a significant exogenous effect at period 51 . Hope this helps .... In closing this is the model summary showing an R-Sq of .99874 enter image description here without the dangerous polynomial factor that you found . This is aprroximately a 65% reduction in the error mean square. (.0042- .00126)/.0042 . All of this from a high-powered analysis that does not require advanced theoretical skills but access to high-powered software (a productivity aid) potentially complementing knowledgeable analysts.

$\endgroup$
  • $\begingroup$ say Excel gives me the following equation for the polynomial curve: $\endgroup$ – apprenticeQ88 Nov 5 '16 at 22:14
  • $\begingroup$ thank you, Sir, for taking the time to reply. y=0.0006x^2 - 0.1082x + 5.5688, and R square = 0.9958. in this context here I am not so much interested in developing a high-powered analysis that requires advanced theoretical skills but more so a quick and dirty reference point that could be considered as part of a triangulation (along with other analysis outputs from, say, people that understand the business context, historical drivers, previous empirical experience....you get the point). The analysis technique I am looking for need to be executed quickly and with limited resources. $\endgroup$ – apprenticeQ88 Nov 5 '16 at 22:22
  • $\begingroup$ i tried to provide (paste in) the data but character limit of 550char doesn't let me do this at this point. is there an alternative way or method? $\endgroup$ – apprenticeQ88 Nov 5 '16 at 22:23
  • $\begingroup$ Excel gives me the following equation for a Linear Trendline: y = 0.0766x = 5.2852 with an R square of 0.9852. The data i have used is based on actual observation (though slightly obscured/modified). And again--it is understood that the result of this exercise cannot be used on its own but need to be evaluated in the context further analysis, adjustments, additiontal/other inputs, modifictions, etc . {in short it's a starting point and possibly part of a triangulation exercise). $\endgroup$ – apprenticeQ88 Nov 5 '16 at 22:35
  • $\begingroup$ here's the data (end-of-day price points) - $5.30 $5.25 $5.16 $5.17 $5.08 $4.99 $4.99 $4.91 $4.84 $4.65 $4.47 $4.38 $4.28 $4.15 $4.05 $3.96 $3.81 $3.73 $3.67 $3.59 $3.52 $3.38 $3.30 $3.22 $3.18 $3.10 $3.04 $3.01 $2.93 $2.93 $2.89 $2.86 $2.78 $2.66 $2.55 $2.43 $2.35 $2.27 $2.23 $2.17 $2.12 $2.09 $2.03 $2.04 $1.93 $1.84 $1.77 $1.70 $1.66 $1.66 $1.58 $1.57 $\endgroup$ – apprenticeQ88 Nov 5 '16 at 22:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.