I have a set of sequences of numbers, each sequence is independent from each other. I'd like to know if, "in general", these sequences increase, decrease or remain the same.

What I have done so far is a fitted a linear model to each sequence, so that I can use the gradient to determine if the sequence is increasing or not.

I am then using a Wilcoxson U test to test if to compare if the positive gradients are as large as the negative ones (in absolute value, of course).

Is this a good solution to my problems? What are the threats of this solution? What would be a better one?

  • $\begingroup$ Is each component of each sequence uncorrelated with each other? If not, this could be misleading, since you have autocorrelation in the responses. $\endgroup$ – Macro Jul 14 '11 at 20:47
  • $\begingroup$ I don't know if they are correlated or not. As this comes from a time series, it might well be, but that is one of the things I wish to find out too! $\endgroup$ – rafalotufo Jul 14 '11 at 21:11
  • $\begingroup$ What's wrong with taking the first difference and performing a test of location (like the t-test)? $\endgroup$ – shabbychef Jul 14 '11 at 23:26
  • $\begingroup$ @shabbychef You are assuming the ARIMA filter which will convert an autocorreleated series to a white noise series. Unfortunately a first difference filter an inject structure see the Slutzky Effect en.wikipedia.org/wiki/Eugen_Slutsky which teaches us that one can create an autocorrelated series by differencing a white noise series. Assuming a filter,any filter can be quite dangerous. ARIMA model identification en.wikipedia.org/wiki/Box%E2%80%93Jenkins is the procedure to follow in order to yield an equation and an error process that is Gaussian. See my comments in my answer . $\endgroup$ – IrishStat Jul 15 '11 at 0:57

It appears that each sequence has a set (equally spaced) of possibly auto-correlated historical values. To answer the question is the sequence expected to increase,decrease or remain the same is at the heart of time series modelling with Intervention Detection. For example each sequence may be described in two possible ways : 1) y(t) = y(t-1) + trend plus etc ARMA structure OR 2 ) y(t)= b0 + b1*t where t=1,2,3,..... . plus ARMA structure. Two possible ways of assessing trend ! Now in general in case (1) there could be multiple differencing operators or in case (2) there could be multiple trend break points . Now just to generalize one step further i.e. make less presumptive specifications about the model sample space, either 1) or 2) could possibly include one or more Level or Step Shifts which are not trend changes but intercept changes. Tons of software confuse i.e. fail to distinguish between trend changes and level changes. Not to make this more complex than it already seems to be, one might have changes in parameters or changes in error variance over time. Thus your problem is solved . Now all you have to is to find out how to implement this in some reasonable time frame. Be careful to challenge a presumed a model-based approach that doesn't verify the Gaussian Assumptions or doesn't adhere to strict tests of necessity and sufficiency of a proposed model. Whew ! This tires me out specifying what you have to do to correctly answer your questions.

  • $\begingroup$ Thanks. Do you know about any tutorial I could use to do this, preferably with R? $\endgroup$ – rafalotufo Jul 15 '11 at 1:32
  • $\begingroup$ @rafalotufo Sorry, I don't know that one exists as what I laid out is state-of-the-art. If I can help with some examples I would certainly do that. You can post 1 of your sequences and I will analyze it ( not with R ) and post the results. $\endgroup$ – IrishStat Jul 15 '11 at 11:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.