Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 4 down vote favorite

I have a set of sequential data around of around 12k points. Each is a 1 or -1. They are split something like 53%/47%. I want to test the hypothesis that this sequence comes from a random walk with a constant parameter. What sort of statistical test should I use?

(I'd also be grateful for any book recommendations in this general area)

Thanks very much

share|improve this question
It sounds to me that what you have are the steps of a process that you want to test whether it is a time-homogeneous simple random walk (possibly asymmetric). So, can the problem be reasonably codified as follows? You observe $\xi_1,\xi_2,\ldots$ each in $\{+1,-1\}$ with $\mathbb P(\xi_i = +1) = p(i)$. You want to test the hypothesis that $p(\cdot)$ is actually a constant function. Is this right? If so, do you know (i.e., can you assume) anything further about the properties of $p(\cdot)$, e.g., a single change in level or that it is smooth, etc? – cardinal Jan 5 '12 at 12:28
Yes, spot on. At this stage I'm trying simply to arrive at a confidence value with which I can reject the hypothesis of p(.) being a constant process. I don't know a priori the value of the asymmetry, although I'm happy that it's roughly 53/47. – Omnium Jan 5 '12 at 14:04
Note that the variance of the sample mean is maximized when $p(\cdot)$ is constant. In other words, suppose $Y_1, Y_2,\ldots$ are $\mathrm{Ber}(p_i)$ in $\{0,1\}$. Then $\mathbb E \overline{Y} = n^{-1} \sum_i p_i = \bar{p}$ and $\mathrm{Var}(\overline{Y}) = n^{-2} \sum_i p_i (1-p_i) \leq n^{-1} \bar{p}(1-\bar{p})$ by Jensen's inequality. The right-hand side is the variance of $\overline{Y}$ if $p(\cdot)$ is constant (always equal to $\bar{p}$). Hence, heuristically we'd expect the walk to be "too close" to the "trend" $\sum_{i=1}^n p_i$ on average if $p(\cdot)$ were not constant. – cardinal Jan 5 '12 at 15:49
You've rather lost me there I'm afraid. – Omnium Jan 5 '12 at 17:16

1 Answer

up vote 3 down vote

You could try Runs Test. Let $n_1$ bet the number of +1 runs and $n_2$ bet the number of -1 runs. You could use the test as in the wiki page.

share|improve this answer
Thanks for amazingly fast response! I don't think it's quite what I'm looking for. It seems to be looking more at non-independence of neighbouring points. I know that my data points are entirely independent. The cumulative graph wanders away from the 53% trend. I don't believe that the 1/-1 split early on (data is time ordered) is generated in the same mix as later. I've produced simulated graphs and they don't wander away from the trend like the data - I'm convinced that it isn't a constant valued random walk, but I can't think how to assess the degree of certainty of that conclusion. – Omnium Jan 5 '12 at 11:19
Are you interested to test the data come from 50%/50%.What did you mean by constant parameter? – vinux Jan 5 '12 at 11:24
A random walk consisting of a +1 move 53% of the time, and a -1 move 47% of the time. – Omnium Jan 5 '12 at 11:32

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.