Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 1 down vote favorite

This is a generic question. Let me put forth an example scenario. Say, I had 2 techniques for allocating my daily budget within 4 stocks. Upon allocation, I get the data about the stock's performance the next day. (Am lazy, and go to a movie after allocation).

Problem 1:I have to evaluate or come up with an experimental design to test the two techniques in this sequential experiment over a given number of days.

Problem 2 with caveat: If I split my budget into half and have "one" of the two techniques allocate it amongst the four stocks in the same quantities in two parallel experiments at the same time-I find the following issue on the second day. Though everything from the budget to the allocation was the same. Both the experiments gave different performances -the next day- due to the randomness in the system. Under this situation- where performances are different even under "one" technique- How would I evaluate or design an experiment for comparing two techniques over a given number of days?

Problem 3: If instead of getting the performances the next day- I get minute by minute or hourly performances, and would like to evaluate the two technqiues-what would be your line of thought?

share|improve this question
Hi there, to clarify, you wish to identify the best method to predict stock prices, the latter being a random/chaotic process? – Michelle Mar 9 '12 at 21:32
Hi-Without caveat: Would like to identify the best method, while the data comes in sequentially. aka- If I used a sequential hypothesis testing framework- I would like to account for testing the parameters from multiple distributions while accounting for nonlinearities (like in a generalized additive model) with distributional assumptions. – Praneeth Vepakomma Mar 9 '12 at 22:06
Question with caveat: Do the same as above. But- when only one method is applied on the same four stocks by splitting the total money in half and allocating the same proportions on the same 4 stocks. i.e two replicates of same allocations on same stocks-The performances vary in both replicates. (varied initial conditions). Now how do you compare two methods sequentially in this scenario? – Praneeth Vepakomma Mar 9 '12 at 22:08
The issue with trying to predict a random or chaotic series is that, basically, you can't. You may have some apparent short-run predictability by chance, but any model you come up with won't apply in the long-term. So by "stocks" do you mean shares (where my comments apply), or is this an inventory question (which could be predictable)? – Michelle Mar 9 '12 at 22:09
Fair enough- Lets exchange 'shares' with a more reasonable stocastic system- like inventory- or a more abstract form of resource allocation, and get back to the sequential hypothesis testing issue in this framework. Would like to do a sequential hypothesis testing- for two methods, where each point is modeled with a exponential distribution with different parameters like in GAM's and we have this setting for two methods. aka- sequential hypothesis testing at every point for two methods as data is collected and some form of an online likelihood ratio is measured. – Praneeth Vepakomma Mar 9 '12 at 22:14

1 Answer

up vote 0 down vote

The question suggests that a repeated measures ANOVA could work, where technique defines the groups. For how frequently you should measure, that depends on what is important, i.e. the research hypothesis. If you're interested in minute outcomes, measure in minute lots. However, that will give you a lot of data points if you're doing this even over the space of a couple of hours, let alone days. If you end up with a lot of data points, the question is likely to be one of practical significance rather than statistical significance, as you are likely to end up with a statistically significant result regardless of how small the difference is between the techniques, just due to sheer volume of data.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.