Timeline for Determining Appropriate Test: two categorical groups with a goal (split test?) (Newb)
Current License: CC BY-SA 3.0
13 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Mar 2 at 19:25 | comment | added | kjetil b halvorsen♦ | @Glen_b: Can you write it as an answer then? | |
| Oct 27, 2014 at 22:32 | comment | added | Glen_b | I was initially thinking this was effectively a duplicate question, but there's enough here now that I think isn't covered by any single question that I should probably write it up as an answer. | |
| Oct 27, 2014 at 22:30 | comment | added | Glen_b | Yes; this is fairly common. You don't have a random sample from the target distribution (because you can't), but you have a group which is hopefully representative. If you allocate to groups A/B at random, you have a randomization argument as far as applying a test for the group effect (and the usual test can be argued to be an approximation to a suitable randomization test); there's still a need to argue that result carries over to the population of interest. Sorry several of my links in my previous couple of comments didn't work as links - you should still be able to copy and paste them | |
| Oct 27, 2014 at 22:26 | comment | added | Doug Fir | Thanks again for the feedback I think I'd typed my last comment as you were typing yours. Purpose of test is to know which email sequence to use in the future, so in a sense is a sample in that the whole population are all the people who will sign up in the future but have not yet done so | |
| Oct 27, 2014 at 22:09 | comment | added | Glen_b | (ctd) ...Both tests are mentioned in the table [here](en.wikipedia.org/wiki/Statistical_hypothesis_testing#Common_test_statistics). $\quad$ Um, I didn't provide a link in my first two comments. It's not clear to me that you do have the whole target population. What use is it to know that something has a higher proportion on some fixed group from the past? What was the purpose of this analysis? | |
| Oct 27, 2014 at 22:07 | comment | added | Glen_b | An example of a two-sample proportions test is [here](stats.stackexchange.com/questions/78474/correct-test-for-proportions/); there are several links describing it here. There's a chi-squared test here but for more than two groups, that shows the subtraction of the count of interest from the population exposed to obtain the second set of counts for the chi-squared test...(ctd) | |
| Oct 27, 2014 at 22:06 | comment | added | Doug Fir | @Glen_b thanks for the feedback. RE your first comment I guess by population I mean it's not actually a sample: either you were placed in group A or B 50/50 so everyone is included, not just a sample. Presumably that removes inference from the equation since we have the whole population in question? Does this change things? Z-test still apply? The link you provided does not appear to work | |
| Oct 27, 2014 at 21:35 | comment | added | Glen_b | Assuming that these groups of 500 are intended to be random samples from the population of interest, you'd either use a two-sample proportions test (typically done as a z-test) or a 2x2 chi-square (A/B vs customer/not-customer). Properly organized the two should be equivalent. (There are other possible tests.).... there are many posts on this topic here. | |
| Oct 27, 2014 at 21:29 | comment | added | Glen_b | When you say "sent out to a population" ... do you mean he population about which you wish to make inference? Or do you simply mean they were sent out to some number in each group? If you intend the results to generalize outside those 1000 people (as you would if you wanted to say "B does better" about other people), they're not the target population but presumably some kind of sample. | |
| Oct 27, 2014 at 20:42 | history | edited | Doug Fir | CC BY-SA 3.0 |
Adding more detail to question title
|
| Oct 27, 2014 at 20:16 | history | edited | Doug Fir | CC BY-SA 3.0 |
formatting
|
| Oct 27, 2014 at 20:15 | review | First posts | |||
| Oct 27, 2014 at 20:16 | |||||
| Oct 27, 2014 at 20:05 | history | asked | Doug Fir | CC BY-SA 3.0 |