5
$\begingroup$

I need help, since my skills and knowledge in statistics are really limited. Using Pearson correlation I found that there is some relationship between a binary one and a second variable taking values between 1 and 4. So now I want to understand if there are significant differences between the two groups defined by the binary variable.

The 2 variables are the following:

X = (0 0 0 0 0 0 0 0 0 0 1 1 1 1)
Y = (4 3 4 3 3 3 3 3 3 3 4 4 4 4)

To determine if there's a significant difference between $Y_{X=0}$ and $Y_{X=1}$, I used a Mann-Whitney test. Was this the correct choice? Would it have been preferable to use a K-S test? And why not a t-test?

I am actively looking for an answer to these questions.

Improve this question
$\endgroup$
5
  • $\begingroup$ Here: elegans.som.vcu.edu/~leon/stats/utest.cgi $\endgroup$
    – user900978
    Commented Mar 21, 2012 at 12:30
  • $\begingroup$ I've made significant edits to your post. Please check carefully that I didn't alter its original meaning. $\endgroup$
    – chl
    Commented Mar 21, 2012 at 19:23
  • 1
    $\begingroup$ It is noteworthy that the answers reflect two very different interpretations of these data, so a clarification is in order. The reply by Matt Parker (and also the edit by @chl) views $X$ as indicating two groups (0 and 1) and takes the question as a request to compare the $Y$ values between the groups. The reply by Glen_b takes the question as a request to compare the 14 values of $X$ to the 14 values of $Y$. Which interpretation is the intended one? $\endgroup$
    – whuber ♦
    Commented Mar 21, 2012 at 19:42
  • $\begingroup$ Thanks to Chi for his edits, now my question seems really clear. The right interpretations is the first, Matt and Chi caught the idea in my mind. $\endgroup$
    – user900978
    Commented Mar 22, 2012 at 13:55
  • $\begingroup$ Try Point Biserial Correlation. $\endgroup$
    – isaias sealza
    Commented Sep 18, 2014 at 7:26

4 Answers 4

Reset to default
5
$\begingroup$

I think the Mann-Whitney/Wilcoxon ranked-sum test is the appropriate test. The KS test is specifically for comparing continuous distributions - your ratings are ordinal, so it doesn't seem appropriate here.

The t-test and the Wilcoxon ranked-sum differ in that the t-test is comparing the means of the two distributions, while the Wilcoxon is comparing the 'locations' by looking at how the values of the two distributions compare when ranked. When your entire ratings distribution has only two values, one group has only ratings of 4 and your sample size is 14, the t-test seems less appropriate to me. It works, but I just have a harder time with it conceptually. This data is more binomial than it is continuous!

Here's how I'd do all of that in R, which is a freely available software for statistical computing (a step up from using websites to compute tests, I think...)

# A vector of data for people with smartphone experience
smartphone <- c(4, 4, 4, 4)

# A vector of data for people without smartphone experience
dumbphone <- c(4, 3, 4, 3, 3, 3, 3, 3, 3, 3)

# The Mann-Whitney/Wilcoxon ranked-sum test
wilcox.test(x = smartphone, y = dumbphone)

# t-test for comparison
t.test(x = smartphone, y = dumbphone)

# And, why not, a test of proportions
# Consider 4 as the event, comparing 4/4 to 2/10
prop.test(x = c(4, 2), n = c(4, 10))
Improve this answer
$\endgroup$
1
  • $\begingroup$ Hi Matt, really tnaks for your help. There's less fog on my mind now! $\endgroup$
    – user900978
    Commented Mar 22, 2012 at 13:21
1
$\begingroup$

Plainly there's a difference between the groups - the locations are completely different, with no overlap in the groups. Any sensible test will reject even at this small sample size.

I assume you want to ask a different question of the data than that.

Improve this answer
$\endgroup$
4
  • $\begingroup$ At he following link: elegans.som.vcu.edu/~leon/stats/utest.cgi i found that with Mann-Whitney thre's a marginally significant difference between the first 10 elements of Y and the last 4. However, my question is: is Mann-Whitney the right test to try for this sampale? Is K-S better? and the T-test? I have some difficult with statistical tests.. $\endgroup$
    – user900978
    Commented Mar 21, 2012 at 12:33
  • 3
    $\begingroup$ @user900978 - Please be clearer about exactly which difference you want to test for significance. Apparently you don't want to test the difference between X and Y, as I think Glen_b (and I) initially thought. If you are testing the first 10 values of Y vs. the last 4, then it is important whether you are doing so based on some prior hypothesis or merely because, after collecting the data, that is where you have noticed the greatest difference. If the latter, any statistical test will be clouded by the "data snooping" that you have done. $\endgroup$
    – rolando2
    Commented Mar 21, 2012 at 12:55
  • $\begingroup$ Tnaks for your answers. Excuse me if I explain bad my questions, I'm not so used to speak in English. However, I need to find if there is the first 10 values of Y are significantly different from the last 4 values of Y. The first 10 values are ratings provided by people who hasn't experience with smartphones, the last 4 from people having experience with smartphones. $\endgroup$
    – user900978
    Commented Mar 21, 2012 at 13:50
  • $\begingroup$ If your decision to test this particular division is based on looking at the data, your test p-values are meaningless (at the least, way too small). $\endgroup$
    – Glen_b
    Commented Feb 17, 2013 at 3:22
1
$\begingroup$

You should t-test the mean difference between smart phone users and non-smart phone users. Pearson's R is not appropriate for binary variables because it assumes that both variables are normally distributed (and X, in your case, cannot be because it is dichotomous).

A t-test will ask if the mean score on the outcome (Y in this case) is significantly different across the two categories of X. This assumes that your Y variable is something that is appropriate for averaging. In other words it should at least be ordinal (meaning rank ordered categories) with 5 or so possible categories, an interval level of measurement is technically required but, in practice, ordinal is fine.

There are several statistical tests and correlation coefficients that can be calculated and what you choose will probably be a function of whatever the convention is in your field for the sort of data you have. The t-test is pretty common across all fields, that's why I suggest it.

Note also that the t-test doesn't give you an effect size - you are comparing the means of the two groups under the null hypothesis that the difference between them is 0 in the population. A statistically significant finding means only that the means are not the same in the population. You might also consider some effect size statistic - that's what Pearson's R is, but again that's not appropriate for binary variables.

Improve this answer
$\endgroup$
3
  • $\begingroup$ I wanted to add that the Mann-Whitney test is essentially the same as the t-test but assumes that the Y variable is ordinal, rather than interval. In practice the results will be the same - the difference between the groups will be significant at the .05 level. $\endgroup$
    – Will
    Commented Mar 21, 2012 at 16:19
  • $\begingroup$ Will, thanks for your answer. You have been really clear. And what about Kolmogorov-Smirnov? If I right understood, it gives similiar results, but is preferred to Mann-Whitney for small samples. Is this right? $\endgroup$
    – user900978
    Commented Mar 21, 2012 at 16:44
  • $\begingroup$ I'm not familiar with the K-S test but after a bit of googling, it looks to me to be a test for comparing the distribution of a continuous variable across two groups or samples. This would not be appropriate for your data. For starters, one group consists of only 4 cases and there is no 'distribution' because they all have the same value (4). More generally, if your data is as you describe (i.e. 10 cases) then there are going to be minimal conclusions you can draw from it, regardless of how thoughtfully you design your analysis. $\endgroup$
    – Will
    Commented Mar 22, 2012 at 15:17
0
$\begingroup$

Another option is Pearson's Chi-square test, which is appropriate for categorical variables. You can use it to test whether the variable Y is independent of X, in other words, whether Y has any influence on X. However, you would need a sample larger than 10 values, and also I'm not sure if this test can be used with a 2x2 contingency table... I seem to recall there was a minimum number of cells.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ Hmm...We're always glad of contributions, but you seem so uncertain, perhaps you should do a little research before committing to a reply? $\endgroup$
    – whuber ♦
    Commented Mar 21, 2012 at 20:27
  • $\begingroup$ I'm uncertain about the calculation details, but I'm pretty certain that the test could be useful in this case. $\endgroup$
    – Ernest A
    Commented Mar 21, 2012 at 20:57
  • $\begingroup$ Nevertheless, it is still for small samples (<25) - see e.g. SPSS stat. soft tutorials for the explanation & this, test it in python $\endgroup$
    – JeeyCi
    Commented Jul 2 at 16:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.