Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 1 down vote favorite

I am running an experiment where I collect two data sets and I wish to measure the difference between the two. The two data sets are independent, with unknown probability distribution, and may not always have the same length.

Calculating the mean difference is easy as pie, but i also want a measure of the standard deviation and I'm not sure how to go about doing that. At first I just got the std deviation of the second data set minus the average of the first set, but in retrospect, I'm not sure that is entirely correct.

Any advice out there?

As an example of the data:

3.98   4.39   4.09   4.31   3.81   3.67   3.94   3.90   4.39   3.60   3.99   3.53   3.82

vs

3.95   4.51   4.49   4.43   4.55   4.41   4.68   4.22   4.45   4.59   4.42

Edit: I want to etch this in stone a bit more to explain why ttest would not be the answer I'm looking for: All computations are done in Matlab. note: var(one) = 3.61e-6, var(two) = 5.01e-06.
Using gui11aume's answer: 

 std = 2.93e-3

when doing

 [h,p,ci,stats] =ttest2(one, two)
     stats.sd = 2.08e-3

realizing the variances are likely unequal, one should rewrite this to:

[h,p,ci,stats] =ttest2(one,two,0.05,'both','unequal')
    stats.sd =  [1.9e-3 2.23e-3]

once again, thank you for your time

share|improve this question
1  
I don't see the problem here. You can't take the difference of two data sets, only the difference between a function (like the mean) of one and the same function of the other. If there's some reason to disapprove of the mean (which I don't understand yet), there are other options (like the median). The standard error of the mean difference is the denominator of the t-test, which you can find anywhere. Can you state your final goal & why you don't want to work w/ the difference in means / t-test? – gung May 25 '12 at 21:25
I think you need to tell us when you are talking about when you say you want to measure the standard deviation. I am as confused as gung. What you seem to be calculating does not resemble any statndard deviation that I know. You have no reason to pair the data and certainly can't when the sample sizes are different. Why not just calculate the standard deviation of the the difference between means. – Michael Chernick May 25 '12 at 21:47
In general it would be s1^2 +s2^2 -2 Cov(m1, m2) where s1 is the sample standard deviation from the sample from population 1 and s2 is the sample standard deviation from the sample from population 2 and Cov(m1, m2) is the covariance between the sample estimates m1 and m2. Since the samples are independent the covariances are 0. So just add the separate sample variances for population 1 and population 2. – Michael Chernick May 25 '12 at 21:49
@gung The mean is fine but doesn't provide a complete perspective. I'm not trying to prove that there is a difference (a fact I'm certain of), but trying the measure the difference in outcomes between these two sets, and a SD helps quantify that value. – Rasman May 25 '12 at 21:52
You aren't obligated to convert your t-score into a p-value, you can just leave it as $t$. The t-score is the difference b/t the 2 means standardized relative to the SD of the sampling distribution of differences. Alternatively, you could standardize the mean difference relative to the pooled SD of the data distributions, under the assumption of homogeneity of variance, this is the square root of the weighted average of the variances (w/ group df's over total df as weights) & is called 'Cohen's d'. – gung May 25 '12 at 22:05
show 2 more comments

2 Answers

up vote 5 down vote accepted

If $X$ and $Y$ are independent, the variance of $X-Y$ is $Var(X) + Var(Y)$. So the variance of the difference of means is the sum of the variances of each mean. This variance is unknown, but you can estimate it easily by the sum of the estimated variances: $S_1^2/n_1 + S_2^2/n_2$.

share|improve this answer
so you're saying if just compute the variance of each set independently, I can add the two and it will show the spread of my difference? I guess it kindda makes sense as I'm doing E(X) - E(Y) – Rasman May 25 '12 at 21:57
you're right, I'm doing a cross-correlation of two indepedant pdfs. I thought it was a simple problem, just couldn't figure out what I was doing wrong. – Rasman May 25 '12 at 22:07
gui11aume's answer is correct I meant to divide the standard deviations in my remarks by the sample sizes as he did so that it is an estimate of the variance of the mean difference. We get this answer because Cov(X,Y)=0 as would appear in the general formula before assuming independence. Then replacing Cov(X,Y) with 0 gives gui11aume's formula. – Michael Chernick May 25 '12 at 23:04
@Rasman said "I can add the two and it will show the spread of my difference?" -- note that variance is not the same as spread! – Glen_b May 26 '12 at 4:57
@Glen_b Sorry my bad, jargon can elude me at times. However, the variance is a measure of spread (one of the many). – Rasman May 26 '12 at 12:01
show 1 more comment
up vote -3 down vote

t is a standard t test and there are easy calculations for a standard deviation of the sampling distribution of differences between means. You can use pooled or separate estimates.

share|improve this answer
6  
Welcome to the site, Susie. Please try to keep your comments/answers constructive - there is no need to criticize the OP for not knowing terminology or concepts, regardless of how basic you think it is. – Macro May 26 '12 at 0:38
(1) Please, take time to register your account (I've merged your two unregistered ones, thanks to @Macro); (2) Consider also reading our FAQ to get familiar with site policy and what we generally expect from Questions and Answers there: in particular, Macro kindly pointed out that we generally look for constructive and thoughtful replies. I have edited some of your reply, but feel free to point to authoritative reference about pooled SD calculation, paired vs. independent t-test, etc. – chl♦ May 26 '12 at 10:21
@Susie thank you for your response, but a) I can't do a t-test in real time given the limitations of the program I'm using (LabView). b) I'm not doing hypothesis testing, and I need a different answer then that which it will provide c) Most Ttests assumes a normal distribution, which this is not (which I explained). d) as a measure of why your answer is insufficient, let's play in matlab: gui11aume's answer: std = 2.93e-3, using ttest2(one, two), one gets 2.08e-3, or more correctly: ttest2(one,two,0.05,'both','unequal'), which results in [1.9e-3 2.23e-3] – Rasman May 26 '12 at 12:16

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.