26

You should use the signed rank test when the data are paired. You'll find many definitions of pairing, but at heart the criterion is something that makes pairs of values at least somewhat positively dependent, while unpaired values are not dependent. Often the dependence-pairing occurs because they're observations on the same unit (repeated measures), but ...


17

You are totally correct in your assumption that error bars representing the standard error of the mean are totally inappropriate for within-subject designs. However, the question of overlapping error bars and significance is yet another topic, to which I will come back at the end of this commented reference list. There is rich literature from Psychology on ...


15

(I'm assuming you mean "sample" and not "population" in your first paragraph.) The equivalence is easy to show mathematically. Start with two samples of equal size, $\{x_1,\dots,x_n\}$ and $\{y_1,\dots,y_n\}$. Then define $$\begin{align} \bar x &= \frac{1}{n} \sum_{i=1}^n x_i \\ \bar y &= \frac{1}{n} \sum_{i=1}^n y_i \\ \bar d &= \frac{1}{n} \...


11

No, it is not a valid nonparametric alternative. The rank sum test (either original Wilcoxon flavor, or New Improved Mann-Whitney $U$ varieties): ignore the rankings used by the Kruskal-Wallis test, and do not employ pooled variance for the pairwise tests. See, for example, Kruskal-Wallis Test and Mann-Whitney U Test. (Also the pairwise.wilcox.test seems ...


10

The question does not seem to be about error bars so much as about the best ways of plotting paired data. In essence error bars here are at most a way of summarizing uncertainty: they do not, and they necessarily cannot, say much about any fine structure in the data. Parallel coordinate plots -- sometimes called profile plots, a term that means different ...


10

If the two sets of data are identical because the variables are defined on a discrete set of values, then the assumptions of the t-test are false, since the random variables aren't even continuous. As such, any normal-theory calculation would not yield the correct p-values. I think the "correct" t-test p-value would be NaN. Alternatively, consider the ...


9

If I understand the context correctly, then McNemar's test is exactly what you want. It compares two binomial variables measured in each subject, sort of a paired chi-square test. The key point is that your data are paired -- you've measured two different binomial outcomes in each subject, so need a test that accounts for that.


9

I see no reason to believe you did something wrong just because the test was significant, even if the mean difference is very small. In a paired t-test, the significance will be driven by three things: the magnitude of the mean difference the amount of data you have the standard deviation of the differences Admittedly, your mean difference is very, very ...


9

To know if a difference is really large or small requires some measure of scale, the standard deviation is one measure of scale and is part of the t-test formula to account in part for that scale. Consider if you are comparing the heights of 5 year olds to the heights of 20 year olds (humans, same geographic area, etc.). Intuition tells us that there is a ...


9

I am trying to pick one from these two tests to analyze paired data. Does anyone know any rules of thumb about which one to pick in general? The signed rank test carries an assumption about symmetry of differences under the null that the sign test need not. (That assumption is necessary in order that the permutations of the signs attached to the unsigned ...


9

Sure, you can do that. You don't have to test against a null hypothesis of $0$ (sometimes called a "nil null"); you can test against any value. You also don't have to do a two-tailed test; you can perform a one-tailed test (when specified a-priori). The paired $t$-test is: $$ t = \frac{\bar x_D - \mu_{\rm null}}{\frac{s_D}{\sqrt N}} $$ Thus, to combine ...


8

Well, if you knew the variances in the unpaired and in the paired (which would generally be a good deal smaller), the optimal weights for the two estimates of difference in groups means would be to have weights inversely proportional to the variance of the individual estimates of the difference in means. [Edit: turns out that when the variances are ...


8

You need McNemar's test (http://en.wikipedia.org/wiki/McNemar%27s_test , http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3346204/). Following is an example: 1300 pts and 1300 matched controls are studied. The smoking status is tabled as follows: Normal |no |yes| Cancer|No |1000|40 | |Yes |200 |60 | Each entry of the table ...


8

It's probably not a problem at all, provided the sample sizes are similar to each other. There could be complications with small sample sizes, though. Intuitively, an average of a small sample is more variable than averages of large samples. If some pairs are both based on small samples, they could create unusual outlying values. It's well known that the ...


8

If you have a number of different measurements in your data.frame, then pairs will show scatterplots of between all pairs of these measures. Example data: x <- rnorm(100) obs <- data.frame(a = x, b = rnorm(100), c = x + runif(100, .5, 1), d = jitter(x^2)) pairs(obs) This is a data.frame with ...


7

I'm not a researcher, I'm a statistics major though. I'll first layout the requirements for the Wilcoxon Signed Rank Sum Test (WSRST). The WSRST requires that the populations be paired, for example, the same group of people are tested on two different occasions or things and MEASURED on the effects of each and we then compare the two things or occasions. ...


7

Given how I understand your aim, I'd just calculate paired differences (bars - dots), then plot these differences in a histogram or kernel density estimate plot. You could also add any combination of (1) a vertical line corresponding to zero difference (2) any choice of percentiles. This would highlight what portion of the data have bars exceeding dots, and ...


6

Here are some thoughts. I basically just arrive to Greg Snow conclusion that This problem has distinct similarities to the Behrens-Fisher problem. To avoid handwaving I first introduce some notations and formalize the hypotheses. we have $n$ paired observations $x_i^{pA}$ and $x_i^{pB}$ ($i = 1, \dots, n$); we have $n_A$ and $n_B$ unpaired observations $x_i^...


6

Guo and Yuan suggest an alternative method called the optimal pooled t-test stemming from Samawi and Vogel's pooled t-test. Link to reference: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.865.734&rep=rep1&type=pdf Great read with multiple options for this situation. New to commenting so please let me know if I need to add anything else....


6

Yes, this is possible and even fairly easy, but additional information is required. Specifically, we have to make an assumption about what the correlation between the observations from each pair are. The effect size as a difference in standard deviation units is usually referred to as $d$. We can apply a correction factor to $d$ to incorporate the ...


6

You can't make much of a distributional assessment at n=5 -- you can (sometimes) see non-normality at n=5 -- but at such small sample sizes the non-normality has to be quite strong to have a good chance to pick it up. The level sensitivity of the paired t-test to heavy tails can be substantial for example, but that can be tricky to spot. If you have no good ...


6

I'm not sure about a reference for this result, but it is possible to derive it relatively easily, so I hope that suffices. One way to approach this problem is to look at it as a problem involving a quadratic form taken on a normal random vector. The pooled sample variance can be expressed as a quadratic form of this kind, and these quadratic forms are ...


5

I think you've made some effort to solve this, so I'll give some hints. "Paired" is a statistics term that means pretty close to its usual meaning. "Independent" means that one subject's data has no relation to another. So, are your data paired or independent? As to this: but I am only familiar with paired t-tests where we test if the difference is zero ...


5

In light of the edit to your question, the confusion appears to be over the specification of the alternative hypothesis. If you are just testing for whether the mean of the differences is not equal to $0$, i.e. $H_1:\mu_{d} \ne 0$ then you want to do: t.test(d,mu=0,alternative="two.sided") You will notice this gives identical results whether d is defined ...


5

I would recommend you to use Generalized Estimating Equations (GEE). This is analogous to Repeated-measures ANOVA but allows for non-continuous or non-normal response because, being a Generalized linear model, it adopts various link functions. With your example, I'd use binomial distribution with logit link (albeit you might prefer probit). The input data ...


5

I figured this out after some searching and consultation with an old stats professor. To employ a nested model ANOVA in SPSS, some editing of the syntax must be done, and the nesting effect is determined in the /DESIGN command. In the end, my syntax looked something like this (depending on your preference of parameters and posthoc tests): UNIANOVA Y BY ...


5

You are right that Fisher's exact test is inappropriate for your data. You will have to re-form your contingency table. The new table will be for pairs, thus it will appear to have half as many data represented (in your case 40 instead of 80). For example, imagine your data looked like this (each set of paired subjects is in its own row, and 1 indicates a ...


5

The difference becomes clear if you understand the null/alternative hypothesis of each test. ANOVA's null hypothesis is that the group means are the same, while the alternative is that at least one group mean is different from the others. This analysis does not tell you which group mean is different, or which differences between groups are significant, it ...


5

Some ideas: The plot shows that the agreement is fairly good, and with your amount of data maybe you want to study the agreement with more details, not only reduce it to one summary statistic. For more ideas on agreement-statistics peruse the tag agreement-statistics. For your series: What about study some plots of the difference $D=B-A$ against $A$? Is ...


4

I don't think fisher exact test is appropriate here is it? It is saying that pre-test there are 34 points arranged in this manner (343344355), and that post test there are 44 points. The arrangement of those 44 points is consistent with the pattern (marginal) established in the pre-test. It is also saying that each pairing looks similar (row part of the ...


Only top voted, non community-wiki answers of a minimum length are eligible