# Hypothesis testing - interpreting contradicting results

I wanted to do a hypothesis testing for paired data. My null hypothesis is that Xi is 4 times as big as Yi. So i create a variable Zi = Xi - 4Yi.

The null hypothesis is that Zi = 0

I do the testing in R as follows:

t.value = (mean(Z) - 0) / (sd(Z) / sqrt(length(Z)))
p.value = 2*pt(-abs(t.value), df=length(Z)-1)


I get a p-value higher than 0.05 and i fail to reject the null hypothesis.

My problem is, when i try different values for Z, for example Zi = Xi - 10Yi and Zi = Xi - 50Yi, i still fail to reject the null hypothesis which is strange to me.

My data looks normal but the values for Xi and Yi are tiny. (Xi mean =0.00245945, Yi mean = 0.00109909)

Is there an explanation for this, or am i doing something wrong?

• Can $X_i$ and $Y_i$ be negative? What are their variances or standard deviations? – Henry Dec 5 '17 at 9:02
• Any particular reason why you compute the p-value manually? This doesn't seem to be a special case and therefore seems to be covered by the existing functions. – David Ernst Dec 5 '17 at 9:15
• Yes it can be, standard deviations of Xi = 0.008099045 and standard deviations of Yi = 0.009010012. – G_z Dec 5 '17 at 9:15

If your n is tiny, your test could be so under-powered that it will fail to reject all but the most ridiculous null hypotheses.

edit: I see now that the standard deviations are 6 or 7 times as large as the difference of means. That makes for a quite small effect, Cohen's $d\approx 0.15$, if your sample is representative. If you construct a variable $Z$ that is different from $X-Y$, like the $X-4Y$ you had, this ratio will change accordingly. This still looks like a power problem. Here you see how low your power is with $n=45$:

power.t.test(45,0.14,0.85,0.05,NULL,"paired","two.sided")

Paired t test power calculation

n = 45
delta = 0.14
sd = 0.85
sig.level = 0.05
power = 0.1896772
alternative = two.sided

NOTE: n is number of *pairs*, sd is std.dev. of *differences* within pairs


Here's how many observations you would need for halfway decent power of $0.8$.

power.t.test(NULL,0.14,0.85,0.05,0.8,"paired","two.sided")

Paired t test power calculation

n = 291.2539
delta = 0.14
sd = 0.85
sig.level = 0.05
power = 0.8
alternative = two.sided

NOTE: n is number of *pairs*, sd is std.dev. of *differences* within pairs


This is based on the assumption that the difference of means and the standard deviations (I didn't have the sd of $Z$, so I approximated it by the mean of the sds of $X$ and $Y$) as measured in your sample of $n=45$ are representative of the population. If you don't want to make those assumptions, you can also reason in terms of Cohen's $d$ and what would constitute a large enough effect to care about, that you would want to be able to detect.

• n = 45, what is the standard for a tiny n? – G_z Dec 5 '17 at 9:18
• Not that for a single t-test, more like 10. Try the built in t-test function just to be sure there isn't a mistake in your code. – David Ernst Dec 5 '17 at 9:19
• I tried t.test() and i got the same results. – G_z Dec 5 '17 at 9:23