# One sample t test or one sample z-test

I have a question as to whether the one-sample t test or the one-sample z test is appropriate in a sample of n=17 BUT with a known population variance.

I would usually use a one-sample t-test for such small samples. However, since I have a reference variance from another independent sample I was wondering if maybe the one-sample z-test would be appropriate.

I have conducted both and have obtained some significant effects. I thought that the tests would be underpowered given the small sample size. How should I interpret the results now? Are the results still taken to be seriously?

Thank you everyone!

• It sounds like you're using the variance from another sample when calculating your test statistic. That's not "a known population variance" it's a sample variance (albeit from a different sample). Can you clarify your question to be consistent with whatever the actual situation is? (to either be talking about a known population variance or about a sample variance from another sample) Dec 5 '19 at 12:56
• Our statistics professor has always been telling us that if we have a variance from another independent sample we can treat it as a known "population" variance. That's why I was talking about the population variance. Dec 5 '19 at 17:03
• Do you get different results with t-test and z-test? 17 is not a such a small sample. Dec 5 '19 at 18:07
• @Milkovena Interesting. Outside of the circumstances where it would be reasonable to treat an insample standard deviation as the population s.d. (i.e. very large sample size), your professor is wrong. I'll come back and write up an answer explaining how to correctly use a sample variance from a different sample (given suitable assumptions) Dec 6 '19 at 1:48

Imagine we have the following situation:

A previous sample of size $$m$$, from which we have an estimate $$s^2_m$$ of its population variance $$\sigma^2$$. (Here the $$m$$ subscript is simply denoting the size of the first sample, rather than the denominator in the variance calculation. It's just there to remind us that it's not calculated on the current sample.)

A current sample of size $$n$$ from which we have an estimate $$\bar{y}$$ of its population mean $$\mu$$. We assume here that the variance of the population from which the current sample was drawn is also $$\sigma^2$$.

We assume both populations are normally distributed. We assume independence within and between groups.

For a one sample test of $$H_0$$: $$\mu = \mu_0$$ (against one or two sided alternatives in the usual fashion), we can form the statistic $$t=\frac{\bar{y}-\mu_0}{s_m/\sqrt{n}}$$. This will have a t-distribution with $$m-1$$ d.f. (NB $$n$$ doesn't come into the df at all)

For a two sample test of equality of the population means of the two samples described above, we can form a test statistic whose numerator is the difference in their sample means and whose denominator is $$s_m \sqrt{\frac{1}{m}+\frac{1}{n}}$$, and again the d.f. is $$m-1$$.

This result is relatively straightforward mathematically, but we can check it via simulation. Here's an example done in R. Assume $$m=10$$ and $$n=5$$ (I choose small samples so that we can easily distinguish the distributions). Let's demonstrate that the null distribution of the statistic for the first case above is consistent with the claim that the distribution is $$t_9$$ (i.e. $$t$$ with $$m-1$$ df). We simulate the t-test statistic and compare the distribution of under the two claims (your professor's and mine). However, it can be hard to visually distinguish a t and a normal distribution. Consequently, I will convert the distribution of the t-statistic into the corresponding p-values under the assumption that the statistic has a normal distribution and the assumption it has a $$t$$ distribution with 9 df. If one of us is correct, the p-values from our claimed distribution should look uniformly distributed between 0 and 1. I will also look at the achieved type I error rate (i.e. actual significance level) for a 5% two tailed test under both approaches.

Here's the R code to do that for the one-sample test:

nsim = 100000
sigma = 1 # without loss of generality but feel free to change things
m = 10
n = 5
mux = 20 # ditto
muy = 10 # ditto, demonstrating these don't have to be the same
tsim = replicate(nsim,{
x = rnorm(m,mux,sigma)
sm = sd(x)
y = rnorm(n,muy,sigma)
ybar = mean(y)
(ybar-muy)/(sm/sqrt(n))
})
opar=par()
par(mfrow=c(1,2))
hist(pt(tsim,9),n=100)   # transforming test stat. to p-values if it's t9
hist(pnorm(tsim),n=100)  # ditto, if it's normal
par(opar)


Which gives:

As you see, my p-values (on the left) are plausibly close to a uniform while your professor's p-values (on the right) are definitely not. This doesn't prove it's t9, but it does demonstrate that it's not normal, and that it can't be very far from a t9. My claim remains plausible.

Let's seem how much this would affect the significance level of a two-tailed t-test that we wish to be at the 5% level:

mean(pt(tsim,9)<.025|pt(tsim,9)>.9755)  # my claim
[1] 0.05011  # about 5%; within sampling variation

[1] 0.08228  # 8.2% - this is quite far out


So you should do the t-test but use as the df that for the sample standard deviation you used in the denominator (i.e. one less than the sample size in the original sample your variance estimate came from).

We could do a simulation for the two sample case in similar fashion; again it's $$m-1$$ df. You'd need to make the population means the same, calculate both sample means and then set the statistic to
(xbar-ybar)/(sm*sqrt(1/m+1/n)).

So the problem with doing a z-test instead of a t-test in this situation is that your significance level is inflated (just as it is if you did a z-test instead of a t-test with the s.d. calculated from the current sample). This lifts both power and the type I error rate (potentially a lot, as we saw in the simulation).

Naturally, if $$m$$ is quite large, you can get a reasonable approximation from treating the t-statistic as normal, but there's no reason to avoid doing the t-test if the assumptions are reasonable (why approximate what is easily done exactly?)

• Thank you so much! :) Dec 16 '19 at 5:52
• What to do when someone answers your question -- in particular note that if you follow the other suggestions, there's no need to say thank you Dec 16 '19 at 6:19