0
$\begingroup$

Skip to the conclusion if TL;DR

What is an intuitive explanation for what a t-score gives you when computing confidence intervals?

Background of my understanding of confidence intervals

I just learned about t scores from AP Stats on Khan Academy, and this is my understanding of confidence intervals, z-scores, and t-scores. Correct me if I'm wrong.

The ideal confidence interval for a sample mean with a normal sampling distribution, population mean $\mu$, population STD $\sigma$ and confidence p% is

$$\bar{x}\pm z^*\frac{\sigma}{\sqrt{n}}$$

because any sample $\bar{x}$ with a z-score in the interval (-z*, z*) will contain the population mean within its confidence interval. It is then easy to choose z*, based on what confidence percentage is needed. For example, if we want a 95% confidence interval, we choose z* = 1.96, because 95% of samples from a normal distribution have a z-score in the interval (-1.96, 1.96).

However, we often don't know the population standard deviation $\sigma$ so we estimate it with the sample standard deviation s.

$$\bar{x}\pm z^*\frac{s}{\sqrt{n}}$$

Since s $\neq \sigma$, this confidence interval usually underestimates the needed interval, as s is usually less than $\sigma$. So, we introduce a t-score to account for the inaccuracy.

$$\bar{x}\pm t^*\frac{s}{\sqrt{n}}$$

My thought was that we use a t-score to remove the error introduced by using s as an approximation for $\sigma$. Khan Academy lists the following requirements for using a t-score confidence interval:

  1. random sampling
  2. a normal sampling distribution (np $\geq$ 10 & n(1-p) $\geq$ 10).
  3. independence of samples

Testing my idea of what a t-score does

So, I wanted to test the theory that a t-score removes error introduced by using s as an approximation for $\sigma$ under those circumstances. I considered a sample mean of n Bernoulli trials with probability p = 0.5. These trials are 1. random and 3. independent. If we make n large, say n=1000, then the sampling distribution can be considered normal. Instead of writing $\bar{x}$, I will write $\hat{p}$ for the mean/proportion.

Then, what constant c will make our interval have 95% confidence?

$$\hat{p}\pm c\frac{s}{\sqrt{n}}$$

If all samples means with z-scores in the interval (-1.96, 1.96) have

$marginoferror > |\hat{p}-\mu| = |\frac{\hat{p}-\mu}{\sigma/\sqrt{n}}|\frac{\sigma}{\sqrt{n}} = |zscore| \frac{\sigma}{\sqrt{n}}$,

then we can be 95% confident, because all sample means with z-scores in (-1.96, 1.96) will contain the population mean. So the following equation should hold true whenever the sample mean z-score is in (-1.96, 1.96)

$$margin of error = c\frac{s}{\sqrt{n}} > |zscore|\frac{\sigma}{\sqrt{n}}$$

If we want to solve for c, we minimize the left side and maximize the right side. Since s = $\sqrt{\hat{p}(1-\hat{p})}$, as $\hat{p}$ moves away from 0.5, s gets smaller, minimizing the left. As $\hat{p}$ moves away from 0.5, the z-score gets larger, maximizing the right. So, we only need to worry about the largest and smallest values of $\hat{p}$ with a z-score still in the interval (-1.96, 1.96). That would be z-score = -1.96 or 1.96, or when $\hat{p}$ = $0.5 \pm 1.96\frac{\sigma}{\sqrt{n}} = 0.5 \pm 1.96 \sqrt{\frac{0.5(1-0.5)}{1000}} = 0.53099$ (just arbitrarily focusing on the larger value)

We can solve for a constant c when $\hat{p}$ = 0.53099.

$$c\frac{s}{\sqrt{n}} > |zscore|\frac{\sigma}{\sqrt{n}}$$ $$c\sqrt{\frac{\hat{p}(1-\hat{p})}n} > |zscore|\sqrt{\frac{p(1-p)}n}$$ $$c\sqrt{\frac{0.53099(1-0.53099)}{1000}} = 1.96\sqrt{\frac{0.5(1-0.5)}{1000}}$$ $$c=1.9638$$

Notice that when $\hat{p}$ = 0.53099, the confidence interval is

$$\hat{p}\pm c\frac{s}{\sqrt{n}}$$ $$= 0.53099 \pm 1.9368\sqrt{\frac{0.53099(1-0.53099)}{1000}}$$

You can see that it just barely includes 0.5 -- if you subtract and didn't round, you get exactly 0.500. Since 0.53099 has a z-score of 1.96, this interval should have a 95% confidence.

Now let's look at the z-scores and t-scores.

Conclusion if you want to jump straight to the question

To construct a confidence interval, we choose a value c for

$$\hat{p}\pm c\frac{s}{\sqrt{n}}$$

If we want a 95% confidence interval for the mean of n=1000 bernoulli trials with p = 0.5,

  1. Using a z-table, c = 1.96.

  2. Using a t-table with 1000-1=999 degrees of freedom, we get 1.962.

  3. Our calculated exact value for c = 1.9638.

The t-score is a better approximation than the z-score, but it is only about 50% closer to the true value than the z-score.

This seems too far away from the exact value to be consistent with the idea that a t-score should remove all error introduced by using s as an approximation for $\sigma$ given

  1. random samples

  2. normal sampling distribution

  3. independent sampling.

It's possible that the error is because n=1000, which isn't exactly normal, but it should be close enough, right? I read on Wikipedia that the t-score assumes that individual samples themselves should have a normal distribution, so is Khan Academy wrong about the conditions necessary? If the individual samples do need a normal distribution, it doesn't seem very handy since many random variables do not have a normal distribution -- the central limit theorem only says that the average of random variables tends toward a normal distribution.

If the t-sample doesn't remove the error, what does it do? Or what is it supposed to give?

$\endgroup$
  • $\begingroup$ Why are people voting this question down? (was -2 prior to this comment) This is one of the more researched and genuine questions on stats.SE. $\endgroup$ – SecretAgentMan Aug 24 '18 at 4:08
  • $\begingroup$ You would enjoy reading "Student's" original (1908) paper. It is republished in Kotz & Johnson, Breakthroughs in Statistics and can be found online in places like JSTOR. $\endgroup$ – whuber Aug 24 '18 at 12:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.