1
$\begingroup$

I have come across two major problems recently, and couldn't solve them. Imagine we have measured an independent variable 2000 times and I'm interested to talk about the mean of the population: So as some textbooks suggested, I can perform some calculations like this:

Variance = Sum of squared/(2000-1)

then standard deviation =sqrt(variance)

Standard error(SE) = standard deviation / sqrt(2000)

mean value - SE and mean value + SE (for approximately 68.2% confidence interval)

But the first problem here is that why we don't put these 2000 measurements into several samples, then calculate the sampling distribution and instead of using sqrt(2000) in the SE denominator, using sqrt(Number of samples) then calculate the mean plus-minus the SE.

1- Which method is better? Is one of these methods wrong?

Second Problem:

Apparently, when the sample size gets bigger, the interpretation of some tests like Shapiro-Wilk, Levene's test should be made cautiously (Also the significant results regarding the p-values). On the other hand, if the sample size is too small, the normality of data(or sampling distribution) won't be valid and some other issues. I found out that for example in psychology departments the sample size less than 30 considered small and greater than 200 will be considered very big, but in other fields it's not the case.

2- How can I be sure about the sample size which is not too big (or small)? Passing the normality test is enough to conclude that the sample size is not small?

Also, I can't partition my population into subpopulations at all. So having a large sample should be the same as having multiple samples with less sample size (approximately)?

Thank you so much in advance.

$\endgroup$
2
  • 1
    $\begingroup$ How exactly would you "calculate sampling distribution"? $\endgroup$ – Tim May 25 '20 at 17:18
  • $\begingroup$ Actually, I'm not interested in the calculation of sampling distribution. I'm only interested in the true mean of the population by estimating the mean of my samples +/- standard error. Due to the heavy computations of getting more samples, I use bootstrapping samples from my 2000 data to generate more samples then check the normality of my sampling distributions. It takes more than 3 months for only 2000 data. $\endgroup$ – Mohammadreza Niknam Hamidabad May 26 '20 at 4:29
1
$\begingroup$

This site works best with one question at a time. You have asked several. I will try to answer the ones that might illustrate general principles.

(1) If your goal is to estimate the population mean $\mu$ from which a random sample of size $n = 60$ is available, then the best estimate comes from analyzing the undivided sample.

Suppose you split the sample into four samples of size $15$ each. Then in order to find four variances, you need to find four sample means. You can combine the four $S_1^2. S_2^2. S_3^2, S_4^2$ to get a pooled estimate of the variance. Each of the $S_i^2$ has $\nu_i = 15-1 = 14$ degrees of freedom. Thus, the pooled variance estimate $S_p^2 = S_w^2$ has $\nu = 4(14) = 56$ degrees of freedom. In effect, this method is used to estimate the variance in a one-factor ANOVA with four levels of the factor. The notation $S_w^2$ refers to the variability within the four groups. In an ANOVA, nothing is wasted because you need the four group means $\bar X_i$ for other purposes.

However, if you estimate the population variance from the entire sample of $n = 60$ observations you will get $S^2,$ which has $\nu = 60 - 1 = 59$ degrees of freedom.

Estimating population variance $\sigma^2,$ you will tend to get a closer estimate from $S^2$ than from $S_w^2$ and a confidence interval for $\sigma^2$ will tend to be shorter using $S^2.$

Below is an illustration in R using one sample of 60 divided into four samples of 16, in a situation where $\sigma^2 = 15^2 = 225.$

I start by making the four samples separately because I think the program is less complicated that way. The two estimates are $S^2 = 245.91$ and $S_w^2 = 254.18.$

set.seed(2020)
x1 = rnorm(15, 100, 15)
x2 = rnorm(15, 100, 15)
x3 = rnorm(15, 100, 15)
x4 = rnorm(15, 100, 15)
v1 = var(x1);  v2 = var(x2)
v3 = var(x3);  v4 = var(x4)
v.w = 14*(v1+v2+v3+v4)/(14*4); v.w
[1] 254.1758

x = c(x1,x2,x3,x4)
var(x)
[1] 245.9138

The confidence interval based on $S_w^2$ is $(181.17, 382.51)$ of length $201.34.$ The confidence interval based on $S^2$ is $(182.62, 378.11)$ of length $195.48.$

ci.w =56*v.w/qchisq(c(.975,.025),56); ci.w
[1] 181.1678 382.5110
diff(ci.w)
[1] 201.3431

ci.c =59*v.w/qchisq(c(.975,.025),59); ci.c
[1] 182.6211 378.1056
diff(ci.c)
[1] 195.4845

(2) As for tests to check assumptions (normality and equal variances), the truth is what matters and the truth will not be changed by chopping your sample into pieces.

Psychologists (and others) offering opinions about sample sizes should try to remember that a small sample carefully collected to be random may give more reliable results than a much larger sample less carefully drawn.

Also, it is best to choose sample sizes (perhaps with the help of a 'power and sample size' program) so that one will use whatever sample size is adequate to give a reasonable chance to detect an effect that is considered to be of practical importance.

  • Samples too small may result in failure to reject, thus not detecting an effect of interest.

  • Samples too large may detect effects too small to matter, thus leading to ever more journal articles giving impressively small P-values for effects too small to be of any practical importance.

$\endgroup$
1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.