What would be the true population rate given the sample?

Question

I have say 100 different schools. In this data, for each school, I have a sample number of students who pass an exam and the percentage of students who pass an exam. This is given for 2 years. Would the true population be calculated using confidence interval wrt to sample size of each school or a single sample size which is the sum total of number of samples in all 100 schools? This is for comparing school performance using hypothesis testing and I cant figure this particular part out.

Description: In each school, students are sampled(not equally in each school), and the pass percentage is calculated. Now the null hypothesis is that the true pass rate of school 1 and school 2 are equal. The alternate hypothesis is that the pass rate at school 1 is higher than school 2. Using only the sample size of each school and the sample pass rate, is it possible to reject the null hypothesis? How will the true pass rate be calculated? wrt to the school itself or all the school, since factors such as location, student family background etc may be influencing factors which are different in each school.

Answer 1

There are several ways to tackle your problem. I'd use a Bayesian approach. Assume the data is binomially distributed i.e. the population parameter $\theta$ is the probability that a given student will pass the test. Your goal is to estimate the distribution of $\theta$. For binomial distributions, the conjugate prior for $\theta$ would be the $\text{Beta}$ distribution. When you have no data at all, it is reasonable to assume that $\theta \sim \text{Beta}(1,1)$. This is a non-informative prior (in other words, you make no assumptions on the parameter $\theta$ apart from the fact that it lies in the interval $[0,1]$). Suppose you have 100 schools, each school $i$ has $n_i$ students (you know the value of $n_i$ since you have the percentage of students who passed and the number of students who passed). Let $y_i$ be the number of students who passed in school $i$. Then the posterior distribution is $$(\theta \vert y,n) \sim \text{Beta}\left(1 + \sum_{j=1}^{100}y_i, 1 + \sum_{j=1}^{100}(n_i - y_i) \right)$$

Now if you have this posterior distribution, you can measure the credible interval of $\theta$ using properties of the $\text{Beta}$ distribution. Now consider school $i$, where the pass rate is $y_i/n_i$. Look at where this value lies on the posterior distribution of $\theta$. This will give you a good idea of how this school compares to the others. In addition, if you get more data, you can just update the posterior distribution using the formula given above.

Answer 2

You already got nice answer about point estimation and comparing your schools from Bayesian point of view, but your question:

Using only the sample size of each school and the sample pass rate, is it possible to reject the null hypothesis?

Can be given simple and straightforward answer from frequentist point of view. Given two proportions $p_1,p_2$ and sample sizes $n_1,n_2$ you can use $z$-test for two proportion to test your hypothesis.

First you need to compute the "aggregate" proportion $\bar p$

$$ \bar p = \frac{p_1 n_1 + p_2 n_2}{ n_1 + n_2} $$

and then the test statistic is

$$ z = \frac{p_1 - p_2}{\sqrt{\bar p (1-\bar p)(\frac{1}{n_1} + \frac{1}{n_2})}} $$

where $z$ follows normal distribution (check here, or here for worked examples).

As with Bayesian approach, in frequentist case you also can compute confidence intervals around the individual proportions so to compare them. There is a number of methods for that, for review you can check Wikipedia article and paper Interval Estimation for a Binomial Proportion by Brown, Cai and DasGupta (2001, Statistical Science, 16(2), pp. 101-117).