Confidence interval for samples from an empirical distribution I have data where a number of respondents are asked to rate the characteristic of a set of subjects on a scale of 1 to 100. A different number of respondents were asked to rate every subject, so we have differing sample sizes. While the actual dataset has much more observations, an example dataset is:
\begin{array}{|c|c|}
\hline
\textbf{Subject ID} & \textbf{Score} \\ \hline
1                   & 72              \\ \hline
1                   & 62              \\ \hline
2                   & 100             \\ \hline
2                   & 55              \\ \hline
2                   & 100             \\ \hline
2                   & 82              \\ \hline
2                   & 72              \\ \hline
3                   & 75              \\ \hline
3                   & 83              \\ \hline
3                   & 100             \\ \hline
\end{array}
Now what I would like to do is construct confidence intervals for the mean scores of the different subjects, preferably based on the empirical distribution found in the entire sample. We can see that $\bar{Y}_1=(72+62)/2=67$, but since sample sizes are different it would be dishonest to only compare means between subjects, confidence intervals would convey more information. What accurate ways are there to construct confidence intervals in a case like this? The standard assumption of underlying normality does not hold, among other things there are large peaks at 100, 75 and 50 in the histogram for 'Score'.
 A: If I understand it correctly, I see a couple of practical difficulties in this situation: 
(a) Apparently, some of your 'respondents' do not understand the rating system or are refusing to cooperate with it. The presence of many scores of 25, 50, 75, 100 shows that some of them would be more comfortable with a 4-point scale
than with a 100-point scale. If the score is supposed to represent something like overall performance, then you might ask respondents to rate subjects on four or five 20 or 25-point sub-scales, and then give the sum of those as the 'score'.
(b) Because you have different respondents and even different numbers of respondents rating each subject, it is not clear that ratings for one subject are comparable with ratings for another. So, depending on how respondents are chosen or volunteer, there may be a difficulty of using this system to compare subjects.
Nonparametric bootstrap CI. However, your precise question seems to be how to find a confidence interval for the population mean $\mu,$ given a set of ratings from a population that is not  normal--and does not have any other known distribution. You can do that with a 'nonparametric bootstrap' procedure.
Rationale for bootstrap. Suppose you have $n = 100$ scores from an unknown population with mean $\mu$ and that the observed sample mean is $\bar X = \bar X_{obs}.$
In order to get a confidence interval (CI) you need to have some idea how the population mean scores $\bar X$ vary about the population mean $\mu.$ Let's call
that random discrepancy $D = \bar X - \mu.$
If you knew something specific about the population you might be able
to find values $L$ and $U$ such that
$$ P(L < D < U) = P(L < \bar X - \mu <U) = P(\bar X - U < \mu < \bar X - L) = 0.95.$$
Then a 95% CI for $\mu$ would be of the form $(\bar X - U, \bar X - U).$
"Bootstrap world." Because we do not know $L$ and $U,$ we enter the bootstrap world to approximate these values. We take a large number $B$ of 're-samples' with replacement of size $n$ from the sample of $n$ observations. We find the sample mean $\bar X_j^*$ for the $j$th re-sample. Because our best guess at $\mu$ is $\bar X_{obs}$ we temporarily, use the notation $\mu^* = \bar X_{obs}.$ Then for each
of the $B$ re-samples, we find $D_j^* = \bar X_i^* - \mu^*.$ 
Then we find quantiles .025 and .975 of $D_1^*, D_2^*,D_3^*, \dots D_B^*,$
calling the quantiles $L^*$ and $U^*,$ respectively.
Bootstrap CI. Then, back in the real world, we use $(\bar X_{obs} - U^*, \bar X_{obs} - L^*)$
as a 95% nonparametric bootstrap CI for $\mu.$
Data for example. For an example, I simulated the suitably 'lumpy' sample of $n = 100$ scores shown in the histogram below. Tick marks show positions of exact scores, where there are a few ties (especially around values divisible by 25).

R-code for bootstrap CI. The R code below shows how to make a 95% nonparamatric bootstrap CI for $\mu;$ the data are in the vector x, its mean is a.obs (67.96) and the suffixes -re are used instead of $*$'s to represent re-sampled quantities.
set.seed(924)
B = 100000;  n = length(x);  a.obs = mean(x)
d.re = replicate( B, mean(sample(x,n,rep=T)) - a.obs )
UL.re = quantile(d.re, c(.975, .025))
a.obs - UL.re
97.5%  2.5% 
63.09 72.98 

Thus a 95% CI for the population mean $\mu$ used to simulate the data is
$(63.09, 72.98).$ The fundamental assumptions of a nonparametric bootstrap CI
are that the data are a random sample from a population that has mean $\mu.$
Confidence intervals based on the t distribution for non-normal samples this large are known to give useful results, provided skewness is not extreme and there are no extreme outliers. For the 100 observations in x, the t 95% CI $(62.91, 73.01)$ is not hugely different. Of course, assuming that the data are normal "adds information" so we can't expect the two CIs to be exactly equal.
Note: The data in x were simulated according to the R program below; the
population mean is $\mu = 68.75,$ which is covered by the bootstrap CI computed above.
set.seed(1018)
x1 = 100*rbeta(75, 2, 1)
x2=c(rep(25,2), rep(50,6), rep(75,7), rep(100,10))
x3 = x2 + runif(25, -2, 0)
x = c(x1, x3); x=round(x)

