# Sample size determination and Confidence interval of a model

I am asked to determine the test sample size of some data in order to get a 95% confidence interval with width $$w$$ of a given loss function $$L$$. The formula in wikipedia is

$$n = (\frac{4\cdot1.96 \sigma_L}{w})^2.$$

However, this question is asked before building any model for the data, so I do not have any information about the standard deviation $$\sigma_L$$ of $$L$$. I tried to use the standard deviation of the trivial model which always predict the mean of the data, but the number is too big. I know that because after building the models I got to a better CI with much fewer data. So

• I want to know if the question itself makes sense.
• Any ideas about how to tackle it.

Thank you in advance!

The question makes sense. The estimate from the trivial model likely over estimates sigma, which will in turn mean your confidence interval will be narrower than you expect. No matter, the calculation will ensure the resulting CI will have width no larger than $$w$$.
EDIT: It might be worth examining where this equation comes from. If you write down the length of a confidence interval and solve for $$n$$, you obtain the formula you've provided. In order for this to be useful, be need an estimate of sigma.
In the case that no variables are predictive of the outcome, then sigma from the trivial model should be sigma for any model which does not over fit on the data (or at the very least, should be very very close to the sigma from any model which does not over fit). Thus, using sigma from the trivial model is a sort of "worst case scenario", and in the case where variables do predict the data then sigma will be smaller and so to $$n$$.
However, there is no way to know what the "right" sigma, and hence what $$n$$, to use to get exactly the desired precision prior to fitting a model. But fitting a model requires a train test split. We're in a catch 22. That is why we use the sigma from the trivial model.