Can a confidence interval algorithm simply alternate between a small and a huge interval?

Question

A confidence interval is basically an interval generated by an algorithm that takes the observed data as input and gives an interval as output so that 95% of the time the interval contains the true underlying parameter.

So basically it's not a property of the interval itself but of the generating algorithm.

But we could easily create an algorithm that gives the interval (-infinity, +infinity) 95% of the time and [0,0] 5% of the time. Even if we have no access to external randomness, we could just inspect the input data and decide based on some carefully designed property that would be true 95% of the time (the property would depend on the assumed model for the distribution of the observed data).

This is obviously an extreme example, but doesn't it conform to the definition of an algorithm for generating confidence intervals? If we rule this out, then what is the requirement precisely?

For a practical example, in confidence-calibration games, one has to answer several questions such as "When was Einstein born?" with a 95% confidence interval, meaning that precisely 95% of your answers should contain the true solutions. I think the best method is to "cheat" and say huge, obviously correct intervals 95% of the time and say tiny and obviously wrong intervals 5% of the time.

How can we "repair" the definition of a confidence interval?

Answer 1

While you can form many intervals to try to capture a value with 95% confidence, the confidence interval is the one of these intervals with the minimum width. It seems like you are considering a sort of non-deterministic way to generate an interval. In this case, you could take the expected value of the width and then the standard, minimum width interval would still win out.