How to select the level of confidence when using confidence intervals? Saw the expression of "95% confidence interval is commonly used" in a lot of literature. I want to understand more about why 95% (or 99%) is a good level to use. Or is there a general methodology to determine what a good level to use is?
Commonly see the need for bootstrap confidence intervals from data with unknown (non-Gaussian) distributions. For various kinds of metrics (e.g. median, rank, specially customized measure). Is 95% a good starting point for all sorts of metrics? For some use cases, a more stringent 99.9999% confidence interval is agreed by the field. In other instances, people use a more relaxed level. I hope to get an understanding of the general theory or methodology to determine when to use what level.
 A: Richard McElreath puts it nicely in Chapter 3 of Statistical Rethinking, second edition:

Rethinking: Why 95%? ... This customary interval
also reflects the customary threshold for statistical significance,
which is 5% or p < 0.05. It is not easy to defend the choice of 95%
(5%), outside of pleas to convention. Ronald Fisher is sometimes
blamed for this choice, but his widely cited 1925 invocation of it was
not enthusiastic:
“The [number of standard deviations] for which $P=0.05$, or 1 in 20,
is 1.96 or nearly 2; it is convenient to take this point as a limit in
judging whether a deviation is to be considered significant or not.”
Most people don’t think of convenience as a serious criterion. Later
in his career, Fisher actively advised against always using the same
threshold for significance.
So what are you supposed to do then? ... For example, why not present
67%, 89%, and 97% intervals, along with the median? Why these values?
No reason. They are prime numbers, which makes them easy to remember.

A: There is no de-constextualised confidence level that is "better" than others.  The point of a confidence interval procedure is that it should be valid for any chosen confidence level from 0% to 100%, and the interval gives an inference at whatever confidence level is desired.  In most contexts we want to use a confidence level that is high enough that the parameter of interest will usually fall within the interval.  Conventionally that has led people to use confidence levels of 90% or higher.  The conventional use of confidence levels 90%, 95%, 98%, 99%, etc., it based solely on an appeal to using "simple" numbers that are easy for users to understand.
Where a particular choice of a confidence level for a confidence interval (or significance level in a hypothesis test) becomes good or bad is when we have an actual decision problem with a decision to make, and there are negative consequences for an incorrect decision.  In such cases we form a confidence level (or hypothesis test) by considering the relative size of the negative consequences that accrue to wrong decisions of different kinds.  As a general principle, if the consequences of having the parameter fall outside the interval are relatively bad then we would use a higher confidence level; if the consequence of having the parameter fall outside the interval are relatively minor then we would use a lower confidence level.
In order to avoid bias in the procedure, one thing that is important here is that the choice of confidence level (or significance level) should not depend on the content of the data.  The confidence level should be formed by reference to exogenous factors relating to the relative consequences of inferential errors (in the contextual case) or based on conventional values that might later be useful (in the non-contextual case).  The sample size should not affect the choice of confidence level, but it does affect the power of the inference, and attention should be paid to the latter to give a holistic understanding of the inference.
