UPDATE 06/2020: I just revisited this question and realised that there is a fairly clear cut answer. Specifically, the required condition is uniform integrability. Basically the class of functions of the distribution that are uniformly integrable can be in inferred using a bootstrap. Without uniform integrability, all bets are off. I'm fairly sure, although can't remember off the top of my head, that there is theorem in one of Brillinger's textbooks for exactly this situation.
Setup: Let $p_n(x) = \mathbb{P}(S_n \leq x)$ and let $p_n^*(x) = \mathbb{P}^*(S_n^* - S_n \leq x)$, where $S_n$ is some zero-mean test statistic that can validly be bootstrapped, eg a sample mean, and $S_n^*$ is a bootstrap re-sample of $S_n$. Note that $\mathbb{P}^*$ implies the probability is conditional on the underlying data. Assuming the conditions of some bootstrap theorem, eg stationary bootstrap, are fulfilled, we have the main bootstrap result:
\begin{equation} \sup_{x \in \mathbb{R}} | p_n(x) - p_n^*(x)| \overset{\mathbb{P}}{\rightarrow} 0, \: \textrm{as} \: n \rightarrow \infty \end{equation}
My question has two parts:
Question 1: The main bootstrap result is usually used to assert that bootstrapped confidence intervals are asymptotically valid. Can we make this assertion because of Slutsky's theorem? If so, why isn't continuity an issue?
Currently, my understanding is that the argument works as follows:
1) For some fixed point $x \in \mathbb{R}$, we use the above result to assert that $p_n^*(x) \overset{\mathbb{P}}{\rightarrow} p_n(x)$, as $n \rightarrow \infty$.
2)Use Slutsky's theorem to assert that $g(p_n^*(x)) \overset{\mathbb{P}}{\rightarrow} g(p_n(x))$, as $n \rightarrow \infty$, for some Borel function $g(a)$ that is continuous at $a$.
3) Choose $g(a)$ to be the quantile function, ie $g(F(x)) = \inf \{ x \in \mathbb{R} : \lambda \leq F(x) \}$ where $F(x)$ is a cdf and $\lambda \in (0, 1)$. The result then follows.
The problem with this argument as it stands is that, for general cdf functions, the quantile function is only guaranteed to be left continuous. I'm assuming that we get around this problem by exploiting the fact that $p_n(x)$ and $p_n^*(x)$ are both converging to the Normal, which has a continuous cdf, but I've never seen this stated formally anywhere, so I am wondering if my reasoning is actually correct.
Question 2: Having established that we can build asymptotically valid confidence intervals, I'm wondering what other characteristics of the distribution of $S_n$ we can consistently estimate?
I know that most bootstrap theorems also include a result of the form $\mathbb{V}^*S_n^* \overset{\mathbb{P}}{\rightarrow} \mathbb{V} S_n$, as $n \rightarrow \infty$, so that takes care of the variance. What about other moments? For example, can we infer from the main bootstrap result that $\mathbb{E}^* S_n^{*4} \overset{\mathbb{P}}{\rightarrow} \mathbb{E} S_n^4$, as $n \rightarrow \infty$? As in the case of confidence intervals, I tried to prove this using Slutsky's theorem, but was not comfortable with the resulting expression:
\begin{equation} \int x^4 dp_n^*(x) \overset{\mathbb{P}}{\rightarrow} \int x^4 dp_n(x), \: \mathrm{as} \: n \rightarrow \infty , \end{equation}
since it is not clear to me that we are not accumulating lots of "small" errors in the integral.
EDIT: I asked the question here since it is about the bootstrap. However, since it is also about probability theory, if users feel it would be more appropriate in stack exchange mathematics, please let me know.
