I have about 2000 daily observations of historical share prices for a handful of companies. I use a rolling window of 1000 observations to model their joint distribution. From each of the windows, I predict the joint distribution of the share prices one day ahead. Based on the prediction, I optimize a portfolio targeting a certain combination of risk and reward. (By optimize a portfolio I mean I find a set of nonnegative weights that sum up to 1. I use numerical optimization for that.) I collect the realized returns on the portfolio over 1000 windows. I also collect the realized returns on an equally-weighted (EW) portfolio over the same period.
I would now like to assess whether the optimally-weighted (OW) portfolio is doing any better than the EW one for a specific measure of risk, namely, the 2.5% expected shortfall1. I can do a naïve comparison by looking at the mean of the 25 lowest returns from each portfolio. I find that the mean of the OW portfolio is higher than the mean of the EW portfolio, just as one would expect. However, I am not sure whether the difference is large enough to be statistically significant. How can I test a null hypothesis that both ways of constructing portfolios from the given set of shares yield the same expected shortfall (against an alternative that the OW way is better)?
I know how I would test a hypothesis that the expected return on one asset is equal to the expected return on another asset. I would assume the returns are i.i.d. and do a $t$-test. Or I would assume the difference in returns follows an ARMA-GARCH process and look at the significance of the intercept in that model. However, I am afraid the problem above is more complicated. I see the following challenges: (1) we are dealing with the tail of the distribution rather than the mean; (2) the weights of the OW portfolio are not fixed but have been estimated; (3) the weights have been estimated from rolling windows that are overlapping. I am rather lost.
If the problem is too hard as is, I would appreciate some insight on its special cases, e.g. ignoring the point (3). For the most basic version of the problem, see this question.
1 $q$% expected shortfall is (the negative of) the expected value of the observations belonging to the left tail that is cut off at the $q$% quantile level. Synonyms of expected shortfall are conditional value at risk (CVaR), average value at risk (AVaR), expected tail loss (ETL), and superquantile.
