# Should the Wilcoxon Rank sum test be used for testing the mean difference significance?

I was looking replicate the results of the paper DOI:10.3905/jpm.2014.40.3.087 (Exploring Macroeconomic Sensitivities: How Investments Respond to Different Economic Environments, Ilmanen Maloney Ross 2014 Journal of Portfolio Management).

For example, I was looking to investigate if the monthly total return of US Treasury bond index (LUATTRUU Index - Bloomberg Barclays US treasury total return index) depends on the magnitude of the US inflation readings (CPI YOY Index - US CPI Urban Consumers YoY NSA »). So i downloaded the US CPI YOY monthly data for the last 50 years and the monthly total returns of US Treasury index for the same period. Then, I've partitioned the sample in "UP" if the monthly US inflation readings is above its last 12 monthly average, and "DOWN" if the opposite happened. Then, i computed the average monthly total return of US Treasury bond index in each subsample. I got the following results.

1. US CPI "UP" -> monthly avarage US Treasury bond index total return 0.40%
2. US CPI "DOWN" -> monthly avarage US Treasury bond index total return 0.65%

So, the data seems to confirm that when inflation is high, bond return are low compared to the opposite situation, as the economic theory would suggest.

I would like to test statistically if the difference in the two return is statistically significant. However, since i'm using financial time series, I'm not sure which statistical test should I use.

Here my R code, that takes as input the time series of US CPI YOY and US TREASURY TOTAL RETURN INDEX:

CPI_YOY_roll<- rollmean(CPI_YOY,k=12, fill=NA, align="right")
a<-ifelse(CPI_YOY[13:nrow(ds)]>=CPI_YOY_roll[12(nrow(ds)-1)],"up","down")
mean(na.omit(ifelse(a=="up",ds$$treasury_return,NA)))*100 mean(na.omit(ifelse(a=="down",ds$$treasury_return,NA)))*100


For example, here Ilmanen Maloney Ross (2014, ExploringMacroeconomic Sensitivities: How Investments Respond to Different Economic Environments, Journal of Portfolio Management) use my same procedure and get the following results (using average Sharpe ratio instead of average return, but the concept is the same):

So i want to test if these differences are statistically significant or not!

• Have you considered something like a time series correlation? stats.stackexchange.com/a/133171/134438 You could also take the month-by-month change in each time series and take a correlation of those two (another answer to same question) stats.stackexchange.com/a/156352/134438 Commented Dec 8, 2023 at 18:21
• Thanks for the comment. Basically i need a statistical test (like t-test) in order to verify if the mean difference is statistically different from 0. I don’t think the correlation is ok for my problem Commented Dec 9, 2023 at 6:50
• You can test whether a correlation is statistically significant than 0. Just that instead of binning the inflation readings into "UP" and "DOWN" you'd use the change in inflation reading from one month to the next. And then do the correlation of that with the index returns. And you'd test whether that correlation in statistically significant. Commented Dec 9, 2023 at 17:25
• “I would like to test statistically if the difference in the two return is statistically significant.” The problem here is that time series tend to correlate a lot because of autocorrelation. The difficulty in defining statistical significance is that the model to be applied is not very clear. One approach might be to use Monte Carlo simulations of time series with similar autocorrelation to obtain a potential distribution of correlations between such time series. But, if your model of the autocorrelation is not representing the series right (e.g. missing a periodic term) then it can fail. Commented Dec 11, 2023 at 8:36
• I just took a look at the curve for US CPI and it looks like a very irregular behaving curve with unstable periods of decline and increase that are not easily explained by a simple random model and are more likely because of complex random events in the world. So, defining a statistical model for it, based upon which we can compute some expression for 'statistical significance' seems unobtainable. One alternative approach might be to downgrade your data. Cut the 50 years into 10 pieces of 5 years, compute correlations in all 10 of them, and apply a test to those 10 correlations. Commented Dec 11, 2023 at 8:47

#### Consider using a regression model with a test of slope parameter instead

While a T-test should serve adequately for testing for a mean difference between two groups, I would counsel against the method you are using here. In particular, there is a substantial loss of information caused by converting the inflation rate into a simple up/down binary. A secondary problem with this method is that your up/down conversion is taken relative to the previous 12-month average, which then induces a dependency in this binary variable on the previous inflation rates. The resulting comparison does not appear to me to capture your initial research hypothesis of interest.

If you want to know how the monthly total return of US Treasury bond index depends on the magnitude of the US inflation readings, I recommend modelling these variables directly against each other without categorising them. This analysis can be done with a regression model and estimation of the relationship can be done using a standard test on the slope of the regression line.

Your question does not specify which particular measures you have downloaded as your data, so I will show an example of the analysis using some data I have chosen instead. In the example below I have measured CPI with the CPALTT01USM661S (Consumer Price Index: All Items: Total for United States, Index 2015=100, Monthly, Seasonally Adjusted) and I have measured treasury yield with DGS10 (Market Yield on U.S. Treasury Securities at 10-Year Constant Maturity, Quoted on an Investment Basis, Percent, Monthly, Not Seasonally Adjusted), with both variables available on a monthly basis from the start of 1962 up to late 2023 (741 months available). I have converted the measurements to "force of interest" using logarithmic changes and used a simple linear regression model to relate these variables.

#Load the data
DATA$$BondRate <- log(1 + DATA$$BondYield/100)
DATA$$Inflation <- 0 DATA$$Inflation[2:n] <- 12*diff(log(DATA$CPI)) DATA <- DATA[-1, ] n <- nrow(DATA) #Model the data MODEL <- lm(BondRate ~ Inflation, data = DATA) #Plot the data plot(100*DATA$$Inflation, 100*DATA$$BondRate, xlab = 'Inflation Rate (%)', ylab = '3-Month T-Bill Rate (%)') abline(a = 100*MODEL$$coefficients[1], b = MODEL$$coefficients[2], lty = 3)  We can test whether there is a relationship between the variables by looking at the estimated slope coefficient in the model. In the summary below we see that there is strong evidence of a non-zero slope coefficient, which means that there is strong evidence of a relationship between the variables. #Show model summary summary(MODEL) Call: lm(formula = BondRate ~ Inflation, data = DATA) Residuals: Min 1Q Median 3Q Max -0.062104 -0.016747 -0.000591 0.013270 0.086906 Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 0.045700 0.001336 34.21 <2e-16 *** Inflation 0.292355 0.024964 11.71 <2e-16 *** --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 Residual standard error: 0.02577 on 738 degrees of freedom Multiple R-squared: 0.1567, Adjusted R-squared: 0.1556 F-statistic: 137.2 on 1 and 738 DF, p-value: < 2.2e-16  • Thanks for your answer, however i would like to replicate the result of this paper: DOI:10.3905/jpm.2014.40.3.087 (Exploring Macroeconomic Sensitivities: How Investments Respond to Different Economic Environments, Ilmanen Maloney Ross 2014 Journal of Portfolio Management), so instead of linear regression I should follow the procedure I described! Commented Dec 11, 2023 at 10:16 • Regarding s.e.s,$t$-values and$p\$-values, should we perhaps worry about possible autocorrelation in the model's residuals and account for it (e.g. using auto.arima on BondRate with Inflaction as an exogenous regressor)? Commented Dec 12, 2023 at 20:59
There are a variety of complicated methods you could use to test this, but I think one that balances simplicity, with accounting for the autocorrelation in financial time series, would be to use generalized least squares. Here you would have a single explanatory variable which is 0 or 1 based on if you are in the US CPI "UP" or US CPI "DOWN" category, and your response variable would be your returns (although you may want to use log returns). You would fit your GLS model and then see if the $$\beta$$ associated with your binary variable is statistically significant.