Apologies for my naivety if the answer to the question is simple, stats is an area I am not comfortable in and am looking to improve.
My problem is within the frame of finance. Simply put, say I have a base strategy which gives a certain time series of returns when backtested. This is my baseline.
I then apply a signal as a hopeful improvement on this baseline return series. That results in a different return series.
There are many questions that could be asked at this point, and finance provides some basic metrics that are commonly used to evaluate the difference between these two time series (sharpe, max drawdown, return to drawdown, ect.). These do not utilize statistics though, so I do not know for sure if, say, a sharpe of 1.3 for the base strategy is different from a sharpe of 1.4 for the alternative strategy.
What I want to know is are these two time series different? Secondly, is one better? The question "what is better?" needs to be answered. For now let's just make it simple, and say that a strategy is better than the base if it's compounded return over the series is greater than the compounded return of the base strategy (it makes more money). Alternatively this could be a higher risk adjusted return, or any number of the other finance metrics.
This question makes sense to me outside of the time series space (so just comparing two samples, are the means different? Is one mean greater than another? simple), but I'm getting a bit hung up on the path dependency of the strategies. Then of course there are the assumptions (if any) on the underlying distribution of returns, a concept I've found is often thrown by the wayside in finance (just assume normal) for ease of computation.
If anyone has any insights into this, any literature that they may suggest reading, or that sort of thing it'd be much appreciated. I just need to be pointed in the right direction, my googling has not yielded great (or maybe more accurately, accessible) results.