Do you know of any studies or articles that shows why backtesting actually works or is useful? I have been reading up on automated and systematic trading, but I do not yet understand how backtesting gives one anything more than warm fuzzies.
For backtesting to work, wouldn't one need to re-run (i.e., playback and not simulate) all inputs from what the real data was (instrument pricing, the weather, social media sentiment, whatever) at a time resolution finer than what your algorithm operates on? If so, that's a massive amount of data that may even be impossible to get.
I'm not sure. I haven't found much on whether or not backtesting works in general. It definitely has its caveats. I think it really depends on what you're backtesting.
Backtesting definitely reveals ways in which trading algorithms can fail though. It can reveal more scenarios than one would initially consider which is super helpful.
Evidence-Based Technical Analysis is a starting point (book), but iirc it was able to show most common indicators aren't able to reject the null hypothesis.
For backtesting to work, wouldn't one need to re-run (i.e., playback and not simulate) all inputs from what the real data was (instrument pricing, the weather, social media sentiment, whatever) at a time resolution finer than what your algorithm operates on? If so, that's a massive amount of data that may even be impossible to get.