The most common malpractice in all of Trading: Back-testing
Given ANY in- or out-of-sample time series, including purely random, synthetic data, anyone can generate (inflate) ANY Sharpe Ratio by repeatedly applying different trading or investment strategies to the same time series sample!
By definition, purely random data has no discernible structure. Consequently, no method can exist to predict such a sequence - I.e., Sharpe Ratio = 0 must hold in all instances.
Yet, ... See main graph!
In the past It has been shown just how easy it is to generate Sharpe Ratios of 4, 5 or even >6, on any data, including on purely random, synthetic time series data when in fact, the only possible value in those instances should be S.R. = 0.
As a matter of fact, this misleading (self-defeatists?) practice is so common and wide spread in finance and trading that the American Statistical Association considers it "unethical" (American Statistical Association ). (More importantly, it is a remarkably expensive way to fool oneself.)
The above stems from applying the same rejection threshold for the null hypothesis under multiple testing will grossly underestimate the probability of obtaining a false positive.
Unlike in the "other sciences", there is no "replication crisis" in finance or trading, simply because such checks don't even exist there - since those would be impossible to carry out. (Is that why the only two kinds of academic papers which never get revised or retracted are written in the fields of Finance and Theology?)
The bottom line;
In the common case of testing a trading or investment system, given a set of out-of-sample time series, one MUST increase the rejection threshold for the null hypothesis in proportion to the number of times ("peeks") such tests are carried out! (Good luck fooling yourself that way!)
Anything less is just simple curve-fitting!
For more in-depth explorations:
Marcos López de Prado, Michael J. Lewis
codemacher.com/wp-co...and_M.Lewis_2018.pdf
By definition, purely random data has no discernible structure. Consequently, no method can exist to predict such a sequence - I.e., Sharpe Ratio = 0 must hold in all instances.
Yet, ... See main graph!
In the past It has been shown just how easy it is to generate Sharpe Ratios of 4, 5 or even >6, on any data, including on purely random, synthetic time series data when in fact, the only possible value in those instances should be S.R. = 0.
As a matter of fact, this misleading (self-defeatists?) practice is so common and wide spread in finance and trading that the American Statistical Association considers it "unethical" (American Statistical Association ). (More importantly, it is a remarkably expensive way to fool oneself.)
The above stems from applying the same rejection threshold for the null hypothesis under multiple testing will grossly underestimate the probability of obtaining a false positive.
Unlike in the "other sciences", there is no "replication crisis" in finance or trading, simply because such checks don't even exist there - since those would be impossible to carry out. (Is that why the only two kinds of academic papers which never get revised or retracted are written in the fields of Finance and Theology?)
The bottom line;
In the common case of testing a trading or investment system, given a set of out-of-sample time series, one MUST increase the rejection threshold for the null hypothesis in proportion to the number of times ("peeks") such tests are carried out! (Good luck fooling yourself that way!)
Anything less is just simple curve-fitting!
For more in-depth explorations:
Marcos López de Prado, Michael J. Lewis
codemacher.com/wp-co...and_M.Lewis_2018.pdf