How to select effective indicators for your strategy
Not all indicators are useful: most are not, and some are downright misleading. Previous posts and studies, such as LuxAlgo's(1), determined that effective indicators need to: 1) produce data to support the trader's decision-making process, not substitute it with automated strategies, 2) produce non-redundant infos. But how do you select indicators in practice? Here, I share my own step-by-step process to select effective indicators for your strategy.
My approach is to use a two-stages process: 1) Expansion, 2) Contraction.
This is the same process that happens in our brains when they develop, first there is neuronal and synaptic expansion, creating lots of new connections that are not necessarily efficient, then there is contraction, which weeds out useless, redundant or ineffective connections. Here, the idea is similar.
## Expansion: try all the indicators you want ##
In the first stage, you just try any indicator that sounds like an interesting idea. The way you select the indicators is up to you, either it can be because it sounds like a good idea, or because it's in line with your main strategy (eg, a volatility indicator when your strategy is contrarian).
Whatever criteria you choose, you should:
1) Remain open to new types of indicators potentially outside your main field, as they can broaden your horizons,
2) Remain skeptic of any claims of effectiveness until you test the indicators and see tha they work for yourself (in the second stage: contraction),
3) Study the indicator to understand how it works and why it works. Don't just blindly use an indicator without knowing what it actually represents precisely, otherwise you will get bit by its limitations and false positives at some point in the future, likely when you will have a lot of money on the table to lose!
Once you have selected a set of indicators, or if you have reached the maximum number of indicators you can add in your TradingView plan (as it happens to me!), then you can go to the next step to weed indicators out.
## Contraction: drop everything that isn't directly useful to you##
In the second stage, we will extensively test the indicators for ourselves, on the assets we are interested in, and in others as well, to "field test" them and see if they work in our strategy. Indeed, trading and investment rely on a balance between collecting enough infos and keeping it simple enough (KISS principle(2)) to support our systematic decision-making process, without information overload which can produce decision paralysis.
The contraction/filtering process is more involved than the first stage, because you have to do the manual, dirty work of testing, it takes time, but this is the only way you can see whether the indicator work as intended and that they work for you. No two people will use the same indicator the same way as I explain in another post (3), so bear in mind that some indicators that may not work for someone else may work for you, and inversely an indicator that works for someone else's strategy may not for you, so the popularity of an indicator is no indication of effectiveness.
Here is a step-by-step outline of my process, feel free to add more steps depending on your needs:
1) Signal-to-Noise test: test on weekly and daily. If the indicator can't be reliable, can't produce good signals with low false positives and high true positives on these long timeframes that are much less noisy than shorter timeframes, then they are useless. Some people claim that there are indicators that work exclusively on lower timeframes, I am not trading such smaller timeframes although I can trade down to 15min, so your mileage may vary, but I remain yet to be convinced that this is true.
2) Redundancy test. If you already found a good indicator that works reasonably well for you, then compare any new indicator to this "best" indicator as a benchmark reference point. This will allow to weed out indicators that cannot provide new, non-redundant data. For example, in the chart of this post, I study correlations, which I compare against the signals generated by my RSI+ (alt) indicator which I consider one of my most reliable. Of course, the signal is of a different kind, but it still provides me a reference point as to whether the correlations can provide me with an additional edge or whether I should just stick to using only the RSI+ indicator. In practice, if the new indicator(s) can provide new, non redundant data, as shown by slightly different predictions in different scenarios or maybe a bit earlier, then great, I keep them. If not, for example the indicator does provide reliable info but it would lead me to take the same decisions at the same time, or worse, later than my best indicator, then I remove it.
3) Generalizabiliy test. Test on multiple markets, on mutiple timeframes, to check generalizability: if it doesn't generalize, the model is overfit on one target market's history, and this likely won't even work for the future if this same market, ie, this is an issue often encountered for models made specifically for bitcoin or ethereum.
4) Misleading test. Use bar replay, to check how the indicator behaves in realtime: does it sprout a lot of false positive in realtime, or is it as useful and predictive, or better, in real-time than when used for historical bars? Or worst being repainting indicators rewriting the past, such as pivots or zigzag, they look super accurate aposteriori but it's only because they cheat (see tradingview pinescript fage about that), using bar replay will help you detect them 100% of the time. Bar replay is one of the best tools you have to test indicators, don't underestimate it. Yes, it's time consuming, but it's well worth it, and you'll become quicker and quicker to use it over time with experience. For more information about the different types of repainting indicators, there is an excellent article in the PineScript documentation, it's worth reading even for non-coders(4).
5) Grouping and intra-class comparison. Finally, group indicators on the same study, so you can quickly answer a question eg about volume and volatility, or about market cycles, etc by checking the adequate chart. Otherwise, if you mix indicators between different charts, it will take you longer to analyze and compare the various signals. Also this allows to compare similar indicators between them to see if they really are useful, non-redundant. For example, in the chart above, it's a Correlations grouped study, so I added almost exclusively correlations indicators; while the delta-agnostic and (pearson) correlation coefficient both provide non-redundant infos, Spearman correlation and Kendall correlation indicators are redundant, although they shouldn't (they should capture non-linear relationships, whereas Pearson can only capture linear ones), their results aren't any different in practice with the pearson correlation coefficient in terms of significant signals they generate that would change my decision process, so we could drop two out of these three correlation coefficients, which would unclutter our chart without losing any data.
## Wrapping-up: continually refine your indicators ##
At the end of the day, it's important to continually try to adapt to the markets. Indicators can continue working, while others may fail, or in the end you find them too difficult to use in practice with your strategy. Your strategy may also evolve over time, and so your indicators should too. Don't ever feel attached to your indicators, you can revisit and question their utility at anytime, and you can go through the steps above again, and drop any indicator at anytime, even if they were useful before, what matters is whether they are still useful now.
There is also a next step for those who are open to learn programming: creating your own indicators. Not so much to create unique opportunities, although they might, but to better understand the market. You should view indicators as a way to better understand some facet of the market, indicators answer the specific questions their authors wanted to find an answer for. So by using indicators of other authors, you are reading the solutions to others questions. But you can also form your own questions, and then the next logical step is to develop your own indicators to find your own answers. And hopefully share them under open-source, so that we can all learn together (and this likely won't impact your profitability, to the contrary, as I explain elsewhere!(3)).
In summary, we can quote Bruce Lee, who described a very similar process for his mastery of martial arts as he taught his own named Jeet Kune Do:
"Absorb what is useful, reject what is useless, add what is essentially your own."
I hope this post was useful to you, and if you have an idea of a criterion or a step you use to select indicators that I didn't list above, please share it in the comments!
Enjoy, Trade Safely!
Tartigradia
(1): Technical indicators: what is useful and what isn't, by LuxAlgo
(2): en.wikipedia.org/wiki/KISS_principle
(3): Why my indicators are open-source, and why yours should be too, by Tartigradia
(4): Repainting — Pine Script™ v5 User Manual v5 documentation
My approach is to use a two-stages process: 1) Expansion, 2) Contraction.
This is the same process that happens in our brains when they develop, first there is neuronal and synaptic expansion, creating lots of new connections that are not necessarily efficient, then there is contraction, which weeds out useless, redundant or ineffective connections. Here, the idea is similar.
## Expansion: try all the indicators you want ##
In the first stage, you just try any indicator that sounds like an interesting idea. The way you select the indicators is up to you, either it can be because it sounds like a good idea, or because it's in line with your main strategy (eg, a volatility indicator when your strategy is contrarian).
Whatever criteria you choose, you should:
1) Remain open to new types of indicators potentially outside your main field, as they can broaden your horizons,
2) Remain skeptic of any claims of effectiveness until you test the indicators and see tha they work for yourself (in the second stage: contraction),
3) Study the indicator to understand how it works and why it works. Don't just blindly use an indicator without knowing what it actually represents precisely, otherwise you will get bit by its limitations and false positives at some point in the future, likely when you will have a lot of money on the table to lose!
Once you have selected a set of indicators, or if you have reached the maximum number of indicators you can add in your TradingView plan (as it happens to me!), then you can go to the next step to weed indicators out.
## Contraction: drop everything that isn't directly useful to you##
In the second stage, we will extensively test the indicators for ourselves, on the assets we are interested in, and in others as well, to "field test" them and see if they work in our strategy. Indeed, trading and investment rely on a balance between collecting enough infos and keeping it simple enough (KISS principle(2)) to support our systematic decision-making process, without information overload which can produce decision paralysis.
The contraction/filtering process is more involved than the first stage, because you have to do the manual, dirty work of testing, it takes time, but this is the only way you can see whether the indicator work as intended and that they work for you. No two people will use the same indicator the same way as I explain in another post (3), so bear in mind that some indicators that may not work for someone else may work for you, and inversely an indicator that works for someone else's strategy may not for you, so the popularity of an indicator is no indication of effectiveness.
Here is a step-by-step outline of my process, feel free to add more steps depending on your needs:
1) Signal-to-Noise test: test on weekly and daily. If the indicator can't be reliable, can't produce good signals with low false positives and high true positives on these long timeframes that are much less noisy than shorter timeframes, then they are useless. Some people claim that there are indicators that work exclusively on lower timeframes, I am not trading such smaller timeframes although I can trade down to 15min, so your mileage may vary, but I remain yet to be convinced that this is true.
2) Redundancy test. If you already found a good indicator that works reasonably well for you, then compare any new indicator to this "best" indicator as a benchmark reference point. This will allow to weed out indicators that cannot provide new, non-redundant data. For example, in the chart of this post, I study correlations, which I compare against the signals generated by my RSI+ (alt) indicator which I consider one of my most reliable. Of course, the signal is of a different kind, but it still provides me a reference point as to whether the correlations can provide me with an additional edge or whether I should just stick to using only the RSI+ indicator. In practice, if the new indicator(s) can provide new, non redundant data, as shown by slightly different predictions in different scenarios or maybe a bit earlier, then great, I keep them. If not, for example the indicator does provide reliable info but it would lead me to take the same decisions at the same time, or worse, later than my best indicator, then I remove it.
3) Generalizabiliy test. Test on multiple markets, on mutiple timeframes, to check generalizability: if it doesn't generalize, the model is overfit on one target market's history, and this likely won't even work for the future if this same market, ie, this is an issue often encountered for models made specifically for bitcoin or ethereum.
4) Misleading test. Use bar replay, to check how the indicator behaves in realtime: does it sprout a lot of false positive in realtime, or is it as useful and predictive, or better, in real-time than when used for historical bars? Or worst being repainting indicators rewriting the past, such as pivots or zigzag, they look super accurate aposteriori but it's only because they cheat (see tradingview pinescript fage about that), using bar replay will help you detect them 100% of the time. Bar replay is one of the best tools you have to test indicators, don't underestimate it. Yes, it's time consuming, but it's well worth it, and you'll become quicker and quicker to use it over time with experience. For more information about the different types of repainting indicators, there is an excellent article in the PineScript documentation, it's worth reading even for non-coders(4).
5) Grouping and intra-class comparison. Finally, group indicators on the same study, so you can quickly answer a question eg about volume and volatility, or about market cycles, etc by checking the adequate chart. Otherwise, if you mix indicators between different charts, it will take you longer to analyze and compare the various signals. Also this allows to compare similar indicators between them to see if they really are useful, non-redundant. For example, in the chart above, it's a Correlations grouped study, so I added almost exclusively correlations indicators; while the delta-agnostic and (pearson) correlation coefficient both provide non-redundant infos, Spearman correlation and Kendall correlation indicators are redundant, although they shouldn't (they should capture non-linear relationships, whereas Pearson can only capture linear ones), their results aren't any different in practice with the pearson correlation coefficient in terms of significant signals they generate that would change my decision process, so we could drop two out of these three correlation coefficients, which would unclutter our chart without losing any data.
## Wrapping-up: continually refine your indicators ##
At the end of the day, it's important to continually try to adapt to the markets. Indicators can continue working, while others may fail, or in the end you find them too difficult to use in practice with your strategy. Your strategy may also evolve over time, and so your indicators should too. Don't ever feel attached to your indicators, you can revisit and question their utility at anytime, and you can go through the steps above again, and drop any indicator at anytime, even if they were useful before, what matters is whether they are still useful now.
There is also a next step for those who are open to learn programming: creating your own indicators. Not so much to create unique opportunities, although they might, but to better understand the market. You should view indicators as a way to better understand some facet of the market, indicators answer the specific questions their authors wanted to find an answer for. So by using indicators of other authors, you are reading the solutions to others questions. But you can also form your own questions, and then the next logical step is to develop your own indicators to find your own answers. And hopefully share them under open-source, so that we can all learn together (and this likely won't impact your profitability, to the contrary, as I explain elsewhere!(3)).
In summary, we can quote Bruce Lee, who described a very similar process for his mastery of martial arts as he taught his own named Jeet Kune Do:
"Absorb what is useful, reject what is useless, add what is essentially your own."
I hope this post was useful to you, and if you have an idea of a criterion or a step you use to select indicators that I didn't list above, please share it in the comments!
Enjoy, Trade Safely!
Tartigradia
(1): Technical indicators: what is useful and what isn't, by LuxAlgo
(2): en.wikipedia.org/wiki/KISS_principle
(3): Why my indicators are open-source, and why yours should be too, by Tartigradia
(4): Repainting — Pine Script™ v5 User Manual v5 documentation
6) Sensitivity test and optimization: feel free to tweak the indicators' parameters, not only to try to find parameters that better fit your strategy, but also to see how the parameters change the indicator's result. This is a common practice in machine learning and statistics to evaluate how much a model is sensitive to parameters and noise, with models that vary less in their results despite parameters change being considered more robust. Also, different set of parameters can make the same indicator detect different kind of signals, eg, a trend indicator can sometimes be converted into a market cycle indicator by increasing the lookback period and other paramaters.
Addendum about the neural expansion and contraction processes, as I realize I didn't explain enough what their purposes are:
* neural expansion create all kinds of neurons and synaptic connections to try to find ways to complete the task we are trying to learn, essentially, it's a "throw anything at the wall and see what sticks" approach.
* Neural contraction is the complementary next step: once our brain found something that works, it will try to optimize it, to make it more efficient, by removing the fluff out of the necessary, in practice by removing synaptic connections that are redundant or even worsening results. It's the same as the difference between someone learning a thousand different kinds of punches, and one who spend years mastering a few ones, the one who mastered a few ones will likely be much more powerful and efficient despite having a less diverse palette.
Of course, there is a trade-off: you don't want to be too restricted, too narrow, but at the same time you want to stick to the minimum to avoid the fluff that can confuse your decision taking process. In computer science, this is called the "exploration-exploitation trade-off", and is usually solved by 1) using an iterative process, 2) leaving some room for new approaches to be tested. Both of these steps are implemented in the process I presented above.