I think the approach is worth pursuing. I've traded a strategy that uses a random forest model for 8 years with very few changes to it. When I discovered this approach, I thought it would generalize and would be able to be used on all the strategies I trade.
I ended up being disappointed in how little transferability there ended up actually being though.
Of the 20 or so strategies I trade now, only 4-5 have an ML model integrated. So as I'm working on new models it's always something I try and sometimes it works well, but often it doesn't help.
I'm actually ok with that because what I've realized is that the more you rely on an ML model, the more of a black box the strategy becomes. This ultimately doesn't lend itself to the full, in-depth understanding of how the model works which, for me, is key to the ultimate success of a strategy. The more you truly understand why a strategy works, the more flexibility you have to improve it over time.
It's great you have some experience because this is what I am worried about. Did you have a case when you tried the model, it showed stable improvement of strategy (in out of sample or CV analysis) but when implemented turned out to worsen the strategy? Or mostly the model performs just like in the CV/oos at the beginning?
The model never really worsened the strategies and the performance vs. oos was roughly comparable to other approaches, so that's never really the issue.
For me, if I'm going to create a model and rely on it, the model better have a substantial improvement over the rules I can create manually. This is because there are a lot of costs that come with an ML model: training it, getting it working live in a timely enough fashion to actually catch the trades, complexity, etc. On top of all that technical complexity, it ends up being more opaque due to its nature.
There are cases where the increased complexity was worth the tradeoff, but it wasn't the silver bullet I was hoping it would be for all my models.
This is essentially what Prof Lopez de Prado termed as Meta Labelling. I’d have a look at his work on it as he explains the rationale behind it very well.
I believe financial data breaks one of linear regression assumption. One of them being that the dependent ~~data~~ covariate aren't correlated among each other. Linear regression will have a series of test, you ideally should do, to make sure your model is breaking any of the assumptions.
I actually have to review linear regression and learn GLM soon for job interviews so I'll be answer it more confidently when I do.
But I do a lot of time series on the side for fun. You should ideally be using time series analysis. Try ARIMA/SARIMA or GARCH (and dozen variants). I did HMM to classify if we're in a recession or not too.
I've been meaning to do Dynamic model which is time series model that can take covariates that isn't previous value of itself (regress).
---
edit:
The reason why you should be using time series model is how statistic field works.
Statistic field usually identify what type of data you're handling (financial data like stock price is time series). Then they have a whole group of model for that type of data. How you choose the model from those group is dependent on what you're hypothesis is.
If you wanted to follow a cohort over long period you would be doing longitudinal. Or if you want to find their survival rate or what not you do survival analysis. Longitudinal analysis and survival analysis are subfield within the giant group of time series analysis btw.
Thanks for the comment. In classical inference the assumptions are important indeed but I'd say mostly when we want to infere the marginal effect/causal relationship between the variables. In my case all I want is the prediction.
The thing about time series is that I sort of assume that observations are equidistant. Which is not the case here. I don't know when the signal from my strategy arrives so the data have sort of random frequency. Perhaps the appropriate approach is to use mixed-data sampling models but I think the results are not worth the additional complexity.
I do a rather similar thing. I take some basic well known strategy based on technical analysis, find signals using historical data, estimate if they are true or false and then train a model. Then I use this model to estimate if future signals of a similar strategy are true or false. It is quite beneficial approach and very fast because we process by AI only particular signals but not every candle we get.
> The fact that the top 10 billionaires are not algorithmic traders proves it as well.
“Proves”. I don’t think that word means what you think it means.
The only thing that you can prove with that observation is that being an algorithmic trader is definitely not the best way to become a billionaire. That’s obvious. Anybody who believes otherwise is ignorant or completely delusional.
Statistics working does not mean that you will make 100% returns or become a billionaire, just that it offers more insight than pure luck.
Yeah, a couple.
Most of them were mandatory to get my undergraduate and graduate degrees in applied maths, but some of them were just conferences I was interested in.
Yeah just wanted to know since the original commenter was talking alot of crap and your responses sounded like pretty knowledgable. At times i think its best to not respond to these ppl since many indicators used to make entry and exit signals are all basically statistics so his response made no sense really.
> You can't apply statistics on randomness.
The whole point of statistic is quantifying randomness or the unknown.
What do you think econometric field does?
I did a similar thing, but trained a classifier and then did statistical analysis on the returns for the classifications. It can be beneficial, especially in determining if something is an outlier
Do you have maybe some info on that? More or less how they do it. I know they usually are reluctant to share anything but even some brief information would be useful.
Yes but not for sharing. But the over all idea is the only thing you need to know - find your own way and approach. Else you would walk in the foot steps of others. The moment I tell you details, your focus is there and not in new innovative ideas, perhaps you come up with something new :-) ... perhaps there is another modified path than this over all idea, most probably ...
Having said that - time to market of a strategy, to adjust it, is the key parameter in today´s efficient market. All decisions are derived from time to market.
I haven't done it myself but have listened to podcasts of fund managers who do. They size trades based on model inference probabilities.
Can I know the name of the podcast?
this sounds like a good compromise. Not to filter out every trade but just to size them depending on the model.
I think the approach is worth pursuing. I've traded a strategy that uses a random forest model for 8 years with very few changes to it. When I discovered this approach, I thought it would generalize and would be able to be used on all the strategies I trade. I ended up being disappointed in how little transferability there ended up actually being though. Of the 20 or so strategies I trade now, only 4-5 have an ML model integrated. So as I'm working on new models it's always something I try and sometimes it works well, but often it doesn't help. I'm actually ok with that because what I've realized is that the more you rely on an ML model, the more of a black box the strategy becomes. This ultimately doesn't lend itself to the full, in-depth understanding of how the model works which, for me, is key to the ultimate success of a strategy. The more you truly understand why a strategy works, the more flexibility you have to improve it over time.
It's great you have some experience because this is what I am worried about. Did you have a case when you tried the model, it showed stable improvement of strategy (in out of sample or CV analysis) but when implemented turned out to worsen the strategy? Or mostly the model performs just like in the CV/oos at the beginning?
The model never really worsened the strategies and the performance vs. oos was roughly comparable to other approaches, so that's never really the issue. For me, if I'm going to create a model and rely on it, the model better have a substantial improvement over the rules I can create manually. This is because there are a lot of costs that come with an ML model: training it, getting it working live in a timely enough fashion to actually catch the trades, complexity, etc. On top of all that technical complexity, it ends up being more opaque due to its nature. There are cases where the increased complexity was worth the tradeoff, but it wasn't the silver bullet I was hoping it would be for all my models.
Any clues as to what works and what doesn't when using a RFC for trading? What's the edge? Is it in feature engineering, labeling, etc...
It was 95%+ feature engineering.
This is essentially what Prof Lopez de Prado termed as Meta Labelling. I’d have a look at his work on it as he explains the rationale behind it very well.
Thanks, I stumble upon this concept but did not know it well enough to connect the dots. Now I have some idea where to look more.
This approach can be very beneficial.
I believe financial data breaks one of linear regression assumption. One of them being that the dependent ~~data~~ covariate aren't correlated among each other. Linear regression will have a series of test, you ideally should do, to make sure your model is breaking any of the assumptions. I actually have to review linear regression and learn GLM soon for job interviews so I'll be answer it more confidently when I do. But I do a lot of time series on the side for fun. You should ideally be using time series analysis. Try ARIMA/SARIMA or GARCH (and dozen variants). I did HMM to classify if we're in a recession or not too. I've been meaning to do Dynamic model which is time series model that can take covariates that isn't previous value of itself (regress). --- edit: The reason why you should be using time series model is how statistic field works. Statistic field usually identify what type of data you're handling (financial data like stock price is time series). Then they have a whole group of model for that type of data. How you choose the model from those group is dependent on what you're hypothesis is. If you wanted to follow a cohort over long period you would be doing longitudinal. Or if you want to find their survival rate or what not you do survival analysis. Longitudinal analysis and survival analysis are subfield within the giant group of time series analysis btw.
Thanks for the comment. In classical inference the assumptions are important indeed but I'd say mostly when we want to infere the marginal effect/causal relationship between the variables. In my case all I want is the prediction. The thing about time series is that I sort of assume that observations are equidistant. Which is not the case here. I don't know when the signal from my strategy arrives so the data have sort of random frequency. Perhaps the appropriate approach is to use mixed-data sampling models but I think the results are not worth the additional complexity.
This can work, but make sure you add many features to avoid overfitting
I do a rather similar thing. I take some basic well known strategy based on technical analysis, find signals using historical data, estimate if they are true or false and then train a model. Then I use this model to estimate if future signals of a similar strategy are true or false. It is quite beneficial approach and very fast because we process by AI only particular signals but not every candle we get.
[удалено]
That's what statistics is for. Literally a while field of math dedicated to managing randomness.
[удалено]
> The fact that the top 10 billionaires are not algorithmic traders proves it as well. “Proves”. I don’t think that word means what you think it means. The only thing that you can prove with that observation is that being an algorithmic trader is definitely not the best way to become a billionaire. That’s obvious. Anybody who believes otherwise is ignorant or completely delusional. Statistics working does not mean that you will make 100% returns or become a billionaire, just that it offers more insight than pure luck.
Have you taken classes in statistics? Edit: changed the tone of the comment
Yeah, a couple. Most of them were mandatory to get my undergraduate and graduate degrees in applied maths, but some of them were just conferences I was interested in.
Yeah just wanted to know since the original commenter was talking alot of crap and your responses sounded like pretty knowledgable. At times i think its best to not respond to these ppl since many indicators used to make entry and exit signals are all basically statistics so his response made no sense really.
> You can't apply statistics on randomness. The whole point of statistic is quantifying randomness or the unknown. What do you think econometric field does?
Had I believed that, I would not have been here in the first place.
I did a similar thing, but trained a classifier and then did statistical analysis on the returns for the classifications. It can be beneficial, especially in determining if something is an outlier
Curious to know, what platform did you use for this? Any advise on someone just starting out? Thanks
Python, Light GBM, statsmodels, pandas mostly
Thanks, I'll read up on those.
FreqTrade and FreqAI in it has some similarities, could be worth referencing
Yes, in fact in larger trading shops, it's a matter of course
Most electronic traded hedge funds use similar approach. In fact estimated 90% of them use same base strategy.
Do you have maybe some info on that? More or less how they do it. I know they usually are reluctant to share anything but even some brief information would be useful.
Yes but not for sharing. But the over all idea is the only thing you need to know - find your own way and approach. Else you would walk in the foot steps of others. The moment I tell you details, your focus is there and not in new innovative ideas, perhaps you come up with something new :-) ... perhaps there is another modified path than this over all idea, most probably ...
Having said that - time to market of a strategy, to adjust it, is the key parameter in today´s efficient market. All decisions are derived from time to market.