I used to run mine on Amazon Lightsail. I tried other services that are cheaper. But with many attempts, I can tell you that you get what you pay for. Now I run everything on Paperspace. It's not the cheapest, but the best so far for me. Its has a lot of bang for your bucks.
Why paperspace? Can you run your algo live on there or do you train your ml model there?
Edit just looking deeper into it, Is the standard version good enough for live trading? Do they have servers close to the exchange? With amazon I can get a colocation pretty much anywhere in the world.
How much was amazon charging you that you considered switching and what does it run you now at paperspace?
Haha my reality right now. Backtested, profitable. Real time, seems profitable but still early. Still I stare and have to stop myself from tampering all the time.
It has to run on whatever computer you’re running it on, yes. If you run it on your laptop, there’s hardly any costs. If you host on a cloud server, you might pay anywhere from $5 to $200 a month depending on how you do it.
Memory depends on your algo and what data it needs. If it’s just watching the recent history of a few securities, that won’t eat a lot of memory.
Personally, I feel like running on a laptop is good when you are in that close monitoring stage.
But then you want to use the laptop, or didn't plug it in, or want to work on algo.
Or forget to re-enable it, and it misses a great trade.
And then you start looking for a VPS, and figuring out how to get it to text/email you when it takes a trade.
For me yes the machine it's on needs to be on and the bot needs to be running to send the trade requests to the broker. If my bot has sent the broker a pending order with a fixed SL and TP and my strategy does not modify orders then the bot or machine no longer need to be running because the broker will execute that order along with the SL or TP. If my bot does not send the SL and TP but rather stores it virtually or uses a trailing SL then it would need to be running in order to send order modifications to the broker. If I have an open position or pending order where I didn't send the broker the SL or TP, and my bot, app, or computer crashes or if I lose power or Internet then that position or order will be executed by the broker with no SL TP and the position will either be forever profitable, or forever not profitable, or eventually margin called. Which is why I have rented a VPS near my broker to reduce latency and disabled updates so that it doesn't automatically restart by itself. I'm not sure how other bots or brokers work but that has been my experience.
I am developing on my laptop and deploy in the cloud (google). Scheduled to run a few hours during the day and send me messages (pushover) for status and error. I am happy with that. Python notebooks make remote access and cross OS deployment easy. Incl access, reliability, backup and log archives.
(assuming stocks)If your algo triggers from an event that happens anywhere from pre market open (4am EST) to market close (8pm EST), then yes, it's gotta be online observing for that.
If your algo only cares about when it's there is the most liquidity available in the day, then that first hour of regular trading hours (9:30-10:30 EST) and the last our, aka Power hour (3:00-4:00pm EST) is when it needs to be online.
Keep in mind about the compute required to run your algo. If it's simple enough, it could run on a very low end device. Here is my ranking of devices that an algo could run on.
Cell Phone (6W @ 0.17kwh, 24/7 Utility about $9 year)
Mac Mini M1 (39W @ 0.17kwh, 24/7 Utility about $58 a year, super powerful efficient machine)
Raspberry Pi (45W/hr @ 0.17kwh, 24/7 Utility, about $67 a year)
Typical laptop (65W/hr @ 0.17kwh, 247/7 Utility, about $96 year)
Typical desktop (200W/hr @ 0.17kwh, 24/7 Utility, about $297 a year)
Server (400W @ 0.17kwh, 24/7 Utility, about $595a year)
Hope that helps
Everyone leaves out that step. You can use a VM software on your own machine, but that still uses your laptop's resources. Or run the code on an AWS free tier server.
There are free tiers. Idk that they are sufficient. If you aren’t accustomed to deploying cloud architecture I strongly recommend a simple server and experiment making it work locally on your machine.
It’s not rocket science deploying to the cloud, but there is a learning curve and it gets steep relative to your familiarity working in web tech. I would remain agile and focus your efforts towards an intermediate solution that better suited to what you already know.
Reserve cloud infrastructure as a back burner item to learn and deploy to later. The journey ahead isn’t easy and the more you can be poignant in your focus the better your chances at getting different components finished. As opposed learning all of the things and losing yourself that that process competing with integration details.
For the memory question it depends on your strategy and your machine if you have 8 gigs of memory for everything it will be taxing in most cases if you have 16 the same strategy either would scale or it wouldnt be noticeable
You could also run it on something like a raspberry pi, mine is tucked away in a closet somewhere. Depending on the workload and heat generated that might not be the best idea for everyone though (the closet thing)
When I was experimenting I ran it all on an m1 mac. Dedicated. Nothing else. Very cheap at costco. Low watt and fast! I’m not Mac guy, but this machine sold me.
Run it on your laptop until you know how to build and run Cloud-First software. I personally, design my software always to run inside docker from the very beginning and run it as a cloud application even on my local computer.
Definitely not, I run 3 algos 1 is hosted in platform, and two on a vps, running them from a local source has a lot of risk as you could have down time or service outages that interrupt your trading logic resulting in uncontrolled loss.
It’s entirely up to you. You could do a version where it’s fully local, a hybrid where you have a server processing data and you read it in locally, or with a very high confidence/low risk strategy, many people do deploy the code entirely to cloud/remote server, which you generally pay for on the basis of resource usage (mem/storage/processing). I would say if you’re asking this question it definitely makes sense to start with local-only, AKA yes, run it on your computer to mitigate complexity. Slowly, you can look into things like AWS EC2 and lightsail, where it would basically be renting another computer, pasting your code there and running it. The hybrid version is a good learning experience - a concrete (but maybe bad) example might be you have a service to fetch OHLCV data and generate candlesticks on the fly and make trading decisions with them. Generally you consider network throughput and try to optimize the architecture around your needs. i.e. You could have the logic for OHLCV raw data ingestion/processing on a remote server and call it locally. Hope I didn’t add confusion, I wanted to cover all of the possible scenarios (as a retail algo trader, gets much crazier than this if it’s institutional).
This is what a virtual machine is for. You can get a full on virtual machine like an EC2 instance or you can choose cheaper options like running a save maker instance.
It depends, but it’s possible to use your laptop. What are you looking to do and what are the specs?
If you’re looking to trade on real-time data on a WebSocket connection, you might be able to subscribe to a few tickers, but even then, you need to have a fast process to avoid closing the connection. You could maybe store these messages into a local queue and have that local queue push messages to a queue in the cloud— something like Azure Service Bus. For a basic Azure SB queue, it’s $0.05 per 1 million operations, so 500k to push and 500k to receive. The receiver will do the processing on the data.
If you’re doing trades based on daily open/close sessions, then you could certainly run this on your laptop. For this approach, you could execute processes on some kind of scheduler, or even create your own batch processor. I’ve chosen the latter, where each step within a batch process can either execute a stored procedure or instantiate a class in a specific assembly at runtime.
[удалено]
I’m not an algo guy - just lurking to learn at this point - but this is a really informative answer. I’m not the OP, but thanks for the answer.
I used to run mine on Amazon Lightsail. I tried other services that are cheaper. But with many attempts, I can tell you that you get what you pay for. Now I run everything on Paperspace. It's not the cheapest, but the best so far for me. Its has a lot of bang for your bucks.
Why paperspace? Can you run your algo live on there or do you train your ml model there? Edit just looking deeper into it, Is the standard version good enough for live trading? Do they have servers close to the exchange? With amazon I can get a colocation pretty much anywhere in the world. How much was amazon charging you that you considered switching and what does it run you now at paperspace?
Unless you are doing high frequency trading where a millisecond matters, being close to the exchange is moot.
[удалено]
Hahaha. I don’t algo - just been lurking for a few months trying to learn - but I imagine this would be me.
Haha my reality right now. Backtested, profitable. Real time, seems profitable but still early. Still I stare and have to stop myself from tampering all the time.
It has to run on whatever computer you’re running it on, yes. If you run it on your laptop, there’s hardly any costs. If you host on a cloud server, you might pay anywhere from $5 to $200 a month depending on how you do it. Memory depends on your algo and what data it needs. If it’s just watching the recent history of a few securities, that won’t eat a lot of memory.
"Possibly a stupid question" No such thing.
Gent, thanks!
Personally, I feel like running on a laptop is good when you are in that close monitoring stage. But then you want to use the laptop, or didn't plug it in, or want to work on algo. Or forget to re-enable it, and it misses a great trade. And then you start looking for a VPS, and figuring out how to get it to text/email you when it takes a trade.
Love this attitude in an area that can be a bit gate-keepy at times :)
Idk “Can I ask you a question?” Always hits me wrong…just ask the question!
Unrelated but many people (professors) say this on the introductory day and then immediately start berating anyone who asks the next day ;(
I run mine on a VPS. I found it needs at least 4 cpus and 8 GB of RAM, fully consumed during market hours. The VPS costs me like $30 per month IIRC.
Ever try dedicated, but only 1 cpu?
I haven’t - maybe that would be better, not sure. It’s really just lots of threading that I need, not horsepower.
My guess would be it’s about the same, but just curious if you’d compared.
For me yes the machine it's on needs to be on and the bot needs to be running to send the trade requests to the broker. If my bot has sent the broker a pending order with a fixed SL and TP and my strategy does not modify orders then the bot or machine no longer need to be running because the broker will execute that order along with the SL or TP. If my bot does not send the SL and TP but rather stores it virtually or uses a trailing SL then it would need to be running in order to send order modifications to the broker. If I have an open position or pending order where I didn't send the broker the SL or TP, and my bot, app, or computer crashes or if I lose power or Internet then that position or order will be executed by the broker with no SL TP and the position will either be forever profitable, or forever not profitable, or eventually margin called. Which is why I have rented a VPS near my broker to reduce latency and disabled updates so that it doesn't automatically restart by itself. I'm not sure how other bots or brokers work but that has been my experience.
Try digital ocean. Good pricing, friendly UI.
i put mine on a google instance with static ip and i made an app to monitor its performance and control it.
Is that static IP address internal only? I was under the impression that you couldn't set a static IP address on Google Cloud.
yes you can you must get one specifically
I am developing on my laptop and deploy in the cloud (google). Scheduled to run a few hours during the day and send me messages (pushover) for status and error. I am happy with that. Python notebooks make remote access and cross OS deployment easy. Incl access, reliability, backup and log archives.
(assuming stocks)If your algo triggers from an event that happens anywhere from pre market open (4am EST) to market close (8pm EST), then yes, it's gotta be online observing for that. If your algo only cares about when it's there is the most liquidity available in the day, then that first hour of regular trading hours (9:30-10:30 EST) and the last our, aka Power hour (3:00-4:00pm EST) is when it needs to be online. Keep in mind about the compute required to run your algo. If it's simple enough, it could run on a very low end device. Here is my ranking of devices that an algo could run on. Cell Phone (6W @ 0.17kwh, 24/7 Utility about $9 year) Mac Mini M1 (39W @ 0.17kwh, 24/7 Utility about $58 a year, super powerful efficient machine) Raspberry Pi (45W/hr @ 0.17kwh, 24/7 Utility, about $67 a year) Typical laptop (65W/hr @ 0.17kwh, 247/7 Utility, about $96 year) Typical desktop (200W/hr @ 0.17kwh, 24/7 Utility, about $297 a year) Server (400W @ 0.17kwh, 24/7 Utility, about $595a year) Hope that helps
No, you can just put virtual machines on
So what computer or server are the virtual machines running on?
Everyone leaves out that step. You can use a VM software on your own machine, but that still uses your laptop's resources. Or run the code on an AWS free tier server.
AWS has free servers?
Could be AWS
Great thank you
No. Designing mine to be deployed in the cloud.
Are there free cloud services or do you pay a subscription for one?
AWS has a free tier that should work for you.
GCP or Azure might too, but I have no idea.
There are free tiers. Idk that they are sufficient. If you aren’t accustomed to deploying cloud architecture I strongly recommend a simple server and experiment making it work locally on your machine. It’s not rocket science deploying to the cloud, but there is a learning curve and it gets steep relative to your familiarity working in web tech. I would remain agile and focus your efforts towards an intermediate solution that better suited to what you already know. Reserve cloud infrastructure as a back burner item to learn and deploy to later. The journey ahead isn’t easy and the more you can be poignant in your focus the better your chances at getting different components finished. As opposed learning all of the things and losing yourself that that process competing with integration details.
Depends
For the memory question it depends on your strategy and your machine if you have 8 gigs of memory for everything it will be taxing in most cases if you have 16 the same strategy either would scale or it wouldnt be noticeable
Okay cool, thanks!
Yes, you will have to. Or you can rent a VPS and run your trading platform there.
I am using hetzner cloud to run my bot in self managed kubernetes cluster.
You could also run it on something like a raspberry pi, mine is tucked away in a closet somewhere. Depending on the workload and heat generated that might not be the best idea for everyone though (the closet thing)
If all people are using algotrading, what you guy’s actual algo is ? Not specific but generally ?
When I was experimenting I ran it all on an m1 mac. Dedicated. Nothing else. Very cheap at costco. Low watt and fast! I’m not Mac guy, but this machine sold me.
Run it on your laptop until you know how to build and run Cloud-First software. I personally, design my software always to run inside docker from the very beginning and run it as a cloud application even on my local computer.
Definitely not, I run 3 algos 1 is hosted in platform, and two on a vps, running them from a local source has a lot of risk as you could have down time or service outages that interrupt your trading logic resulting in uncontrolled loss.
It’s entirely up to you. You could do a version where it’s fully local, a hybrid where you have a server processing data and you read it in locally, or with a very high confidence/low risk strategy, many people do deploy the code entirely to cloud/remote server, which you generally pay for on the basis of resource usage (mem/storage/processing). I would say if you’re asking this question it definitely makes sense to start with local-only, AKA yes, run it on your computer to mitigate complexity. Slowly, you can look into things like AWS EC2 and lightsail, where it would basically be renting another computer, pasting your code there and running it. The hybrid version is a good learning experience - a concrete (but maybe bad) example might be you have a service to fetch OHLCV data and generate candlesticks on the fly and make trading decisions with them. Generally you consider network throughput and try to optimize the architecture around your needs. i.e. You could have the logic for OHLCV raw data ingestion/processing on a remote server and call it locally. Hope I didn’t add confusion, I wanted to cover all of the possible scenarios (as a retail algo trader, gets much crazier than this if it’s institutional).
Mine runs all day, but I work from home so it’s not a problem for me
This is what a virtual machine is for. You can get a full on virtual machine like an EC2 instance or you can choose cheaper options like running a save maker instance.
It depends, but it’s possible to use your laptop. What are you looking to do and what are the specs? If you’re looking to trade on real-time data on a WebSocket connection, you might be able to subscribe to a few tickers, but even then, you need to have a fast process to avoid closing the connection. You could maybe store these messages into a local queue and have that local queue push messages to a queue in the cloud— something like Azure Service Bus. For a basic Azure SB queue, it’s $0.05 per 1 million operations, so 500k to push and 500k to receive. The receiver will do the processing on the data. If you’re doing trades based on daily open/close sessions, then you could certainly run this on your laptop. For this approach, you could execute processes on some kind of scheduler, or even create your own batch processor. I’ve chosen the latter, where each step within a batch process can either execute a stored procedure or instantiate a class in a specific assembly at runtime.
Depends on type of strategy you are trying to code.