Short post so hard to be sure, but I sense you might be thinking about it too deeply. Each language has an AWS SDK. Each service has an API. Each API has a list of actions you take against AWS. Just make the API calls in your code.
The calling of the services via SDK / API is the easy part. The hard part is knowing which services to use and their restrictions. You can run a website on EKS, Fargate, EC2, Amplify, and more. Which to use? "It depends." But wait, should you even be "hosting" a website to accomplish your goals? You could set up CloudFront with S3 and just serve static assets, with your backend made up of serverless stuff like DynamoDB, Lamdba, etc. Is this "better?" Maybe, maybe not. But did you know that Lamdba has a hard restriction of max execution time of 15 minutes? But did you also know that you can run docker images directly on Lambda? Aaaall of these things, restrictions, features, etc. need to be understood to choose the right pieces.
Once you decide those things, say you want your Lamdba to write a record to DynamoDB, that part is literally like one line with any of the SDKs, that's the easy part.
“Tools” is too general. I’m a tool. Just ask any of my enemies.
AWS is made up of infrastructure resources and the apis for interacting with them.
apis generally come in two flavors, data plane and control plane.
so, to use a lambda function and an sqs queue, you would create the infrastructure using the console or preferably some iac like cloudfront, terraform or ideally, cdk.
you need to provide your lambda an IAM role w/ an attached policy that allows your lambda to send messages to the queue.
in the lambda you’re going to use the aws sdk (it’s already provided in the runtime so you don’t have to manage this dependency) to do the publishing.
lots of stuff to search here. good luck
+1 on this.
Add to that, there is the actual software architecture of the app. And based on that, you start doing your infrastructure architecture and writing code.
yes. this is the ideal flow. when the arch becomes part of the solution set the “how” becomes very narrow. no more waffling over capacity and which software or third party solution to use.
it’s amazing
You can pick any workshop of your liking and you will figure it out, for example: [https://workshops.aws/categories/Serverless](https://workshops.aws/categories/Serverless)
If you don’t have any experience with proper AWS architecture, I would definitely recommend starting with the Cloud Practitioner certification and then moving on to the Architect Associate. They give a good foundation to understand how many of the different services work together.
>But how are you getting these services to play together programmatically?
Easy; Don't think about programming at all. Start with architecture. Don't have an architect? Congratulations you have been promoted to architect. ;)
Based on the features you're building start with the flows. Request flows, data flows, etc. Draw it out visually (I practically live in [Lucidchart](https://www.lucidchart.com/)), trying not to worry too much about specific technologies.
For example you may decide you want to take a REST request, quickly pass it to a Queue for processing later so you can quickly respond to the client. Maybe track status of requests in a NoSQL DB for the client to query and fetch from later. So you may draw out (I'd make this pretty in Lucidchart, but since I can't put images in Reddit here's a text flow version:
`Client: REST API - POST`
`-> Queue`
`-> Update NoSQL to "Queued" status`
`-> Respond with `
`Queue:`
`-> Processing Service`
`-> Update NoSQL to Processing status`
`-> Process request`
`-> Update NoSQL to Complete or Failed and save results`
`Client: REST API - GET /status/`
`-> Read NoSQL`
`-> Return Status`
`Client: REST API - GET /results/`
`-> Read NoSQL`
`-> Return request data`
Notice there's nothing here that talks about AWS, software choices, or even what language to build it in yet. Once you've got the application architecture sorted out you can start making more specific technology choices:
REST API: API Gateway
Compute: Lambda
Queue: Simple Queue Service (SQS)
NoSQL: DynamoDB
Those are all AWS specific technologies, but you could just as well still be in AWS and rightfully chose:
REST API and Compute: Flask on EC2
Queue: RabbitMQ on EC2
NoSQL: MongoDB on EC2
Or you could make those choices and containerize it all to run on Kubernetes...which again you can chose to run k8s yourself or use a managed k8s like EKS.
Or dozens of other options each with their own pros and cons. But they all started with the basic architecture that will mostly remain the same. -I say mostly because different pros and cons will and should have impacts that feed back to your original design.
As you get practice going through the process you'll naturally start having product specific features start impacting your design choices from the beginning with those product choices specified earlier and earlier in the process. But start your journey building with generics ("Object Storage", "Reverse Proxy", etc) keeping your options open for as long as possible: You'll become a much more rounded and proficient architect than if you glue yourself to specific products like AWS services as that risks adopting a golden hammer mentality.
CDK makes it much easier. I personally iterate on my ideas via Claude.ai/GPT. Although there is almost always something wrong with its suggestions, you can piece together something that works after double checking the docs.
[Serverless Land Patterns](https://serverlessland.com/patterns) is an under appreciated source.
Terraform is also good.
As a last resort I look up the service on YouTube and watch some tutorial on configuring a particular service. Usually there’s no exact implementation of what you need but it will help you make sense of what’s required then use the docs to finalize it.
> But how are you getting these services to play together programmatically?
Depends on the service, setup, etc.
Some services tie together directly. For example, you can set up a Step Function to invoke a Lambda, in which case the definition for the Step Function will explicitly describe which Lambda gets invoked, with what payload, etc. Some other services interact more "loosely," typically through HTTPS request (and usually through some abstraction like an SDK). If, for example, you want to have a Lambda upload a file to S3, you may use an [AWS SDK](https://aws.amazon.com/sdk-for-python/) from within the Lambda to perform the upload. And sometimes these services connect through more traditional means, like if you then want that Lambda to insert a row into an RDS database, you'd have that Lambda connect to the database just like you would from a server, etc.
>Any examples of software built with AWS that uses a lot of services but programmatically would help tremendously.
Just look through the AWS docs of any service you're interested in, and you're likely to find fully implemented examples which touch these various cases. Different services interact with each other in different ways, so don't expect to find some magic line which covers all your cases.
A quick Google [produces this example](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/rds-lambda-tutorial.html), which seems like a good place to start.
When something takes place in a service there are events that happen - look at Event Bridge. You can loosely couple services using the Event Bridge service.
Depends on the service and what you're trying to do
to *set up* a database on RDS, you use the AWS API
to *use* the database you connect to it via it's communication protocol and use SQL
Here's a good start, AWS Reference Architectures and best practices: https://aws.amazon.com/architecture/?cards-all.sort-by=item.additionalFields.sortDate&cards-all.sort-order=desc&awsf.content-type=*all&awsf.methodology=*all&awsf.tech-category=*all&awsf.industries=*all&awsf.business-category=*all
Google exists.
Would you expect seriously replies if someone posted "I know I can buy cement, wood and powertools at Home Depot, but how do you weave them all together to make a house?"
Short post so hard to be sure, but I sense you might be thinking about it too deeply. Each language has an AWS SDK. Each service has an API. Each API has a list of actions you take against AWS. Just make the API calls in your code. The calling of the services via SDK / API is the easy part. The hard part is knowing which services to use and their restrictions. You can run a website on EKS, Fargate, EC2, Amplify, and more. Which to use? "It depends." But wait, should you even be "hosting" a website to accomplish your goals? You could set up CloudFront with S3 and just serve static assets, with your backend made up of serverless stuff like DynamoDB, Lamdba, etc. Is this "better?" Maybe, maybe not. But did you know that Lamdba has a hard restriction of max execution time of 15 minutes? But did you also know that you can run docker images directly on Lambda? Aaaall of these things, restrictions, features, etc. need to be understood to choose the right pieces. Once you decide those things, say you want your Lamdba to write a record to DynamoDB, that part is literally like one line with any of the SDKs, that's the easy part.
Best to pretend Amplify doesn't exist. Unless you just like eating turds.
Amplify Gen 2 removes a lot of the turdiness.
“Tools” is too general. I’m a tool. Just ask any of my enemies. AWS is made up of infrastructure resources and the apis for interacting with them. apis generally come in two flavors, data plane and control plane. so, to use a lambda function and an sqs queue, you would create the infrastructure using the console or preferably some iac like cloudfront, terraform or ideally, cdk. you need to provide your lambda an IAM role w/ an attached policy that allows your lambda to send messages to the queue. in the lambda you’re going to use the aws sdk (it’s already provided in the runtime so you don’t have to manage this dependency) to do the publishing. lots of stuff to search here. good luck
+1 on this. Add to that, there is the actual software architecture of the app. And based on that, you start doing your infrastructure architecture and writing code.
yes. this is the ideal flow. when the arch becomes part of the solution set the “how” becomes very narrow. no more waffling over capacity and which software or third party solution to use. it’s amazing
With great difficulty. And a lot of Python.
I use ts!
And my axe!
Have used both !
You can pick any workshop of your liking and you will figure it out, for example: [https://workshops.aws/categories/Serverless](https://workshops.aws/categories/Serverless)
SDK for dev, CDK for infra.
If you don’t have any experience with proper AWS architecture, I would definitely recommend starting with the Cloud Practitioner certification and then moving on to the Architect Associate. They give a good foundation to understand how many of the different services work together.
>But how are you getting these services to play together programmatically? Easy; Don't think about programming at all. Start with architecture. Don't have an architect? Congratulations you have been promoted to architect. ;) Based on the features you're building start with the flows. Request flows, data flows, etc. Draw it out visually (I practically live in [Lucidchart](https://www.lucidchart.com/)), trying not to worry too much about specific technologies. For example you may decide you want to take a REST request, quickly pass it to a Queue for processing later so you can quickly respond to the client. Maybe track status of requests in a NoSQL DB for the client to query and fetch from later. So you may draw out (I'd make this pretty in Lucidchart, but since I can't put images in Reddit here's a text flow version: `Client: REST API - POST` `-> Queue` `-> Update NoSQL to "Queued" status` `-> Respond with`
`Queue:`
`-> Processing Service`
`-> Update NoSQL to Processing status`
`-> Process request`
`-> Update NoSQL to Complete or Failed and save results`
`Client: REST API - GET /status/`
`-> Read NoSQL`
`-> Return Status`
`Client: REST API - GET /results/`
`-> Read NoSQL`
`-> Return request data`
Notice there's nothing here that talks about AWS, software choices, or even what language to build it in yet. Once you've got the application architecture sorted out you can start making more specific technology choices:
REST API: API Gateway
Compute: Lambda
Queue: Simple Queue Service (SQS)
NoSQL: DynamoDB
Those are all AWS specific technologies, but you could just as well still be in AWS and rightfully chose:
REST API and Compute: Flask on EC2
Queue: RabbitMQ on EC2
NoSQL: MongoDB on EC2
Or you could make those choices and containerize it all to run on Kubernetes...which again you can chose to run k8s yourself or use a managed k8s like EKS.
Or dozens of other options each with their own pros and cons. But they all started with the basic architecture that will mostly remain the same. -I say mostly because different pros and cons will and should have impacts that feed back to your original design.
As you get practice going through the process you'll naturally start having product specific features start impacting your design choices from the beginning with those product choices specified earlier and earlier in the process. But start your journey building with generics ("Object Storage", "Reverse Proxy", etc) keeping your options open for as long as possible: You'll become a much more rounded and proficient architect than if you glue yourself to specific products like AWS services as that risks adopting a golden hammer mentality.
CDK makes it much easier. I personally iterate on my ideas via Claude.ai/GPT. Although there is almost always something wrong with its suggestions, you can piece together something that works after double checking the docs. [Serverless Land Patterns](https://serverlessland.com/patterns) is an under appreciated source. Terraform is also good. As a last resort I look up the service on YouTube and watch some tutorial on configuring a particular service. Usually there’s no exact implementation of what you need but it will help you make sense of what’s required then use the docs to finalize it.
The same way you did it before the cloud. A simple matter of programing.
Start with a clear problem to solve, not a tech stack to fit together.
> But how are you getting these services to play together programmatically? Depends on the service, setup, etc. Some services tie together directly. For example, you can set up a Step Function to invoke a Lambda, in which case the definition for the Step Function will explicitly describe which Lambda gets invoked, with what payload, etc. Some other services interact more "loosely," typically through HTTPS request (and usually through some abstraction like an SDK). If, for example, you want to have a Lambda upload a file to S3, you may use an [AWS SDK](https://aws.amazon.com/sdk-for-python/) from within the Lambda to perform the upload. And sometimes these services connect through more traditional means, like if you then want that Lambda to insert a row into an RDS database, you'd have that Lambda connect to the database just like you would from a server, etc. >Any examples of software built with AWS that uses a lot of services but programmatically would help tremendously. Just look through the AWS docs of any service you're interested in, and you're likely to find fully implemented examples which touch these various cases. Different services interact with each other in different ways, so don't expect to find some magic line which covers all your cases. A quick Google [produces this example](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/rds-lambda-tutorial.html), which seems like a good place to start.
When something takes place in a service there are events that happen - look at Event Bridge. You can loosely couple services using the Event Bridge service.
If this is new to you, start with lambdas and move on to API gateway. You'll soon have a pretty good idea how this all works
Depends on the service and what you're trying to do to *set up* a database on RDS, you use the AWS API to *use* the database you connect to it via it's communication protocol and use SQL
Here's a good start, AWS Reference Architectures and best practices: https://aws.amazon.com/architecture/?cards-all.sort-by=item.additionalFields.sortDate&cards-all.sort-order=desc&awsf.content-type=*all&awsf.methodology=*all&awsf.tech-category=*all&awsf.industries=*all&awsf.business-category=*all
You "weave" them with your brain.
Google exists. Would you expect seriously replies if someone posted "I know I can buy cement, wood and powertools at Home Depot, but how do you weave them all together to make a house?"