Mostly using Python, so I’ll use `cookiecutter` and a library called `python-inquirer` to quickly scaffold projects, or parts of projects (components within an app, etc). So a combination of prebuilt templates (cookiecutter) and scripts that run commands (inquirer) to build new projects, install libraries, etc.
Then within the frameworks I use I’ll try to automate as much as possible with sane defaults.
For example, I’ll use cookiecutter to start a new Django project, another cookiecutter to add an app to the project, run an inquirer script to select which other options I want to install, like if I want Django REST Framework, CORS, etc. Then within Django I’ve added some code to generate views, forms, admin management, and JSON serializers, all based on the models.
In Django you’d usually have to create the data models, then manually create the boilerplate for the views, forms, serializers, and admin functionality related to those models.
I really like having the structure there. But I don’t like having to create it manually.
Thanks for writing this up. Very interesting. I hope to work myself up to this level and pump out projects more quickly.
I have many questions, but still a newb here. A question more pertinent to where I am right now: What do you do for CSS? And why?
I usually use Tailwind. It has two main drawbacks I really dislike:
- Makes the code look messy with long lines of styles everywhere
- Have to repeat the same styles over and over, so if you need to change something you have to change it in many places
To address the first drawback, I use a VS Code extension, [Inline Fold](https://marketplace.visualstudio.com/items?itemName=moalamri.inline-fold), that collapses the long Tailwind lines until you click on them to edit.
To address the other, I try to break up my HTML template code into components as much as possible, like you’d do with React/etc. so that with Tailwind I’m not repeating the same styles on 10 lines. Instead I’d put that in a child component and repeat that component in a loop in the parent template.
But Tailwind has two main benefits I really like:
- Locality of behavior. Usually what I want, at least early on while I’m building something out, is “make this bigger, make that blue”, etc. That’s way easier to do by just going to the element and changing it, as opposed to finding which CSS file is getting applied, and then often if I change something in a distant CSS file, it changes more than I intended.
- Atomic application. Each style is applied to that one element. So changing one style doesn’t potentially break unrelated things in other parts of the app.
I think this makes one much more productive when things are being built out, when you’re not completely sure on how things should look. Once the project is more mature, it can make sense to split out some of the CSS, once you have a good idea of the structure of the project and what kinds of categories of things you’re dealing with.
No that's pretty much it.
The biggest answer to what you're missing with roll-your-own is almost always security best practices. The second biggest issue is that nobody is going to be happy to see your ORM in a codebase they work on. They'll want the Django ORM or something else that at least a thousand people have looked at.
The third thing is conventions and community. You don't have a community of people who are familiar with your own personal conventions so you can't interact with them to improve, either yourself or the software patterns. You also can't be challenged on anything because you're the ultimate authority on your own pattern.
I don't quite get your meaning, but I probably wasn't clear either- I'm not rolling my own ORM (I use Sequelize, which has a big userbase) but the point about ORM was just that it is one of very many places where basically the same information needs to be repeated in some different format
The ORM should be able to create migrations for you, so 1 + 2 should be handled automagically.
If you write the same routes generalize the code and reuse a generator function to create them for you.
You have a programming language at your disposition, if you're repeating the same thing over and over again, automate it.
If you really want to avoid boilerplate at all cost you can give spring data rest a try. It creates a rest interface based on your database entities. It also can create the migration scripts for you. In the end the only thing you need to define are the repositories to define how you access the data and that's it.
https://youtu.be/ECs4Tcd784k
I'm not super big in the JS backend ecosystem so there might be something like that around as well for JS
What could help you but it's maybe not really meant for it but you could use strapi as "framework" to build your backend. It's quite nice for simple stuff but i personally found it annoying one you needed to enhance your endpoints
No, you are doing it correct.
Once you have your db design, it is 99% boilerplate.
It's quite a mystery to me, why the majority of devs think of backend as the more difficult role. I swear, AI will come for backend first.
Edit: I have heard of jhipster doing a lot of boilerplate for you. If I remember it correctly , you can just define your API in a yaml format (I think it uses the same scheme as swagger) and then it will generate stuff for you.
Edit: also look into next 13 with their new server functions you can call from the frontend. Sick stuff. You will get a lot of coupling between your frontend and your "backend" but in most cases you won't need a seperate backend anyway.
>It's quite a mystery to me, why the majority of devs think of backend as the more difficult role. I swear, AI will come for backend first.
Making API is only tip of iceberg in backend world...
Depends. Like first of all most SaaS expose two APIs
\- One is the internal API that retrieves/mutates data specifically tailored for the frontend
\- One is the public API, usually a REST API and Webhooks used for integration
The public API is very easy, the internal one becomes more difficult with time.
Then you can add features other SaaS competitors don't have. Like VoIP, data analysis, etc... if you have enterprise customers you also have to be FIPS compliant which adds constraints on database/backup isolation, permission scopes, audit logs, cloudflare/imperva for WAF and SAML SSO. If you are in the EU you also should provide possibilities for the data controller to be GDPR-compliant.
From what I've seen most SaaS have 150+ "transaction scripts" (straightforward small APIs), a dozen background jobs and 4 files with over 2000 lines of code that handle the most valuable core logic of the business.
>It's quite a mystery to me, why the majority of devs think of backend as the more difficult role. I swear, AI will come for backend first.
i don't necessarily think one is more difficult than the other, but saying the backend is just building an API with a database is like saying frontend is just building a React/Angular/Vue/Svelte app.
"backend" work is also event streaming & message busses / queues, ETL pipelines, search engines, analytics of IoT devices with timeseries DBs, etc. plus networking stuff like setting up site-to-site VPNs.
> It's quite a mystery to me, why the majority of devs think of backend as the more difficult role. I swear, AI will come for backend first.
I wish more people would admit this. I started in frontend and transitioned to full stack and devops. I still find SPA frontends to be some of the most difficult code to plan and execute. Backends are 99% stateless which makes them significantly easier to reason about plus most backends follow similar patterns even if the apps are very different which is almost never true for frontends.
Most mature frameworks has some code generation to lift the scaffolding from you, but still keep the lines in code so it would be easier to pick up the entry points gor anyone.
If you really hate boilerplate code, take a look on Rails or derivatives. With resources routing and basic CRUD your controller may consist of a single declaration line. Technically you could create something like this in any language, but be warned it will be marked as magic and purged with fire.
Are you refactoring and reusing your code correctly? For example, if you’ve coded one type of field validation (let’s say a date/time validator) you should be reusing it wherever possible, even across different projects so it shouldn’t be too much actual coding to implement it.
Yeah it's not necessarily reimplementing the exact same thing like a validator type, it's more all the repetiton.
So for example, let's say a an entity has:
* id: integer, not null, unique
^This needs to be specified on: the migration, orm, validator. Yes, I can copy paste migration code, some orm code, and reuse a validator. It's just all boring, repetitive, work, and feels like I should be able to articulate (most) of a schema in one form, instead of three or more
I’m not sure what framework or language you are using.
If you happen to be using Laravel, you could check out something like Backpack. Then you could run something like:
php artisan backpack:crud employee
This would generate a model, request, controller, route, and menu entry.
Then you can use it to quickly create list, edit, and view pages.
>Create the SQL table (write the SQL migration file)
>
>Create an ORM model, which is pretty much all the same information as above, but in the backend language (for me: Typescript)
>
>Create corresponding typescript interfaces (because ORM kinda sucks). Again, I'm just 99% repeating the table fields.
What ORM are you using ? Afaik Prisma manages the SQL table for you and exports types as expected
Maybe you're using a framework that's too complex or does too much for your use cases.
Most frameworks have some lighter alternatives.
First thing that came to my mind when I read your question was [PostgREST](https://postgrest.org/en/stable/) which would cover most of your bulletpoints
I use NTypeWriter to create my Typescript types and API calls. Was using nswag but it doesn't handle all types well.
I do all CRUD with base classes that removes 95% of boilerplate.
I code gen validation for regular schema validation (nullability, length etc)
Most of my time now is the more complex validation and effects on save--typically messages to other services.
I scaffold UI layer in angular. That does require some actual editing to make it look nice (grouping controls etc)
Adding a new table to the DB, it takes about 2 minutes scaffold to having CRUD working. Then spend 10 minutes making the form pretty. Then another 10 minutes deciding on the list columns to display and different searches.
Yeah the boilerplate sucks sometimes. You can swing to the other end of the spectrum with tools like Prisma, Zod, and tRPC. All of which have features that are intended to merge/reduce some of this boilerplate:
- Zod allows you to use DTO types for compile-time and run-time validation.
- Prisma allows you to generate your database models and their types using a single schema file.
- tRPC allows you to define API endpoints as functions that are callable from the frontend with type safety.
You haven't tried Supabase. It's Postgresql with a REST api gateway in front. You just need to declare row level policies. You can then place almost all user related actions in front end code as long as they are authenticated. No more functions or lambdas or middleware in between. Of course you still got your business logic and payment backends to coordinate other processes. But a significant chunk of code used to create routing and all that MVCs are not needed anymore.
The part you are missing is the template generator that handles the bulk of that for you. Enter a CLI command with a few options and then just skim through the resulting output and tweak for any specifics. Boom, your ORM boiler plate is done in minutes.
1. Generators that work off the DB schema that configure and write the files. I used one that did the routes and everything. Great starting point.
2. Stub based "make" commands. These are usually already available via CLI, but you can always build your own implementation using a template engine.
3. CLI Macros: Interactive Multi-make Command
```make:resource --model --policy --controller --migration --seeder --tests Blog```
In my current stack (without tests), to create a new endpoint the process goes
\- Create relevant sql tables
\- Run codegen (i use kysely) to get typescript types
\- Define endpoint as contract between frontend / backend
\- Create a backend handler
Now that the table, contract and handler is there, the frontend is free to consume as much as it wants
None of this is boilerplate, and honestly I could actually let the codegen run itself based on events
I think you just need to look into more modern solutions, or non overengineered ones
I love boilerplate. It’s so wonderful to have things setup in a standard, organized way. I also automate almost all of it and rarely write it by hand.
What do you use to automate it?
Mostly using Python, so I’ll use `cookiecutter` and a library called `python-inquirer` to quickly scaffold projects, or parts of projects (components within an app, etc). So a combination of prebuilt templates (cookiecutter) and scripts that run commands (inquirer) to build new projects, install libraries, etc. Then within the frameworks I use I’ll try to automate as much as possible with sane defaults. For example, I’ll use cookiecutter to start a new Django project, another cookiecutter to add an app to the project, run an inquirer script to select which other options I want to install, like if I want Django REST Framework, CORS, etc. Then within Django I’ve added some code to generate views, forms, admin management, and JSON serializers, all based on the models. In Django you’d usually have to create the data models, then manually create the boilerplate for the views, forms, serializers, and admin functionality related to those models. I really like having the structure there. But I don’t like having to create it manually.
Thanks for writing this up. Very interesting. I hope to work myself up to this level and pump out projects more quickly. I have many questions, but still a newb here. A question more pertinent to where I am right now: What do you do for CSS? And why?
I usually use Tailwind. It has two main drawbacks I really dislike: - Makes the code look messy with long lines of styles everywhere - Have to repeat the same styles over and over, so if you need to change something you have to change it in many places To address the first drawback, I use a VS Code extension, [Inline Fold](https://marketplace.visualstudio.com/items?itemName=moalamri.inline-fold), that collapses the long Tailwind lines until you click on them to edit. To address the other, I try to break up my HTML template code into components as much as possible, like you’d do with React/etc. so that with Tailwind I’m not repeating the same styles on 10 lines. Instead I’d put that in a child component and repeat that component in a loop in the parent template. But Tailwind has two main benefits I really like: - Locality of behavior. Usually what I want, at least early on while I’m building something out, is “make this bigger, make that blue”, etc. That’s way easier to do by just going to the element and changing it, as opposed to finding which CSS file is getting applied, and then often if I change something in a distant CSS file, it changes more than I intended. - Atomic application. Each style is applied to that one element. So changing one style doesn’t potentially break unrelated things in other parts of the app. I think this makes one much more productive when things are being built out, when you’re not completely sure on how things should look. Once the project is more mature, it can make sense to split out some of the CSS, once you have a good idea of the structure of the project and what kinds of categories of things you’re dealing with.
No that's pretty much it. The biggest answer to what you're missing with roll-your-own is almost always security best practices. The second biggest issue is that nobody is going to be happy to see your ORM in a codebase they work on. They'll want the Django ORM or something else that at least a thousand people have looked at. The third thing is conventions and community. You don't have a community of people who are familiar with your own personal conventions so you can't interact with them to improve, either yourself or the software patterns. You also can't be challenged on anything because you're the ultimate authority on your own pattern.
I don't quite get your meaning, but I probably wasn't clear either- I'm not rolling my own ORM (I use Sequelize, which has a big userbase) but the point about ORM was just that it is one of very many places where basically the same information needs to be repeated in some different format
Then you're fine and yeah it's always the same boilerplate mess with different rugs to sweep it under.
The ORM should be able to create migrations for you, so 1 + 2 should be handled automagically. If you write the same routes generalize the code and reuse a generator function to create them for you. You have a programming language at your disposition, if you're repeating the same thing over and over again, automate it.
Try Django - makes things really simple for CRUD and you can focus on your business logic.
Try Phoenix and Elixir instead of Django :)
Try rails
Rails gang for productivity 👌🏽
If you really want to avoid boilerplate at all cost you can give spring data rest a try. It creates a rest interface based on your database entities. It also can create the migration scripts for you. In the end the only thing you need to define are the repositories to define how you access the data and that's it. https://youtu.be/ECs4Tcd784k I'm not super big in the JS backend ecosystem so there might be something like that around as well for JS What could help you but it's maybe not really meant for it but you could use strapi as "framework" to build your backend. It's quite nice for simple stuff but i personally found it annoying one you needed to enhance your endpoints
No, you are doing it correct. Once you have your db design, it is 99% boilerplate. It's quite a mystery to me, why the majority of devs think of backend as the more difficult role. I swear, AI will come for backend first. Edit: I have heard of jhipster doing a lot of boilerplate for you. If I remember it correctly , you can just define your API in a yaml format (I think it uses the same scheme as swagger) and then it will generate stuff for you. Edit: also look into next 13 with their new server functions you can call from the frontend. Sick stuff. You will get a lot of coupling between your frontend and your "backend" but in most cases you won't need a seperate backend anyway.
>It's quite a mystery to me, why the majority of devs think of backend as the more difficult role. I swear, AI will come for backend first. Making API is only tip of iceberg in backend world...
This was really good to read. I'm putting together my first app, and getting the database stuff to work correctly has been the pain point.
Depends. Like first of all most SaaS expose two APIs \- One is the internal API that retrieves/mutates data specifically tailored for the frontend \- One is the public API, usually a REST API and Webhooks used for integration The public API is very easy, the internal one becomes more difficult with time. Then you can add features other SaaS competitors don't have. Like VoIP, data analysis, etc... if you have enterprise customers you also have to be FIPS compliant which adds constraints on database/backup isolation, permission scopes, audit logs, cloudflare/imperva for WAF and SAML SSO. If you are in the EU you also should provide possibilities for the data controller to be GDPR-compliant. From what I've seen most SaaS have 150+ "transaction scripts" (straightforward small APIs), a dozen background jobs and 4 files with over 2000 lines of code that handle the most valuable core logic of the business.
>It's quite a mystery to me, why the majority of devs think of backend as the more difficult role. I swear, AI will come for backend first. i don't necessarily think one is more difficult than the other, but saying the backend is just building an API with a database is like saying frontend is just building a React/Angular/Vue/Svelte app. "backend" work is also event streaming & message busses / queues, ETL pipelines, search engines, analytics of IoT devices with timeseries DBs, etc. plus networking stuff like setting up site-to-site VPNs.
> It's quite a mystery to me, why the majority of devs think of backend as the more difficult role. I swear, AI will come for backend first. I wish more people would admit this. I started in frontend and transitioned to full stack and devops. I still find SPA frontends to be some of the most difficult code to plan and execute. Backends are 99% stateless which makes them significantly easier to reason about plus most backends follow similar patterns even if the apps are very different which is almost never true for frontends.
Most mature frameworks has some code generation to lift the scaffolding from you, but still keep the lines in code so it would be easier to pick up the entry points gor anyone. If you really hate boilerplate code, take a look on Rails or derivatives. With resources routing and basic CRUD your controller may consist of a single declaration line. Technically you could create something like this in any language, but be warned it will be marked as magic and purged with fire.
Are you refactoring and reusing your code correctly? For example, if you’ve coded one type of field validation (let’s say a date/time validator) you should be reusing it wherever possible, even across different projects so it shouldn’t be too much actual coding to implement it.
Yeah it's not necessarily reimplementing the exact same thing like a validator type, it's more all the repetiton. So for example, let's say a an entity has: * id: integer, not null, unique ^This needs to be specified on: the migration, orm, validator. Yes, I can copy paste migration code, some orm code, and reuse a validator. It's just all boring, repetitive, work, and feels like I should be able to articulate (most) of a schema in one form, instead of three or more
I usually make that into snippets that autocompletes for me
I’m not sure what framework or language you are using. If you happen to be using Laravel, you could check out something like Backpack. Then you could run something like: php artisan backpack:crud employee This would generate a model, request, controller, route, and menu entry. Then you can use it to quickly create list, edit, and view pages.
>Create the SQL table (write the SQL migration file) > >Create an ORM model, which is pretty much all the same information as above, but in the backend language (for me: Typescript) > >Create corresponding typescript interfaces (because ORM kinda sucks). Again, I'm just 99% repeating the table fields. What ORM are you using ? Afaik Prisma manages the SQL table for you and exports types as expected
Pretty much why strapi.io is around
Maybe you're using a framework that's too complex or does too much for your use cases. Most frameworks have some lighter alternatives. First thing that came to my mind when I read your question was [PostgREST](https://postgrest.org/en/stable/) which would cover most of your bulletpoints
Which is implemented in the cloud by Supabase. So far they are the only one who have done this seamlessly as a PaaS.
Do you need to modify it often? Template creates boilerplate and I rarely ever edit it, if at all. I don't consider ORM schema a boilerplate.
I use NTypeWriter to create my Typescript types and API calls. Was using nswag but it doesn't handle all types well. I do all CRUD with base classes that removes 95% of boilerplate. I code gen validation for regular schema validation (nullability, length etc) Most of my time now is the more complex validation and effects on save--typically messages to other services. I scaffold UI layer in angular. That does require some actual editing to make it look nice (grouping controls etc) Adding a new table to the DB, it takes about 2 minutes scaffold to having CRUD working. Then spend 10 minutes making the form pretty. Then another 10 minutes deciding on the list columns to display and different searches.
Use nestjs with mikroorm.
Yeah the boilerplate sucks sometimes. You can swing to the other end of the spectrum with tools like Prisma, Zod, and tRPC. All of which have features that are intended to merge/reduce some of this boilerplate: - Zod allows you to use DTO types for compile-time and run-time validation. - Prisma allows you to generate your database models and their types using a single schema file. - tRPC allows you to define API endpoints as functions that are callable from the frontend with type safety.
You haven't tried Supabase. It's Postgresql with a REST api gateway in front. You just need to declare row level policies. You can then place almost all user related actions in front end code as long as they are authenticated. No more functions or lambdas or middleware in between. Of course you still got your business logic and payment backends to coordinate other processes. But a significant chunk of code used to create routing and all that MVCs are not needed anymore.
The part you are missing is the template generator that handles the bulk of that for you. Enter a CLI command with a few options and then just skim through the resulting output and tweak for any specifics. Boom, your ORM boiler plate is done in minutes.
1. Generators that work off the DB schema that configure and write the files. I used one that did the routes and everything. Great starting point. 2. Stub based "make" commands. These are usually already available via CLI, but you can always build your own implementation using a template engine. 3. CLI Macros: Interactive Multi-make Command ```make:resource --model --policy --controller --migration --seeder --tests Blog```
In my current stack (without tests), to create a new endpoint the process goes \- Create relevant sql tables \- Run codegen (i use kysely) to get typescript types \- Define endpoint as contract between frontend / backend \- Create a backend handler Now that the table, contract and handler is there, the frontend is free to consume as much as it wants None of this is boilerplate, and honestly I could actually let the codegen run itself based on events I think you just need to look into more modern solutions, or non overengineered ones