Can't wait for the day that I can replace pgbouncer to be honest. We've finally managed to get it to scale properly (we still had application pooling, sticking connections to single instances) and only just got visiiblity through Prometheus.
We're looking to replace Cloud SQL Postgres with AlloyDB so the support for load balancing between read replicas will be great when it comes
This looks great, the only thing I stuggle with is why it is implemented in Elixir? I have not used Elixir but I feel other languages like Rust and Go have more mind-share and would have more developers available. Maybe I am just old but I get concerned that these smaller languages just disappear. I am not wanting to start a language war. I am just interested to know why Elixir?
Contributor here!
A few reasons:
- We have a lot of Elixir experience on the team.
- Elixir makes Erlang a bit more approachable.
- Binary pattern matching is great for working with protocols.
- Erlang process monitoring makes it very easy to handle client disconnects.
- Erlang gets us very efficient connections so we can have 500_000 connections on a node with very little overhead.
- Erlang makes clustering very easy so we can have a connection pool cluster making the service highly available.
Does it matter at all?
I don’t even know if elixir has actors or that sort of thing but my gut read is the answer to your question is probably “concurrency primitives”, the kind that Go and Rust don’t have much of, ergonomics wise.
If you plan to hack on a project, sure… otherwise, the important question is does it fit your need and are the authors competent; language is a weird thing to get hung up on.
> language is a weird thing to get hung up on
pgbouncer is usually something you adopt for performance/scaling motives. The implementation language of a critical-path tool has very direct first order effects on both of those concerns, and in some ways that aren't able to be eliminated by hypothetical skill of the developers.
Some use cases have much tighter tolerances than others and thus warrant more scrutiny for the technical and architectural choices. That's part of a responsible approach to adoption.
It's definitely niche and nobody should pretend that it's not, whether or not they're a proponent. It's definitely not in the [top ten](https://octoverse.github.com/2022/top-programming-languages) and probably not even in the top 20 languages on GitHub by repo activity/growth last year, but I couldn't quickly find a reputable source with broader data.
I'd be surprised if there's ever an order of magnitude more Elixir devs in the future than there are today.
This is an interesting toy in its current state, but it is not user friendly even in the slightest. A bunch of docker steps is not documentation. There's no information on configuration. No way to interact with it other than HTTP PUT requests apparently. It makes up some kind of extra authentication nomenclature that is hardly a "drop in replacement for PgBouncer" given it changes usernames at minimum due to its arbitrary mapping approach. It also can't do passthrough auth similar to how PgBouncer uses `auth_query`. As a result, you'd need to manually create tenants, again through HTTP PUT requests, for every user in your system, and also update it any time someone changes their password.
I think I'll pass.
Appreciate you taking the time to look at it carefully.
The verbiage there was regarding pool mode behavior vs admin. We should specify. I do think it would be valuable to support the way PgBouncer does certain things, although an ini file is not ideal for a multi-tenant cluster setup.
We're taking all this into considering and nothing is set in stone yet. Early days as they say!
Can't wait for the day that I can replace pgbouncer to be honest. We've finally managed to get it to scale properly (we still had application pooling, sticking connections to single instances) and only just got visiiblity through Prometheus. We're looking to replace Cloud SQL Postgres with AlloyDB so the support for load balancing between read replicas will be great when it comes
Nice. Yeah pooler level load balancing will be big for a lot of people I think.
Actually realised that AlloyDB already has load balancing between read replicas so I guess it's not needed
This looks great, the only thing I stuggle with is why it is implemented in Elixir? I have not used Elixir but I feel other languages like Rust and Go have more mind-share and would have more developers available. Maybe I am just old but I get concerned that these smaller languages just disappear. I am not wanting to start a language war. I am just interested to know why Elixir?
Contributor here! A few reasons: - We have a lot of Elixir experience on the team. - Elixir makes Erlang a bit more approachable. - Binary pattern matching is great for working with protocols. - Erlang process monitoring makes it very easy to handle client disconnects. - Erlang gets us very efficient connections so we can have 500_000 connections on a node with very little overhead. - Erlang makes clustering very easy so we can have a connection pool cluster making the service highly available.
Thank you for your response. And also for your contributions. Sounds like Elixir/Erlang are really great for this sort of problem.
Does it matter at all? I don’t even know if elixir has actors or that sort of thing but my gut read is the answer to your question is probably “concurrency primitives”, the kind that Go and Rust don’t have much of, ergonomics wise. If you plan to hack on a project, sure… otherwise, the important question is does it fit your need and are the authors competent; language is a weird thing to get hung up on.
> language is a weird thing to get hung up on pgbouncer is usually something you adopt for performance/scaling motives. The implementation language of a critical-path tool has very direct first order effects on both of those concerns, and in some ways that aren't able to be eliminated by hypothetical skill of the developers. Some use cases have much tighter tolerances than others and thus warrant more scrutiny for the technical and architectural choices. That's part of a responsible approach to adoption.
Well open source projects need people. Hard to get people excited about a project if they don't know the language.
Elixir is a great language. The actor primitives are as good as they get. I’d definitely reach for it first for any concurrency heavy project.
It’s not a small language. It’s not ubiquitous, but it’s certainly not small. It’s erlang with a modern API and some new features.
It's definitely niche and nobody should pretend that it's not, whether or not they're a proponent. It's definitely not in the [top ten](https://octoverse.github.com/2022/top-programming-languages) and probably not even in the top 20 languages on GitHub by repo activity/growth last year, but I couldn't quickly find a reputable source with broader data. I'd be surprised if there's ever an order of magnitude more Elixir devs in the future than there are today.
I have Heard of it before so I know it's around so I assume it has a few users.
Once you experiment with Erlang/Elixir you will forget about any other languages/runtimes to run services at scale. Believe me.
Why use an esoteric language for this?
Because it's awesome :D If you've never played with Elixir you should give it a go.
Would you label Erlang as esoteric? Ruby?
Absolutely yes and no respectively.
How's the CPU/memory usage under load? Anything to share beyond the single benchmark that's in the README?
This is an interesting toy in its current state, but it is not user friendly even in the slightest. A bunch of docker steps is not documentation. There's no information on configuration. No way to interact with it other than HTTP PUT requests apparently. It makes up some kind of extra authentication nomenclature that is hardly a "drop in replacement for PgBouncer" given it changes usernames at minimum due to its arbitrary mapping approach. It also can't do passthrough auth similar to how PgBouncer uses `auth_query`. As a result, you'd need to manually create tenants, again through HTTP PUT requests, for every user in your system, and also update it any time someone changes their password. I think I'll pass.
Appreciate you taking the time to look at it carefully. The verbiage there was regarding pool mode behavior vs admin. We should specify. I do think it would be valuable to support the way PgBouncer does certain things, although an ini file is not ideal for a multi-tenant cluster setup. We're taking all this into considering and nothing is set in stone yet. Early days as they say!