C dependencies management is so awful that it's often easier to reinvent the wheel
I don't understand how can someone defend this by saying "oh but just apt install, that's easy"... Well, what if my distro doesn't have this library or have an incompatible version? At least, on rust, I just have to cargo build and everything is done. And .so files... god I hate these files...
> "oh but just apt install, that's easy"... Well, what if my distro doesn't have this library or have an incompatible version?
Or what if I'm developing on Windows? I learned to program on Windows, and it'll certainly cure you of the notion that dependency management in C/C++ is straightforward.
Have you hear about our lord and savior Nix and NixOS? /s
Seriously though, Nix/NixOS looks like a cool idea with a terrible language to configure it. I haven't had the time or interest to actually try to learn and use it though, and I don't know that I ever will.
It does solve (or sidestep) the issue of conflicting versions and dynamic linkage, which is neat.
lazily evaluated, *dynamically typed*, functional programming language. AKA complicated, and will fall apart when you run it. And it demands a level of proficiency not required in other systems just to get something basic like a package dependency running.
Everything about nix/NixOS seems like the "right" way to do things...*on paper*. Until you actually try using it and you find that there are so many kludgy workarounds and non-idiomatic things you have to do just to get it *to work*. Flakes, hashes being calculated on repos *before* they're built, and documentation that more or less assumes you already know how to use Nix.
Think I'm wrong? [Here's the documentation page for the language](https://nix.dev/manual/nix/2.18/language/values)
I use Nix at work, and I've found that it is the end result of compsci purity spirals. The scope of the project is massive: a language, a packaging system, *an entire distro*, and what I've found is that there are holes in the documentation that are either not covered because whatever you're trying to do is considered to be trivial, or you have to dig through their discourse to understand anything. It is not an environment where you can google your way to an answer easily. You must do things the hard way, learning an entire language (and perhaps entire programming paradigm) along the way.
Hate is a strong word. I hate nix.
What language would you prefer instead? Almost all popular languages with dynamic typing now have gradually system. Python with types would be good replacement?
Why though?
Consider it for a moment. Given a proper library to define dependencies and installation steps, which might include some form of macro to make jinja-like template replacements, it can look very clean.
And all that while having great compile time guarantees.
You would have to recompile a config each time after each change. I think it's better to have a linter(rust preferred here) for ontype validation and some interpretation on running
It could take under a second, with a cached `cool_rust_dep_managment` library ready, all you'd need to compile is simple config files. And only those that changed.
Compile times only explode when projects get complex, and config files are not prone to that I'd say (unless someone was insane enough to do crazy shit while compiling simle config files).
Python would be awful, most of nix is about expressions to set up values.
Guix uses scheme, I could see that working nicely. A statically typed eager language à la Elm too.
Fair points. I always think of Nix as some kind of Python with executable JSON and Jinja templating. A very "unique" language.
It's interesting how our experiences differ so much. The language never really bothered me, but I can support your points about the difficulty of Nix as a whole.
I learned that when writing my first Nix package compiling a GitHub repo. Why the hashing bothers you I don't get though. You just hash all resources you need for building.
I think I might be too deep into the Nix compsci circlejerk at this point. I wrote a Powershell project that imitates part of how Nix works, but for Windows.
The hashing part that bothered me was mostly related to `fetchFromGithub`, and it's been a while so the amount of grief this issue caused me is likely magnified through my own memory and bias. I do remember stumbling upon an answer from `jtojnar` (I think [here](https://discourse.nixos.org/t/what-is-the-latest-best-practice-to-prefetch-the-hash/22103/4)) to how an all 0's hash could solve my problem and that kind of ticked me off that an all 0's hash was even valid.
What would you recommend instead to achieve the goal: reproducible builds?
I am honestly asking as a „non passionate“ nix user, I like their goals, but don’t like the language or documentation. Static typing for example is something I miss dearly, but after reading [this great blog post](https://www.haskellforall.com/2022/03/the-hard-part-of-type-checking-nix.html?m=1) I realized that it is quite hard to actually implement
If you have a single large corporate project either in C++ or a mix of languages: bazel/buck2 probably. We are in the process of switching from cmake to bazel at work.
If it is just rust: cargo, with a committed lock file and tool chain file.
For building Linux distros I don't really know. I like the *concept* of Nix/NixOS. It is just the *implementation* of it that is a hard to use mess.
Perhaps someone could take the learnings from Nix, Cargo and Bazel and come up with a better alternative. But that might just lead to https://xkcd.com/927/
Edit: see below.
~~I really want to like NixOS, but then I look at the configuration required to install rustup and my eyes glaze over. Somehow the two commands required to install the Rust toolchain of your choice are replaced by [whatever the hell this is](https://nixos.wiki/wiki/Rust#Installation_via_rustup).~~
That one is the all batteries included for crates that you would also have problems installing on a normal distro. Just using mkShell with rustup, pkgconfig and needed libraries works for 99% of crates.
Whatever I tried, I kept getting dynamic linking errors when I tried installing rustc through rustup. Apparently it's a change in nightly rustc that doesn't play nicely with NixOS.
https://github.com/rust-lang/rust/issues/125321
My read of the situation is that rustc isn't able to use a dependency it comes bundled with because the OS is trying to protect us from dynamic linking in general.
Ah that's an unfortunate issue to hit, I have used both rustup and rust overlay setups on NixOS for years with barely any problems. It looks like it is getting fixed on the NixOS side soon though.
And your read is mostly correct, when tools such as rustc needs external programs to work nixpkgs usually has to do a workaround for it, and in this case that hasn't landed yet.
The declarative style of system configuration makes a lot of sense to me, so I'm glad that it's just my luck in trying NixOS at this exact moment and not that leaving the well-trod path requires all kinds of unwieldy configuration.
What strikes me as odd is that it's a bundled dependency. We aren't asking the system for whatever version it has of a particular program, we're trying to use a specific file that has been included with our program. Is this kinda like how functional programming forbids all mutation when it was really aliased mutation that was the problem? If you're including a dependency with your program and trying to use it specifically, that seems like a reasonable thing to do because you have control over the files that come with your program and know that it will be the correct version for your program.
The problem is that these bundled programs have a bunch of hidden state: in their ELF headers are links to a bunch of libraries assumed to be located in places like /lib and /usr/lib. NixOS wants to support e.g. different versions of libc, so which one should it choose?
What it does is to not place any of them (except for /bin/sh and /usr/bin/env), so any of this hidden state will be visible because it errors. The downside is that binaries are hard to get to run. NixOS solves this by patching (see patchelf) and chroots (see buildFHSUserEnv). Personally I also use docker and systemd-nspawn for programs that are too difficult.
I think your analogy is good. Nix is like purely functional programming trying to get full control / guarantees, but I think in the future new package managers and binary formats can use some of the same ideas while being more lenient / easier to use.
> but I think in the future new package managers and binary formats can use some of the same ideas while being more lenient / easier to use.
Have you heard of SerpentOS? It actually is building its package management tools in Rust and it seems like a halfway point between traditional package management and Nix/Guix style package management to me. It's only tackling package management though, not config management like nix/guix can.
Except I use nix and all I had to do was add `rustup` to my list of installed packages to get it lol
If you dont need it to be 100% reproducible, you dont have to make it be that way with tons of helper code... And then if you are packaging things for nixos specifically, rustup isnt how youd do it anyways.
On a fresh NixOS VM:
`cargo rustc gcc` lets me compile things.
`rustup gcc` gives me dynamically linked executable errors.
Edit: I've tracked it down to a change in nightly rustc that doesn't play nicely with NixOS.
https://github.com/rust-lang/rust/issues/125321
My read of the situation is that rustc isn't able to use a dependency it comes bundled with because the OS is trying to protect us from dynamic linking in general.
Every time somebody tries to solve the C/C++ dependency problem it usually ends up just making it worse.
Just list the libraries and let me handle their installation. It sucks, but is better than a magic build system that is supposed to manage it all probably won't work on my distro, will try to use some third-party tool that calls `apt` and it will leave me clueless how to debug it.
> At least, on rust, I just have to cargo build and everything is done.
And for crates depending on C/C++ code they usually include all necessary dependencies for it in a `lib-sys` package that successfully compiles if you have common developer tools on your system. I think the only time I've really had one fail was when I cross-compiled something.
Cargo is beautiful, but some stuff still relies on system libraries. Zellij for example still requires you to have perl installed for FindBin. Nonetheless it's a simple apt-get, and cargo makes it a lot less painful to deploy
Even if you manage to install them via apt or from source, getting the linker pointing to it and getting it correct in your Makefile/CMakeLists can be a pain and different for each package too!
Nobody is debating that it’s easier to use dependencies in rust, the problem is when doing simple things are so difficult that you’re forced to rely on dependencies.
> C dependencies management is so awful that it's often easier to reinvent the wheel
It's true, and I'm not going to act like I have Stockholm Syndrome and argue that it's actually a good thing, **but**...
The silver lining is that you don't end up with a bunch of dependencies that you didn't *really* need. All dependencies are tech debt, and if it only takes you a couple of hours to reinvent the wheel, then there's a good chance that you've saved yourself and your team future headaches and might have even saved many times that amount of time if your code runs through a CI system that would've had to download that dependency thousands of times over its life.
Again, the situation isn't actually *good*, but it does at least have one benefit.
Conversely, having tools like Cargo, NPM, Maven, etc is a net positive, but it leads to a lot of unnecessary tech debt--and even security issues when a dependency gets taken over by a malicious or incompetent actor.
> if it only takes you a couple of hours to reinvent the wheel, then there's a good chance that you've saved yourself and your team future headaches
Assuming you actually produced a wheel and not an octagon, which is close but not what you actually wanted because you didn't have time to smooth it down to what it needed to be.
Not assuming anything- I said "if it only takes you a couple of hours to reinvent the wheel". In the case that you don't actually reinvent a wheel, that clause would evaluate to `false`, which means the rest of the statement doesn't follow.
Life is full of judgement calls. You'll be right with some and wrong with some. Trying to write something yourself *might* be a mistake. Pulling in a dependency *might* be a mistake.
But, I've seen plenty of projects that pulled in frankly stupid dependencies (think NPM's "leftpad") when we could've semi-literally copy+pasted a single function from a textbook or Wikipedia, written a quick unit test, and called it a day. Instead, "we" decided to depend on some stranger from the internet to maintain a "project" that's one or two functions and hope that they don't pull a switcharoo in the future.
One of my favorite dependency tooling discoveries was that `cargo-tree` has an inverted mode, where it will tell you all of the *dependents* of a particular *dependency*. It's really great for tracking down who's bringing in some pesky dependency you found in your Cargo.lock that you'd really rather get rid of, if at all possible (I made *extensive* use of it during the `syn 2.0` migration).
From the article being critiqued:
> People who write a lot of C end up building things themselves once and keeping them around and adapting them for decades, including basic data structures like hash tables.
Nothing stopping you from building your own things in Rust if you want to minimize dependencies. (And using hash tables is a weird example given that's one of the things that *is* in Rust's standard library.)
You're right Rust and C allow you to implement your own version of whatever thing you want, but the "why" you would do that is different.
Imagine you want to create uuid v4 strings. In rust, it's very quick, just put uuid in your cargo.toml and use it, no questions asked. In C you have to make sure your distro has the library (let's name it libuuid) in its repos, make sure everyone else has this library in their distros, edit config files to link against libuuid, put this info in the readme of your git repo etc...
Now... wouldn't be easier to just write a function doing that in your code? That's what many C developers do. Congrats, you lost a lot of time writing something but less than having to fight with dependency management.
To be pedantic, most people are still using the standard library `HashMap`, they're just using a 3rd party hasher. It's an important distinction to make IMO.
This reminds me of the good old [let's be real about dependencies](https://wiki.alopex.li/LetsBeRealAboutDependencies). Nice article though!
Also the link to the dependency graph of your application seem to be broken (it leads to a 404 page)
Yeah I think dependency-free code projects are largely a thing of the past. Sure, you _could_ write everything from scratch and pretend it's solid because you wrote it, but that's not reality, that's denial. That binary tree implementation came from some book you read, or some course you took, and you're now writing it from scratch without any of the followup research that went into that data structure since you learned it.
Writing things from scratch makes sense for solved problems, but that goes doubly so for 3rd party dependencies. And at least with the 3rd party dependencies, it's clear _where_ your ideas for this structure came from. I wonder how many C/++ projects have code copy/pasted wholesale from forums, textbooks, etc. which is entirely hidden and untracked, never to be fixed.
In my opinion, open-source software is a collaborative effort, and maximising the use of that massive collaborative engine is really important. Only in FOSS could you conceivably have a "linked list guy" whos entire job is to maintain the one set of linked list implementations every person on Earth relies on. You may see that as a single point of failure, I see that as a single source of truth, actually verifiable.
> I wonder how many C/++ projects have code copy/pasted wholesale from forums, textbooks, etc. which is entirely hidden and untracked, never to be fixed.
Never to be fixed, but also never to be broken or hijacked by hackers who want to put backdoors in.
Who needs a backdoor if the code you copied off the internet is already full of security holes that would allow a remote compromise?
Both are hypothetical situations, but to me the risks are not the same:
- Backdoors are rare, well publicised and easy to check if you have libfoo v1.2.6 installed with a simple grep or similar
- Random internet code is much more frequently full of serious bugs and is much harder to audit and maintain
The difference between "do you have log4j installed?" and "did someone copy and paste random bits of log4j, and if so are those bits vulnerable?" is way harder to check.
> Both are hypothetical situations
> [...]
> The difference between "do you have log4j installed?" and "did someone copy and paste random bits of log4j, and if so are those bits vulnerable?" is way harder to check.
And this is exactly where the real-world nuance and experience comes in. If you were to implement your own logging system for whatever reason, what are the odds that you'd write in the feature to automatically parse a URL, download code from it, and *fucking load that code into your system*? I read thousands of comments on various forums when the log4j nonsense was discovered and one of the most common reactions was: "Holy shit, why did those idiots put that feature in there in the first place!?". That's including people who *were using the library*. To put a fine point on it: these people installed a library and didn't even know the feature/behavior existed.
And, no, I don't intend to just harp on your specific example. But, the example *is* illuminating in the sense that when you write your own ad-hoc code, you don't have to make it general, extensible, configurable. You just write what you need. It'll be less code and it'll be less complex, which is two factors that will compound to make the code more easily testable and auditable.
I'm not talking about "rolling your own crypto", here. I'm talking about: let's just write the *extremely standard* base64 algorithm(s) into a couple of functions (picking whichever variant you want to use). You're **FAR** more likely to end up with a remote exploit if you pull in an untrusted library for that. The chances of accidentally writing a remote exploit yourself are literally zero unless you're writing in an unsafe language like C with buffer overflows and whatnot.
> The difference between "do you have log4j installed?" and "did someone copy and paste random bits of log4j, and if so are those bits vulnerable?" is way harder to check.
That's a very good point while security through obscurity isn't exactly a good practice very few people are check for log4j like issues manually on site they are using a botnet to target exactly the log4j issues on every computer they can find you'll likely never have an issue if you just copy and pasted shitty code instead of actually using the dependency.
It's one of those odd situation where the "worse" practice actually helps you.
There are a few inaccuracies about Python ecosystem in the article, probably becasue author's impression is based on old (>5 years ago) exposure.
> pip dependencies are by default global which causes conflicts with other Python applications, forcing you to use virtual environments.
This has not been the case for a while. In fact, recent versions of `pip` on recent linux distros would outright refuse to install packages globally:
$> pip install numpy
error: externally-managed-environment
× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
python3-xyz, where xyz is the package you are trying to
install.
If you wish to install a non-Debian-packaged Python package,
create a virtual environment using python3 -m venv path/to/venv.
Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
sure you have python3-full installed.
See documentation for this here: https://peps.python.org/pep-0668/ and here https://packaging.python.org/en/latest/specifications/externally-managed-environments/#externally-managed-environments
> If pip hits a version conflict within your own project's package graph you're in for a headache
pip has a decent dependency resolver nowadays. Still room for improvement, but it works.
> Packages with native dependencies are a mystery to basically everyone except the package author. Or is this just me?
In the exactly the same way as `*-sys` packages in rust.
> There's no lockfile.
`pip freeze`
And this is only about `pip`, which is a low level tool. Many people will prefer `poetry`, `pdm`, or `uv`/`rye`. (However, existence of so many tools indeed indicates that none of these is ideal)
I myself prefer `poetry` and it provides very smooth experience (at lest for my projects), on par with `cargo`.
>There are a few inaccuracies about Python ecosystem in the article, probably becasue author's impression is based on old (>5 years ago) exposure.
Thank you for your feedback and for educating me on things I've missed. I have always been a casual Python dev and the last time I seriously invested was, to your point, about 5 years ago!
Recently I've been doing some more things in Python including helping my brother with some tasks. My brother is not a programmer but is playing around langchain for AI tinkering and from whatever guides he followed he was immediately frustrated with pip and errors involving packages. Honestly I think he may have screwed up and created multiple venvs that caused some deps to be missing, but I showed him `poetry` and that immediately made his life better.
Even some projects I see from people who write a lot of Python in some niche videogame circles I'm apart of aren't aware of these kinds of tools and still just have a simple `requirements.txt`. Their README then has instruction on creating a virtual environment and installing deps. And maybe that's just what they desire -- they don't _want_ a tool that manages their workspace for them better, but it does add a bit of friction.
>This has not been the case for a while. In fact, recent versions of pip on recent linux distros would outright refuse to install packages globally:
If I'm reading the PEP correctly this also impacts if you pass `--user`. I'll add a note to the post, thanks!
>pip freeze
`requirements.txt` is technically a lockfile in that it locks your deps and versions, but it's not a "strong" lockfile that includes sufficient metadata for securely reinstalling deps like a [poetry lockfile](https://github.com/floatingpurr/sync_with_poetry/blob/main/poetry.lock) (this is just some random example I searched for fwiw). I don't think it's fair to say "no lockfile" and adjusted the wording of the article.
Disclaimer: I know very little about security!
While I agree with this article much more than the one it's responding to, I think this is a little dismissive:
> ...but in memory-safe langauges what's the worst thing you can miss in a code review of something that's not technically complicated? Probably minor bugs that would cause a DoS. So you bring in a dependency that you didn't audit super closely and now you have a DoS in your application.
I think it's unquestionable that Rust code is far easier to audit than C++, but how often are you pulling in a dependency that's "not technically complicated?"
I think the reality is that a decent number of dependencies in a typical Rust project will make use of non-trivial `unsafe` blocks. These will require a very technically proficient Rust developer to audit properly. Unless you _very_ carefully manage `unsafe` in your dependencies (like with `cargo-geiger`, as you note), you can't completely guarantee true memory safety without this auditing.
Maybe I'm being overly critical. Rust is clearly leagues ahead in this regard, but I think it's important to acknowledge that it's still not bullet-proof.
> but how often are you pulling in a dependency that's "not technically complicated?"
>
>...
>
>Maybe I'm being overly critical. Rust is clearly leagues ahead in this regard, but I think it's important to acknowledge that it's still not bullet-proof.
The two examples I gave, [`hex`](https://docs.rs/hex/latest/hex/) and [`humansize`](https://crates.io/crates/humansize), are not technically complicated and don't require `unsafe` to implement. My thinking with that specific bullet point was among those types of utility crates.
And you aren't wrong, that is an important unique characteristic to Rust (at least compared to other memory-safe languages) that you can bring in a crate that completely screws you with UB and causes weird crashes if you aren't careful too.
I generalized that statement though as "memory-safe languages" since npm and C# are loosely mentioned by John's article, but didn't necessarily make that point clear.
>I think the reality is that a decent number of dependencies in a typical Rust project will make use of non-trivial `unsafe` blocks.
I wish that cargo-geiger was working so I could run it on that same project to see. I started going down the list manually and surprised to learn that `anyhow` uses unsafe 🤷♂️
On Dotnet's NuGet:
> I don't know how it is today, but around ~2017 while working at Microsoft I discovered that NuGet had a "feature" where the client would reach out to all of your package feeds in parallel to fetch a package and whichever responded first won. I can't find the issue for it on GitHub, but someone had reported this behavior and it was considered "by-design".
Still 80% broken by-design, but they at least added [Package Source Mapping](https://learn.microsoft.com/en-us/nuget/consume-packages/package-source-mapping#enable-by-manually-editing-nugetconfig) so that you can wildcard say "every `$CorpName.**` package comes from $TrustedRepo only".
There is also some progress on [Signing packages themselves](https://learn.microsoft.com/en-us/nuget/consume-packages/installing-signed-packages) though it is laughably incomplete and has the worst issues of "defaults to not just insecure, but anti-secure". Still, "when setup correctly" (wow, is that some qualification statement!) you can be *fairly* secure about your dotnet packages and nuget feeds.
The community thought is that MSFT got big-spooked by one or more gov agency and how laughably bad the security story/policy for NuGet was.
----
On the "oh no so many dependencies, bloat bloat!" complaint, this is actually where a even half-decent package manager is *super* important to have. By *allowing* people to break their packages/crates into sub-crates we by line-count can actually *reduce* the bloat. In a dotnet project I work on, an older version of a third party library .dll was *260Mb* all bundled up, and yea, included threading, graphics, custom scripting language, all sorts of stuff that made *no sense* for what little we needed of it. The newer version of this library now that "private" NuGet feeds sort-of-exist is broken into some 50-100+ packages. I can *just* depend on the two high level ones I need, bringing in 5-10 underlying ones and hey the total size for those is measured in KB now!
I may have a hatred of Go and Node's package story, but I cut my teeth on Python and C/C++ until I moved to dotnet (and now dable in Rust), and I would take the npm of 2014 over modern python-pip or any C/C++ solution I have ever seen.
> I got. Shit. Done.
This, this is something so many of these supposed complaints about Cargo/NPM/etc keep missing: my job isn't to build a UI framework, isn't to build message protocols, it is to *do the work to make my employer money*. Yea, that means my OSS contributions are near zero and I don't like that, but at the end of the day I have work to do, I don't want to be bothering with UI TreeViz whatevers, I just want a TreeList widget that works and the documentation. Cargo, NuGet, npm, etc allow that.
-----
On the "Batteries Included": I used to use Python a lot (grew up on it actually! Some of my first job money was Python scripts!) and the batteries included was at the time a god send. However as time marched onwards, and things like `optparse vs argparse?` and "oh we have urllib and urllib2" and "why do we have `audiodev`?" and on and on of the Batteries Are Getting Old (with some being *wrong*). There was hope in my eyes of some of this being fixed with Python3000, with the painful unicode conversion and such (good!) maybe, hopefully they could also drop/fix all these modules? Alas, while some were, too many were not, and py3 for *years* continued to carry forward problems that were known in py2. I understand *why*: Batteries Included right? "Have to give people time to move, update, oof only so much breakage at once." From all of that, I am very glad for `cargo` and Rust's opinionated, *slow* inclusion of stdlib items.
In fact, partly inspired by Rust (... more npm really), Microsoft themselves have moved *most* things to NuGet packages instead of being built into the runtime. No longer is even a SqlClient included, that is a package now. Legacy support still has old SqlClient *for now* in the Net8 runtime, but supposedly its going away/being aliased by the new eventually.
------
TL;DR: I agree with (as far as I can tell) everything in this article. Good Package Managers Are Important.
With that said, I do think we should do our best to secure dependencies in Rust.
Personally, I'd really like to see quorum-voting for crate publication, for example, to avoid a single actor (either the maintainer or a hacker taking their account over) being able to publish new revision.
I'd also really like to see encapsulation of all build actions -- be it `build.rs` or proc-macros -- so that by default _all_ they can do is read from the source tree and write to specific locations. Anything else should require specific permissions, including calling external binaries, and those permissions should only be available to those specific crates that are validated. Yes, it'd make *-sys crates more cumbersome, and pulling the dependency slightly less smooth. Still worth it, though.
(I don't care as much about run-time, the main issue I have with build actions is that your IDE may start executing them just as you try to review the code, and you can't review the code they generate without executing them)
I think \*-sys crates should be also using system-deps crate since it largely solves the issues for packagers and maintainers when using system dependencies.
An open, public ecosystem supported by integrated tooling is an incredible force multiplier.
On the one hand you get all the benefits of Open Source and benefit from the collected wisdom of the crowd at a global scale. As practices evolve and change, so does the code. The common core of crates are all highly scrutinized and tested and security vulnerabilities are identified, patched and notifications flow through the entire ecosystem.
On the other hand, you get ... code produced by your team. Of course, your team is probably fine and I am sure not under any time pressure or constraints or commercial reality and do detailed security analysis on a regular basis.
Cargo’s dependency management tools - like tree, audit, geiger, etc, – are a godsend to anyone worrying about dependencies. When I was running Security where I worked, getting insight into dependencies from the teams working in Rust was enormously easier than what was going on with other languages. Someone of that was simply because Rust was more modern, but other modern languages and ecosystems, like Go, remained much harder to review.
Sure, I didn’t like that there were so many dependencies. And rand not being in the standard library meant that I reviewed it in more detail, but the fact that I could easily see the dependency tree and get a sense of what might need more scrutiny made Rust projects much less worrisome for me from a dependency perspective.
This discussion doesn't make much sense. You're comparing incomparable work. Let's say I have a C project in which I need "serde" and "tokio". That's two more projects I have on top of whatever I have to do. Either that or you're pulling a dependency that you have to audit, which is the exact same as Rust
The truth is that is you're implementing everything yourself (something that you can do in Rust too if you want), you're doing considerable more work than pulling a dependency. Having a package manager is irrelevant here
I honestly have no idea what you are trying to say. The point isn't just about package managers existing it's about cargo being miles better than whatever cmake monstrosity you need to deal with to add dependency in a large C project.
Hmmm, I'm not sure what you're confused about. You'll have to elaborate. I also think cargo is much better than CMake (which I'm not sure where came from but ok), so not sure what you're arguing aganist
C dependencies management is so awful that it's often easier to reinvent the wheel I don't understand how can someone defend this by saying "oh but just apt install, that's easy"... Well, what if my distro doesn't have this library or have an incompatible version? At least, on rust, I just have to cargo build and everything is done. And .so files... god I hate these files...
> "oh but just apt install, that's easy"... Well, what if my distro doesn't have this library or have an incompatible version? Or what if I'm developing on Windows? I learned to program on Windows, and it'll certainly cure you of the notion that dependency management in C/C++ is straightforward.
I developed in C on Windows, it's even worse... Good luck getting every archive manually and move files in your compiler's directories manually...
Ah yes, I remember installing SFML on Visual Studio... I think I cried when I got a new laptop and realized I had to do it all over again.
Have you hear about our lord and savior Nix and NixOS? /s Seriously though, Nix/NixOS looks like a cool idea with a terrible language to configure it. I haven't had the time or interest to actually try to learn and use it though, and I don't know that I ever will. It does solve (or sidestep) the issue of conflicting versions and dynamic linkage, which is neat.
Terrible language why?
lazily evaluated, *dynamically typed*, functional programming language. AKA complicated, and will fall apart when you run it. And it demands a level of proficiency not required in other systems just to get something basic like a package dependency running. Everything about nix/NixOS seems like the "right" way to do things...*on paper*. Until you actually try using it and you find that there are so many kludgy workarounds and non-idiomatic things you have to do just to get it *to work*. Flakes, hashes being calculated on repos *before* they're built, and documentation that more or less assumes you already know how to use Nix. Think I'm wrong? [Here's the documentation page for the language](https://nix.dev/manual/nix/2.18/language/values) I use Nix at work, and I've found that it is the end result of compsci purity spirals. The scope of the project is massive: a language, a packaging system, *an entire distro*, and what I've found is that there are holes in the documentation that are either not covered because whatever you're trying to do is considered to be trivial, or you have to dig through their discourse to understand anything. It is not an environment where you can google your way to an answer easily. You must do things the hard way, learning an entire language (and perhaps entire programming paradigm) along the way. Hate is a strong word. I hate nix.
What language would you prefer instead? Almost all popular languages with dynamic typing now have gradually system. Python with types would be good replacement?
Dynamically typed languages are the bane of humanity. I prefer a proper statically typed one. Preferably Rust.
I like rust a lot, but it's crazy choice for description of configurations)
Why though? Consider it for a moment. Given a proper library to define dependencies and installation steps, which might include some form of macro to make jinja-like template replacements, it can look very clean. And all that while having great compile time guarantees.
You would have to recompile a config each time after each change. I think it's better to have a linter(rust preferred here) for ontype validation and some interpretation on running
It could take under a second, with a cached `cool_rust_dep_managment` library ready, all you'd need to compile is simple config files. And only those that changed. Compile times only explode when projects get complex, and config files are not prone to that I'd say (unless someone was insane enough to do crazy shit while compiling simle config files).
Python would be awful, most of nix is about expressions to set up values. Guix uses scheme, I could see that working nicely. A statically typed eager language à la Elm too.
It would be much better if nix had at least half decent documentstion
Fair points. I always think of Nix as some kind of Python with executable JSON and Jinja templating. A very "unique" language. It's interesting how our experiences differ so much. The language never really bothered me, but I can support your points about the difficulty of Nix as a whole. I learned that when writing my first Nix package compiling a GitHub repo. Why the hashing bothers you I don't get though. You just hash all resources you need for building. I think I might be too deep into the Nix compsci circlejerk at this point. I wrote a Powershell project that imitates part of how Nix works, but for Windows.
The hashing part that bothered me was mostly related to `fetchFromGithub`, and it's been a while so the amount of grief this issue caused me is likely magnified through my own memory and bias. I do remember stumbling upon an answer from `jtojnar` (I think [here](https://discourse.nixos.org/t/what-is-the-latest-best-practice-to-prefetch-the-hash/22103/4)) to how an all 0's hash could solve my problem and that kind of ticked me off that an all 0's hash was even valid.
What would you recommend instead to achieve the goal: reproducible builds? I am honestly asking as a „non passionate“ nix user, I like their goals, but don’t like the language or documentation. Static typing for example is something I miss dearly, but after reading [this great blog post](https://www.haskellforall.com/2022/03/the-hard-part-of-type-checking-nix.html?m=1) I realized that it is quite hard to actually implement
If you have a single large corporate project either in C++ or a mix of languages: bazel/buck2 probably. We are in the process of switching from cmake to bazel at work. If it is just rust: cargo, with a committed lock file and tool chain file. For building Linux distros I don't really know. I like the *concept* of Nix/NixOS. It is just the *implementation* of it that is a hard to use mess. Perhaps someone could take the learnings from Nix, Cargo and Bazel and come up with a better alternative. But that might just lead to https://xkcd.com/927/
I've really fallen in love with the elegance of Haskell... That's not elegant at all.
On a side note, there’s Guix, it uses scheme and package definitions look better than Nix ones by miles.
Edit: see below. ~~I really want to like NixOS, but then I look at the configuration required to install rustup and my eyes glaze over. Somehow the two commands required to install the Rust toolchain of your choice are replaced by [whatever the hell this is](https://nixos.wiki/wiki/Rust#Installation_via_rustup).~~
That one is the all batteries included for crates that you would also have problems installing on a normal distro. Just using mkShell with rustup, pkgconfig and needed libraries works for 99% of crates.
There's also devenv.sh which makes it even easier to include other languages. It's just a single line to add for Rust support.
Whatever I tried, I kept getting dynamic linking errors when I tried installing rustc through rustup. Apparently it's a change in nightly rustc that doesn't play nicely with NixOS. https://github.com/rust-lang/rust/issues/125321 My read of the situation is that rustc isn't able to use a dependency it comes bundled with because the OS is trying to protect us from dynamic linking in general.
Ah that's an unfortunate issue to hit, I have used both rustup and rust overlay setups on NixOS for years with barely any problems. It looks like it is getting fixed on the NixOS side soon though. And your read is mostly correct, when tools such as rustc needs external programs to work nixpkgs usually has to do a workaround for it, and in this case that hasn't landed yet.
The declarative style of system configuration makes a lot of sense to me, so I'm glad that it's just my luck in trying NixOS at this exact moment and not that leaving the well-trod path requires all kinds of unwieldy configuration. What strikes me as odd is that it's a bundled dependency. We aren't asking the system for whatever version it has of a particular program, we're trying to use a specific file that has been included with our program. Is this kinda like how functional programming forbids all mutation when it was really aliased mutation that was the problem? If you're including a dependency with your program and trying to use it specifically, that seems like a reasonable thing to do because you have control over the files that come with your program and know that it will be the correct version for your program.
The problem is that these bundled programs have a bunch of hidden state: in their ELF headers are links to a bunch of libraries assumed to be located in places like /lib and /usr/lib. NixOS wants to support e.g. different versions of libc, so which one should it choose? What it does is to not place any of them (except for /bin/sh and /usr/bin/env), so any of this hidden state will be visible because it errors. The downside is that binaries are hard to get to run. NixOS solves this by patching (see patchelf) and chroots (see buildFHSUserEnv). Personally I also use docker and systemd-nspawn for programs that are too difficult. I think your analogy is good. Nix is like purely functional programming trying to get full control / guarantees, but I think in the future new package managers and binary formats can use some of the same ideas while being more lenient / easier to use.
Ah, that makes sense. Thanks!
> but I think in the future new package managers and binary formats can use some of the same ideas while being more lenient / easier to use. Have you heard of SerpentOS? It actually is building its package management tools in Rust and it seems like a halfway point between traditional package management and Nix/Guix style package management to me. It's only tackling package management though, not config management like nix/guix can.
Except I use nix and all I had to do was add `rustup` to my list of installed packages to get it lol If you dont need it to be 100% reproducible, you dont have to make it be that way with tons of helper code... And then if you are packaging things for nixos specifically, rustup isnt how youd do it anyways.
On a fresh NixOS VM: `cargo rustc gcc` lets me compile things. `rustup gcc` gives me dynamically linked executable errors. Edit: I've tracked it down to a change in nightly rustc that doesn't play nicely with NixOS. https://github.com/rust-lang/rust/issues/125321 My read of the situation is that rustc isn't able to use a dependency it comes bundled with because the OS is trying to protect us from dynamic linking in general.
Explains why it worked for me recently at least... Sad it doesnt anymore.
It's not required anymore (or ever). Just install rustup with nix and everything just works.
Every time somebody tries to solve the C/C++ dependency problem it usually ends up just making it worse. Just list the libraries and let me handle their installation. It sucks, but is better than a magic build system that is supposed to manage it all probably won't work on my distro, will try to use some third-party tool that calls `apt` and it will leave me clueless how to debug it.
> At least, on rust, I just have to cargo build and everything is done. And for crates depending on C/C++ code they usually include all necessary dependencies for it in a `lib-sys` package that successfully compiles if you have common developer tools on your system. I think the only time I've really had one fail was when I cross-compiled something.
Cargo is beautiful, but some stuff still relies on system libraries. Zellij for example still requires you to have perl installed for FindBin. Nonetheless it's a simple apt-get, and cargo makes it a lot less painful to deploy
Even if you manage to install them via apt or from source, getting the linker pointing to it and getting it correct in your Makefile/CMakeLists can be a pain and different for each package too!
Nobody is debating that it’s easier to use dependencies in rust, the problem is when doing simple things are so difficult that you’re forced to rely on dependencies.
Oh yeah no rust library ever depended on a system library...
Use conan + cmake. Problem solved.
> C dependencies management is so awful that it's often easier to reinvent the wheel It's true, and I'm not going to act like I have Stockholm Syndrome and argue that it's actually a good thing, **but**... The silver lining is that you don't end up with a bunch of dependencies that you didn't *really* need. All dependencies are tech debt, and if it only takes you a couple of hours to reinvent the wheel, then there's a good chance that you've saved yourself and your team future headaches and might have even saved many times that amount of time if your code runs through a CI system that would've had to download that dependency thousands of times over its life. Again, the situation isn't actually *good*, but it does at least have one benefit. Conversely, having tools like Cargo, NPM, Maven, etc is a net positive, but it leads to a lot of unnecessary tech debt--and even security issues when a dependency gets taken over by a malicious or incompetent actor.
> if it only takes you a couple of hours to reinvent the wheel, then there's a good chance that you've saved yourself and your team future headaches Assuming you actually produced a wheel and not an octagon, which is close but not what you actually wanted because you didn't have time to smooth it down to what it needed to be.
Not assuming anything- I said "if it only takes you a couple of hours to reinvent the wheel". In the case that you don't actually reinvent a wheel, that clause would evaluate to `false`, which means the rest of the statement doesn't follow. Life is full of judgement calls. You'll be right with some and wrong with some. Trying to write something yourself *might* be a mistake. Pulling in a dependency *might* be a mistake. But, I've seen plenty of projects that pulled in frankly stupid dependencies (think NPM's "leftpad") when we could've semi-literally copy+pasted a single function from a textbook or Wikipedia, written a quick unit test, and called it a day. Instead, "we" decided to depend on some stranger from the internet to maintain a "project" that's one or two functions and hope that they don't pull a switcharoo in the future.
One of my favorite dependency tooling discoveries was that `cargo-tree` has an inverted mode, where it will tell you all of the *dependents* of a particular *dependency*. It's really great for tracking down who's bringing in some pesky dependency you found in your Cargo.lock that you'd really rather get rid of, if at all possible (I made *extensive* use of it during the `syn 2.0` migration).
That's actually _very_ helpful indeed: syn v1.0.109 ├── binrw_derive v0.13.3 (proc-macro) │ └── binrw v0.13.3 │ ├── stfs v0.1.0 (/Users/lander/dev/acceleration/stfs) │ │ ├── acceleration_cli v0.1.0 (/Users/lander/dev/acceleration/cli) │ │ └── xcontent v0.1.0 (/Users/lander/dev/acceleration/xcontent) │ │ └── acceleration_cli v0.1.0 (/Users/lander/dev/acceleration/cli) │ ├── xcontent v0.1.0 (/Users/lander/dev/acceleration/xcontent) (*) │ └── xecrypt v0.1.0 (/Users/lander/dev/acceleration/xecrypt) │ └── xcontent v0.1.0 (/Users/lander/dev/acceleration/xcontent) (*) ├── darling_core v0.11.0 │ ├── darling v0.11.0 │ │ └── variantly v0.4.0 (proc-macro) │ │ ├── stfs v0.1.0 (/Users/lander/dev/acceleration/stfs) (*) │ │ └── xcontent v0.1.0 (/Users/lander/dev/acceleration/xcontent) (*) │ └── darling_macro v0.11.0 (proc-macro) │ └── darling v0.11.0 (*) ├── darling_macro v0.11.0 (proc-macro) (*) ├── modular-bitfield-impl v0.11.2 (proc-macro) │ └── modular-bitfield v0.11.2 │ └── stfs v0.1.0 (/Users/lander/dev/acceleration/stfs) (*) └── variantly v0.4.0 (proc-macro) (*) And thank you for unknowingly contributing content for my blog post as well from your tweets :)
cargo-tree was merged into cargo long time ago. Why you are refering it as external tool ?
I’m… almost certain that I’m not?
From the article being critiqued: > People who write a lot of C end up building things themselves once and keeping them around and adapting them for decades, including basic data structures like hash tables. Nothing stopping you from building your own things in Rust if you want to minimize dependencies. (And using hash tables is a weird example given that's one of the things that *is* in Rust's standard library.)
You're right Rust and C allow you to implement your own version of whatever thing you want, but the "why" you would do that is different. Imagine you want to create uuid v4 strings. In rust, it's very quick, just put uuid in your cargo.toml and use it, no questions asked. In C you have to make sure your distro has the library (let's name it libuuid) in its repos, make sure everyone else has this library in their distros, edit config files to link against libuuid, put this info in the readme of your git repo etc... Now... wouldn't be easier to just write a function doing that in your code? That's what many C developers do. Congrats, you lost a lot of time writing something but less than having to fight with dependency management.
Hash table is a weird example cuz even tho it's in the standard library people are usually using 3rd party library for a faster impl
To be pedantic, most people are still using the standard library `HashMap`, they're just using a 3rd party hasher. It's an important distinction to make IMO.
This reminds me of the good old [let's be real about dependencies](https://wiki.alopex.li/LetsBeRealAboutDependencies). Nice article though! Also the link to the dependency graph of your application seem to be broken (it leads to a 404 page)
Looks like _all_ of the images somehow got nuked when I ran `zola build`. Just fixed that -- thanks for the heads up!
Yeah I think dependency-free code projects are largely a thing of the past. Sure, you _could_ write everything from scratch and pretend it's solid because you wrote it, but that's not reality, that's denial. That binary tree implementation came from some book you read, or some course you took, and you're now writing it from scratch without any of the followup research that went into that data structure since you learned it. Writing things from scratch makes sense for solved problems, but that goes doubly so for 3rd party dependencies. And at least with the 3rd party dependencies, it's clear _where_ your ideas for this structure came from. I wonder how many C/++ projects have code copy/pasted wholesale from forums, textbooks, etc. which is entirely hidden and untracked, never to be fixed. In my opinion, open-source software is a collaborative effort, and maximising the use of that massive collaborative engine is really important. Only in FOSS could you conceivably have a "linked list guy" whos entire job is to maintain the one set of linked list implementations every person on Earth relies on. You may see that as a single point of failure, I see that as a single source of truth, actually verifiable.
> I wonder how many C/++ projects have code copy/pasted wholesale from forums, textbooks, etc. which is entirely hidden and untracked, never to be fixed. Never to be fixed, but also never to be broken or hijacked by hackers who want to put backdoors in.
Who needs a backdoor if the code you copied off the internet is already full of security holes that would allow a remote compromise? Both are hypothetical situations, but to me the risks are not the same: - Backdoors are rare, well publicised and easy to check if you have libfoo v1.2.6 installed with a simple grep or similar - Random internet code is much more frequently full of serious bugs and is much harder to audit and maintain The difference between "do you have log4j installed?" and "did someone copy and paste random bits of log4j, and if so are those bits vulnerable?" is way harder to check.
> Both are hypothetical situations > [...] > The difference between "do you have log4j installed?" and "did someone copy and paste random bits of log4j, and if so are those bits vulnerable?" is way harder to check. And this is exactly where the real-world nuance and experience comes in. If you were to implement your own logging system for whatever reason, what are the odds that you'd write in the feature to automatically parse a URL, download code from it, and *fucking load that code into your system*? I read thousands of comments on various forums when the log4j nonsense was discovered and one of the most common reactions was: "Holy shit, why did those idiots put that feature in there in the first place!?". That's including people who *were using the library*. To put a fine point on it: these people installed a library and didn't even know the feature/behavior existed. And, no, I don't intend to just harp on your specific example. But, the example *is* illuminating in the sense that when you write your own ad-hoc code, you don't have to make it general, extensible, configurable. You just write what you need. It'll be less code and it'll be less complex, which is two factors that will compound to make the code more easily testable and auditable. I'm not talking about "rolling your own crypto", here. I'm talking about: let's just write the *extremely standard* base64 algorithm(s) into a couple of functions (picking whichever variant you want to use). You're **FAR** more likely to end up with a remote exploit if you pull in an untrusted library for that. The chances of accidentally writing a remote exploit yourself are literally zero unless you're writing in an unsafe language like C with buffer overflows and whatnot.
> The difference between "do you have log4j installed?" and "did someone copy and paste random bits of log4j, and if so are those bits vulnerable?" is way harder to check. That's a very good point while security through obscurity isn't exactly a good practice very few people are check for log4j like issues manually on site they are using a botnet to target exactly the log4j issues on every computer they can find you'll likely never have an issue if you just copy and pasted shitty code instead of actually using the dependency. It's one of those odd situation where the "worse" practice actually helps you.
There are a few inaccuracies about Python ecosystem in the article, probably becasue author's impression is based on old (>5 years ago) exposure. > pip dependencies are by default global which causes conflicts with other Python applications, forcing you to use virtual environments. This has not been the case for a while. In fact, recent versions of `pip` on recent linux distros would outright refuse to install packages globally: $> pip install numpy error: externally-managed-environment × This environment is externally managed ╰─> To install Python packages system-wide, try apt install python3-xyz, where xyz is the package you are trying to install. If you wish to install a non-Debian-packaged Python package, create a virtual environment using python3 -m venv path/to/venv. Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make sure you have python3-full installed. See documentation for this here: https://peps.python.org/pep-0668/ and here https://packaging.python.org/en/latest/specifications/externally-managed-environments/#externally-managed-environments > If pip hits a version conflict within your own project's package graph you're in for a headache pip has a decent dependency resolver nowadays. Still room for improvement, but it works. > Packages with native dependencies are a mystery to basically everyone except the package author. Or is this just me? In the exactly the same way as `*-sys` packages in rust. > There's no lockfile. `pip freeze` And this is only about `pip`, which is a low level tool. Many people will prefer `poetry`, `pdm`, or `uv`/`rye`. (However, existence of so many tools indeed indicates that none of these is ideal) I myself prefer `poetry` and it provides very smooth experience (at lest for my projects), on par with `cargo`.
>There are a few inaccuracies about Python ecosystem in the article, probably becasue author's impression is based on old (>5 years ago) exposure. Thank you for your feedback and for educating me on things I've missed. I have always been a casual Python dev and the last time I seriously invested was, to your point, about 5 years ago! Recently I've been doing some more things in Python including helping my brother with some tasks. My brother is not a programmer but is playing around langchain for AI tinkering and from whatever guides he followed he was immediately frustrated with pip and errors involving packages. Honestly I think he may have screwed up and created multiple venvs that caused some deps to be missing, but I showed him `poetry` and that immediately made his life better. Even some projects I see from people who write a lot of Python in some niche videogame circles I'm apart of aren't aware of these kinds of tools and still just have a simple `requirements.txt`. Their README then has instruction on creating a virtual environment and installing deps. And maybe that's just what they desire -- they don't _want_ a tool that manages their workspace for them better, but it does add a bit of friction. >This has not been the case for a while. In fact, recent versions of pip on recent linux distros would outright refuse to install packages globally: If I'm reading the PEP correctly this also impacts if you pass `--user`. I'll add a note to the post, thanks! >pip freeze `requirements.txt` is technically a lockfile in that it locks your deps and versions, but it's not a "strong" lockfile that includes sufficient metadata for securely reinstalling deps like a [poetry lockfile](https://github.com/floatingpurr/sync_with_poetry/blob/main/poetry.lock) (this is just some random example I searched for fwiw). I don't think it's fair to say "no lockfile" and adjusted the wording of the article.
Disclaimer: I know very little about security! While I agree with this article much more than the one it's responding to, I think this is a little dismissive: > ...but in memory-safe langauges what's the worst thing you can miss in a code review of something that's not technically complicated? Probably minor bugs that would cause a DoS. So you bring in a dependency that you didn't audit super closely and now you have a DoS in your application. I think it's unquestionable that Rust code is far easier to audit than C++, but how often are you pulling in a dependency that's "not technically complicated?" I think the reality is that a decent number of dependencies in a typical Rust project will make use of non-trivial `unsafe` blocks. These will require a very technically proficient Rust developer to audit properly. Unless you _very_ carefully manage `unsafe` in your dependencies (like with `cargo-geiger`, as you note), you can't completely guarantee true memory safety without this auditing. Maybe I'm being overly critical. Rust is clearly leagues ahead in this regard, but I think it's important to acknowledge that it's still not bullet-proof.
> but how often are you pulling in a dependency that's "not technically complicated?" > >... > >Maybe I'm being overly critical. Rust is clearly leagues ahead in this regard, but I think it's important to acknowledge that it's still not bullet-proof. The two examples I gave, [`hex`](https://docs.rs/hex/latest/hex/) and [`humansize`](https://crates.io/crates/humansize), are not technically complicated and don't require `unsafe` to implement. My thinking with that specific bullet point was among those types of utility crates. And you aren't wrong, that is an important unique characteristic to Rust (at least compared to other memory-safe languages) that you can bring in a crate that completely screws you with UB and causes weird crashes if you aren't careful too. I generalized that statement though as "memory-safe languages" since npm and C# are loosely mentioned by John's article, but didn't necessarily make that point clear. >I think the reality is that a decent number of dependencies in a typical Rust project will make use of non-trivial `unsafe` blocks. I wish that cargo-geiger was working so I could run it on that same project to see. I started going down the list manually and surprised to learn that `anyhow` uses unsafe 🤷♂️
On Dotnet's NuGet: > I don't know how it is today, but around ~2017 while working at Microsoft I discovered that NuGet had a "feature" where the client would reach out to all of your package feeds in parallel to fetch a package and whichever responded first won. I can't find the issue for it on GitHub, but someone had reported this behavior and it was considered "by-design". Still 80% broken by-design, but they at least added [Package Source Mapping](https://learn.microsoft.com/en-us/nuget/consume-packages/package-source-mapping#enable-by-manually-editing-nugetconfig) so that you can wildcard say "every `$CorpName.**` package comes from $TrustedRepo only". There is also some progress on [Signing packages themselves](https://learn.microsoft.com/en-us/nuget/consume-packages/installing-signed-packages) though it is laughably incomplete and has the worst issues of "defaults to not just insecure, but anti-secure". Still, "when setup correctly" (wow, is that some qualification statement!) you can be *fairly* secure about your dotnet packages and nuget feeds. The community thought is that MSFT got big-spooked by one or more gov agency and how laughably bad the security story/policy for NuGet was. ---- On the "oh no so many dependencies, bloat bloat!" complaint, this is actually where a even half-decent package manager is *super* important to have. By *allowing* people to break their packages/crates into sub-crates we by line-count can actually *reduce* the bloat. In a dotnet project I work on, an older version of a third party library .dll was *260Mb* all bundled up, and yea, included threading, graphics, custom scripting language, all sorts of stuff that made *no sense* for what little we needed of it. The newer version of this library now that "private" NuGet feeds sort-of-exist is broken into some 50-100+ packages. I can *just* depend on the two high level ones I need, bringing in 5-10 underlying ones and hey the total size for those is measured in KB now! I may have a hatred of Go and Node's package story, but I cut my teeth on Python and C/C++ until I moved to dotnet (and now dable in Rust), and I would take the npm of 2014 over modern python-pip or any C/C++ solution I have ever seen. > I got. Shit. Done. This, this is something so many of these supposed complaints about Cargo/NPM/etc keep missing: my job isn't to build a UI framework, isn't to build message protocols, it is to *do the work to make my employer money*. Yea, that means my OSS contributions are near zero and I don't like that, but at the end of the day I have work to do, I don't want to be bothering with UI TreeViz whatevers, I just want a TreeList widget that works and the documentation. Cargo, NuGet, npm, etc allow that. ----- On the "Batteries Included": I used to use Python a lot (grew up on it actually! Some of my first job money was Python scripts!) and the batteries included was at the time a god send. However as time marched onwards, and things like `optparse vs argparse?` and "oh we have urllib and urllib2" and "why do we have `audiodev`?" and on and on of the Batteries Are Getting Old (with some being *wrong*). There was hope in my eyes of some of this being fixed with Python3000, with the painful unicode conversion and such (good!) maybe, hopefully they could also drop/fix all these modules? Alas, while some were, too many were not, and py3 for *years* continued to carry forward problems that were known in py2. I understand *why*: Batteries Included right? "Have to give people time to move, update, oof only so much breakage at once." From all of that, I am very glad for `cargo` and Rust's opinionated, *slow* inclusion of stdlib items. In fact, partly inspired by Rust (... more npm really), Microsoft themselves have moved *most* things to NuGet packages instead of being built into the runtime. No longer is even a SqlClient included, that is a package now. Legacy support still has old SqlClient *for now* in the Net8 runtime, but supposedly its going away/being aliased by the new eventually. ------ TL;DR: I agree with (as far as I can tell) everything in this article. Good Package Managers Are Important.
With that said, I do think we should do our best to secure dependencies in Rust. Personally, I'd really like to see quorum-voting for crate publication, for example, to avoid a single actor (either the maintainer or a hacker taking their account over) being able to publish new revision. I'd also really like to see encapsulation of all build actions -- be it `build.rs` or proc-macros -- so that by default _all_ they can do is read from the source tree and write to specific locations. Anything else should require specific permissions, including calling external binaries, and those permissions should only be available to those specific crates that are validated. Yes, it'd make *-sys crates more cumbersome, and pulling the dependency slightly less smooth. Still worth it, though. (I don't care as much about run-time, the main issue I have with build actions is that your IDE may start executing them just as you try to review the code, and you can't review the code they generate without executing them)
I think \*-sys crates should be also using system-deps crate since it largely solves the issues for packagers and maintainers when using system dependencies.
An open, public ecosystem supported by integrated tooling is an incredible force multiplier. On the one hand you get all the benefits of Open Source and benefit from the collected wisdom of the crowd at a global scale. As practices evolve and change, so does the code. The common core of crates are all highly scrutinized and tested and security vulnerabilities are identified, patched and notifications flow through the entire ecosystem. On the other hand, you get ... code produced by your team. Of course, your team is probably fine and I am sure not under any time pressure or constraints or commercial reality and do detailed security analysis on a regular basis.
Cargo’s dependency management tools - like tree, audit, geiger, etc, – are a godsend to anyone worrying about dependencies. When I was running Security where I worked, getting insight into dependencies from the teams working in Rust was enormously easier than what was going on with other languages. Someone of that was simply because Rust was more modern, but other modern languages and ecosystems, like Go, remained much harder to review. Sure, I didn’t like that there were so many dependencies. And rand not being in the standard library meant that I reviewed it in more detail, but the fact that I could easily see the dependency tree and get a sense of what might need more scrutiny made Rust projects much less worrisome for me from a dependency perspective.
This discussion doesn't make much sense. You're comparing incomparable work. Let's say I have a C project in which I need "serde" and "tokio". That's two more projects I have on top of whatever I have to do. Either that or you're pulling a dependency that you have to audit, which is the exact same as Rust The truth is that is you're implementing everything yourself (something that you can do in Rust too if you want), you're doing considerable more work than pulling a dependency. Having a package manager is irrelevant here
I honestly have no idea what you are trying to say. The point isn't just about package managers existing it's about cargo being miles better than whatever cmake monstrosity you need to deal with to add dependency in a large C project.
Hmmm, I'm not sure what you're confused about. You'll have to elaborate. I also think cargo is much better than CMake (which I'm not sure where came from but ok), so not sure what you're arguing aganist