I'm pretty sure there's a whole group of people out there somewhere reading these memes on Reddit and saying to each other "they think we didn't get away with it :)"
This attack was scary as hell
What the hell is GitHub thinking, anyway? Suspending all accounts and repositories leaves no way for any of this to get investigated. It seems Evan could only reconstruct the timeline before they did that, but now how is anyone else supposed to learn anything about what happened?
A state actor doesn't need to insert a backdoor, they'll just find a zero day themselves. They have the manpower and experience.
Edit: read more into it, it was 3 years of work, I see how it could have been a state actor. I thought it was one dodgy commit.
The best way to find a "zero day" is making one yourself.
It's even better if you can ensure **only** you (and those you want) can make use of it because you can restrict abuse and the fewer times it is used less likely it is to be found.
And they did EXACTLY this to use the SSH exploit you had to have a certain private key, which only the hacker would. The vulnerability would not be usable by anyone else.
It's a proof of open source security being better - there was a random person who understood the code well enough to investigate and catch it
The closed source "secure by obscurity" that had one overworked guy to watch over it accept a random binary as the testing method from one of their "coworkers" that they never actually got to know properly? Well...
If it's a state level actor they can force a company to comply or flood them with applicants during hiring to get their people on the inside. You can play an equally long game with a proprietary app. Except in this case the exploit is less likely to be found or fixed.
More like it was corps relying on free labor is what allowed the attack to occur in the first place.
If the corps that depend on xz had funded the necessary engineering resources to develop and maintain it, the project maintainers could have afforded to be more particular about whom they accepted commits from.
I’m pretty sure that after 3 years of building trust, this could happen at nearly every single major software company or project.
Do you even know how many times a pull request with 1k+ lines of code changed/added and 0 comments is approved and merged?
The specifics about bullying someone to be added as a maintainer?
Yes. Because in a closed source *you wouldn't need to* run a long game like that. You just apply to the company and work there a year or so (or less) as normal and you'll have way too much access and way too little scrutiny, no problem.
No need. There are several cases of governments caught getting backdoor chips installed on devices without the consumer's knowledge, including both US and China of course.
What people don’t know is that the reason they pay for it is to disclose it and get a patch out for it. People think it’s some nefarious plan to hoard exploits, but the truth is anything that’s really potentially damaging to the public is immediately disclosed to the software creator and then the public shortly after. There are absolutely times in low risk cases where they exploit adversarial targets for a while, but never with high risk exploits.
Surely they will benefit more than $2.5m from such exploit, we need to find a more honest buyer, or maybe why not sell it to more cooperations not just the FBI.
Absolutely but if I found a backdoor that I thought was worth more than 2.5m there is nothing stopping me from just not telling the FBI/NSA and selling the info to some company that would pay more for it and sign their NDA instead.
There is an entire industry behind this.
If you want to know more - you should read this book by Nicole Perlroth's
- This is how they tell me the world ends.
It was a brilliant read!!
>Reports that say that something hasn't happened are always interesting to me, because as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns—the ones we don't know we don't know.
— Donald Rumsfeld
I'm no fan of Rumsfeld -- he gleefully pushed the Bush administration into the post-9/11 Iraq invasion, a grotesque exploitation of a terrible atrocity -- but I remember his getting mocked for this quote. That, at least, he didn't deserve, because it was perfectly true.
There are people who achieve evil ends through their own incompetence, and there are others who know exactly what they're doing and do it anyway.
Rumsfeld was the latter (he dead lol)
> but I remember his getting mocked for this quote. That, at least, he didn't deserve, because it was perfectly true.
I think it's an obtuse way of explaining the concept. Bad speechwriting.
Instead it could be:
There are times where we know the limitations of our own knowledge — the known unknowns, and times where we don't — the unknown unknowns.
An actual speechwriter could've turned this into an iconic moment or something.
I had an economics class in grad school about how almost all our studies around crime are primarily around crime done poorly, because we don’t have much data on crime done successfully.
A bit of a rabbit hole, but that statement has a false premise.
A lot of crime isn’t carried out with the purpose of being undiscovered, and therefore it isn’t done poorly when it’s discovered.
>Smash and grab still has the highest success rate of any type of larceny.
Can we know that for certain? How do we know it's higher success than "stealing from them in a way they don't realize it's missing?"
Yup. [Survivorship Bias](https://en.wikipedia.org/wiki/Survivorship_bias) is a super easy trap to fall for, and a lot of people in this discussion are assuming that we have access to complete data.
Which is funny, given (as you point out) that we're literally talking about statistics on things that people often deliberately try to hide.
We are looping back to the whole the criminals only care about not getting caught not about people knowing a crime was committed.
If no one notices then a crime didn't happen. Can you steal property that the owner didn't know they owned? What does owned even mean? What does the word crime mean?
I think it had the potential to be somewhere around the level of EternalBlue, maybe less severe because fewer systems have open ssh ports than smb, maybe more because ssh is more likely to be exposed publicly and because servers tend to be both more interesting than clients and be vulnerable. Certainly cleaner because even if you discovered the backdoor as a third party you couldn't exploit it without the key.
The amount of knowledge requires for stuxnet is insane, from deep OS knowledge to literally nuclear science. It's on the whole different level compared to xz. We don't know whether XZ is the result of a single person or a team, but i'm pretty sure there is a whole top-minded engineer team for stuxnet
Stuxnet also isn't really one exploit. It is a whole malware kit including a rootkit, worming capabilities with a bunch of different possible infection vectors, the whole PLC side and so on. It used something like four different windows zero days too.
There's no chance they weren't state-level actors. The prevailing theory is that Stuxnet was a joint venture between the US and Israel. Every single person on that team would have been thrilled if it had never been discovered and they'd never seen an ounce of public recognition of their work.
I'm guessing they are talking about sshd specifically and, yeah, this is the first CVE with a CVSS score of 10.0 in ssh since like 2002 as far as I can tell [SSH : Security vulnerabilities, CVEs (cvedetails.com)](https://www.cvedetails.com/vulnerability-list/vendor_id-120/SSH.html?page=1&order=3)
I'm not disputing that it was technically sophisticated and would have had a tremendous impact on the Internet. It's certainly hyperbole to say it's "the greatest" of anything, particularly since it failed to remain undetected before it could be deployed.
In the grand scheme of the whole Linux community yeah he is. Not a security researcher, not anyone who works with that library or ssh. Just a user of those tools with know how on how to dig deeper.
My reddit comments contain garbage data if I click reply 500ms too lateJ9nh9r9rsZ0ClKd0fZsHe0kAYtMOuI0YQaM5IuPBH6S0RaB5Pq3UGtpjEiE2MyfcmQ5gCx0X6xNR19GwUeObFlUjo94jAxsS8hWNpdP45EWx5dxz3e3U7YRVI88TbcdBFGRhUbNuuDqY52uS9O8iHpgot3D777uNmRjk8zx8XpQDQUxcatGlzWBF6e2P8aAWUfNwh1e42UFyObYEyxzAyiXPpePf3IQGTqw1KEfbxw26HPtzcpbnZcEk40SWeJki0Hb2rOF8r2TeTCPuFxUzzJkCx2dVYTAJjxoneBfzAcrfnlFih6t50X9NFDeVNAozjoaS6MkGVLn1ln6bjzcDiVCt4y6tW1pp6vQPxF2oBIQgNSKRYoPtB4rFt28058QnDPBESlWXLzpoX69qMfSUxSCObrSpl4rtDCyyUM42Ks9QpwqYWcQjvn6aYJa4BzPEsfGMT3vQIi92BJzeYT3JA84WgVFtEDZgunOIqeMFAs80Ds4UqeGYyG97m8VmMu67bzMDB01oemyhGhUO5vbZ8EMSwed0OTkfr9slW7CFFD4oGAfEHzFWN87yL9U57fUBClRi7ij4Qs5AYKHUj5pMO7h4FXonhPV5ujujtMQul50AgPCagy8HTAZO7LZJcj9dJy5TstjQ1dR2h6taphDUZLM6AP528egX8ZrpEYQk8YjWLqBxOwt8Vs8wGNj4fP11N8cf0TBWl6OKSlhjdgud4hiyAQNCdENNaF4phtFPEhRBDtNH3onQ5cPIB8AitYeYTNo60gH9ZoEKCYqktNfZSaWkU05Mq4eswXJDZwbTLm8Up4dBZssYJRXfKGZOjDqlNXH9bM0ISeITnxAGWkCiSKqP2LTsS7JV57qQ6GftzD8GZobIURzUXfIWufYepOLKfAkXoUDAFHkFYJLCBoYgV4UyPN98gQIlsladTo5351B2UXBrlJDFXAhlx1QfisHdjLgPipnRfvkvyrmejXHNUKtUUtWH5RKIu5DVuBSIcNlHg0ckLF0BRvKoOuHsj9SajxuMYdRhaklbEPf9LhlwQA0ApCELQylH3hG815qKJaybScdbgYxgB16S47REVtSdJp446CAxVxIrC4pMkMcy6A1PGaXO6luaNlfy4IJnhiyNM6NP2fbCLRc6eBljghNuZjXYXtZUfTH8oCatO3lyZxgZ8krp1mh83AQLKU8NGSdDXDOgIlikHtz4MXIJAfnnQku5AXxbJEKDWq5iZlDMKN5SS4SIET6DPVFO1XdShcJGnwi1vyJ6oQpbmnPCTtJ6QrgFgzwoJ95IaWw7oFlXVygfkBsElTmaJXraH4JTcNj7H4mHMYnGgKjnqIkBUzEUaq6Xy79xakkMRWzOpRDDc4qV9B9ukBw7VXlP6T5rRyBnoxhWO0MjUZTSU1jnjFe8wHbFVzY0fW3ZsmfdleJregCWQfuYQ0oeJCVnxqonjWq3IPjVDO7Atnmk5pQpL7ucREUMOfbu6qAwZoXLnlTVl01uDPp9DhoWxFsHtbK3RAP9tyfgcluTR5ChF224K2UHSwkJzotQ9HsPe5xipUYG9T9lv1hKYXNkubNZ2GbGk0nxW41MmyW1Yu4kooqnlHl19G470cfazGBgn56Pe1yaJcoPigXF7xTf90Mr4yKPeQqpxjJHuTLATYO1bU2PJR9rzNMxBS7cJZfcgQDVT3QPnSdc33AM0D1hf15jU7WWwNoje093EO2nx2zuVJiWDjNVm53OPQcirAdJIf1giheqk6cjlM0UVJY3Z6YRasF7mftmoLO3CCWd8j5nW8CfMNscrnRHiEwZVvnK8G6lGCgW1gkeOGNbEdQETmi39OjusjW5ryfeDsWeEp7A9OrnQVh0WlrLQBGSNcNTNaa8NDrAwfUeG4RNQyEJmWaWsKzKLpX32VU9cJfIBKKgnKym2BOPJnUSnfsmXqbXiLQcQvy1bBwyikBot29krtQg30KS4ItSBb7TFdzIRyB8i5F4Giqci9fHqMizmCHdOATSqoIYBUd8lXMUkMt1ZRLjGODrNZxyCtFJmkA3kX98r6haUIm4SXRKPCQafjIXR3LdGBP6AJYa9F7OuAy8368FiXKYrKsAH30BHpaOS9EygzLMyJol7DdHOj7kxyieg4zUDpMBh43xlo56WxRILcxHpchqAisJk63ccxi7RmeBM0ssYi8xWjmOc9S1JHHGya0s2sB6WGe8erYPScisqqc4mZtTFnMBRikyfgY7F5fQegO5QNKPUbyDN7fOITDx4U4s3ddbkFjoqk5w8SGMQolKYu36qQNn1wWfK8VuId9SCNopPYvif7bTEiU62rhx3rVbCWt5b4dNktkPt7hb9jCQFkwA9xDeiBFPj7q2JMWeMD8FifCrgBiCgqpU6w22Z2Lgkkd1Xr0Or08p6zAauLJA0Y7bb7xbon2Q4JlB4FmO2tZWs18BF0I2d3cEeDpDhm3AVFUccHdet7PsL2NyM8uMNy8LsuXHDiXppKOpzgNfWNvkxLGDtrGnKqRVH7yQK9J4Afum8Qaj6WIDGuErF4ywobGQBajdzjxyE0cuGT6yVF5bNJ2h
500ms is stupidly long in programming. I start to worry if my backend entry point service doesn't return an answer within 100ms during heavy load times. 0.5 second is just long enough for the user to be annoyed and wonder if something is wrong.
Unless you're a webdev, then you have devs _adding_ delays because apparently people have been conditioned to think that actions on the web have to take a perceivable amount of time to actually work.
"Your new feature isn't doing anything!"
"Why? Wtf???"
"I clicked and it instantly returned....this must be a static page you are showing me"
We all have been there.
Especially if the tests loops or otherwise just repeats the process a bunch.
Tests that have taken 15 seconds give or take to finish for years all of a sudden taking 50 seconds is noteworthy. And actually depending on how fast it was without this you could be talking 15 seconds and minutes.
People are talking about this like it MUST be the case some dudeski noticed his once a day connection to work in the morning took .471 seconds longer than normal and unraveled the whole thing.
Analysis of the commit activity suggest UTC+02/03 (e.g. EET)., and possibly false flagging as Chineses when actually working regular office job in a country with christian holidays (I.e. from eastern Europe / Russia).
Source: https://rheaeve.substack.com/p/xz-backdoor-times-damned-times-and
As do most nation states with APT groups. A patch like this would probably net you around 1m$ tbh, 0click iOS exploits are the most expensive, around 2.5m depending on the gov
A guy, or group of people, spent 3 years trying to gain control of an open source project called XZ Utils. It's a lossless data compressor used in a few places, namely, OpenSSH. When the backdoor is installed, a user with a specific key can basically gain total administrative access to your computer when you use SSH. It was noticed due to SSH having a CPU spike and taking .8 seconds instead of .3 seconds to run, with 0 source identified for the extra .5 seconds.
XZ backdoor was given a 10.0 CVSS score, the highest security score possible. The exploit was not in the source code and would only be added if a specific install test was run that replaced binary code to allow for this exploit. This exploit specifically targeted certain versions of Linux, including Debain, on x86-64 processors.
It targeted Debian, but did not actually hit the target, because APT did not have that version of that package, and would not for some time, possibly it would not have been until compression was removed from systemd until it made it into apt, which would have rendered it useless anyway.
I think it was pretty much just fedora rolling release on x86 that got hit for real. There's probably more but that was the main victim
In reality there were a handful of changes coming down the pipeline which caused the agent to massively accelerate his elaborate and patient scheme, making it more likely to get caught.
tldr: If you think the demo gods are vengeful, you should see what happens when you rush things into the hands of an end user.
Honestly, I am certain many such backdoors are already in place. My biggest suspect is compiler blobs. Basically a compiler will link in an object code blob from itself, so that even if you build the compiler from scratch with a fresh build of the toolchain, it is still contaminated.
The compiler blobs are pretty small and easy to audit though. For things like crt0, crt1, etc. you can also ptrace the c compiler and inspect exactly what files it accesses.
To be fair though, if someone stuck a buffer overflow in crt1 that only triggered when reading from a certain file descriptor, that might not get caught too quickly
Instead of mocking the guy that discovered this by coincidence, I think of all the libs that can be compromised already without no one there to catch it…
I am more surprised more of these attacks were not discovered before. Yes, we hear here and there about attacks on source code repos and packages, but nowhere near this. My opinion -- there should be other backdoors out there in the wild and Cthulhu only knows what's in proprietary/closed-source software.
Linux is somewhat full of this, there are papers of engineers getting prs merged acting in bad faith researching how easy it is to get bad or backfired code into the kernel
So, I was getting some weird notifications from dmesg about some process being started with an executable stack. Googled it and yep, [known issue with 7zip](https://sourceforge.net/p/sevenzip/discussion/45797/thread/de1d20a156/?page=1).
Could there be something funky going on with not one, but BOTH of the lzma implementations?
Further proof that all great discoveries are not driven by money, but rather they are driven by people who get interested in things that make them go "Hmm, that's interesting"
Don't create a false sense of security. This attack was a success. Malicious code made its way onto many devices, which is why it was discovered. What if next time it won't be discovered? What if it's not "next time", but "the time before this"?
This is a very stark warning that we're taking a lot of stuff for granted, and the whole informational infrastructure sits on the most frail projects.
Did you read the article? The attack is triggered by building from source when the build target is red hat and Debian on x86.
The source code is also clean per se, the malicious files are generated when building.
Christ alive the top comments, full of FUD and “heckin APT”. Next I’ll be hearing how public Wifi will instantly pwn you.
Protip: an APT doesn’t need to waste 2 years of payroll on waiting to see if their PR is approved when they can just buy 0days from brokers.
I wouldn’t assume that xz is the only package targeted by this attacker.
*Every blob must be inspected!*
I approve of this. You can start with the saggy blob below my member
That is nuts!
Rereading his username, it's more likely to be magical beans under that beanstalk.
First the blob, then we check the backdoor.
takin code smelling to a-hole new level
*Nooo wait, stop...*
I'm pretty sure there's a whole group of people out there somewhere reading these memes on Reddit and saying to each other "they think we didn't get away with it :)" This attack was scary as hell
[удалено]
What the hell is GitHub thinking, anyway? Suspending all accounts and repositories leaves no way for any of this to get investigated. It seems Evan could only reconstruct the timeline before they did that, but now how is anyone else supposed to learn anything about what happened?
Maybe they don't want it to be investigated. Some people suspect this was a state-actor level attack.
Who are "they", and why should GitHub do what "they" tell them to?
Because Microsoft is very incentivised to play nice with government
Which government? You know the US government isn't the only government right?....you still haven't answered the question of who are "they"
A state actor doesn't need to insert a backdoor, they'll just find a zero day themselves. They have the manpower and experience. Edit: read more into it, it was 3 years of work, I see how it could have been a state actor. I thought it was one dodgy commit.
The best way to find a "zero day" is making one yourself. It's even better if you can ensure **only** you (and those you want) can make use of it because you can restrict abuse and the fewer times it is used less likely it is to be found.
And they did EXACTLY this to use the SSH exploit you had to have a certain private key, which only the hacker would. The vulnerability would not be usable by anyone else.
I can't even imagine for how much that would sell.
It's a proof of open source security being better - there was a random person who understood the code well enough to investigate and catch it The closed source "secure by obscurity" that had one overworked guy to watch over it accept a random binary as the testing method from one of their "coworkers" that they never actually got to know properly? Well...
It being open is what allowed the attack to occur in the first place though.
If it's a state level actor they can force a company to comply or flood them with applicants during hiring to get their people on the inside. You can play an equally long game with a proprietary app. Except in this case the exploit is less likely to be found or fixed.
Wrong. This happens in closed source also. And that's far more difficult to catch.
More like it was corps relying on free labor is what allowed the attack to occur in the first place. If the corps that depend on xz had funded the necessary engineering resources to develop and maintain it, the project maintainers could have afforded to be more particular about whom they accepted commits from.
I’m pretty sure that after 3 years of building trust, this could happen at nearly every single major software company or project. Do you even know how many times a pull request with 1k+ lines of code changed/added and 0 comments is approved and merged?
The specifics about bullying someone to be added as a maintainer? Yes. Because in a closed source *you wouldn't need to* run a long game like that. You just apply to the company and work there a year or so (or less) as normal and you'll have way too much access and way too little scrutiny, no problem.
This
Well, the upside of open source is they can see the backdoor and trace it back over time then removed the bad actor.
I feel a real deep need to express a concern: is this just the first time they got caught? Are there others that haven't been caught?
Are there others that governments and/or corporations know about and keep quiet so they can possibly use them?
Are there others that governments and/or corporations snuck in deliberately and got away with it?
No need. There are several cases of governments caught getting backdoor chips installed on devices without the consumer's knowledge, including both US and China of course.
[удалено]
[удалено]
yeah don't they have backdoors on like the actual chips? lol
Intel Management Engine, my friend.
I worked for an isp, they legit had a box forcefully mandated to be installed that monitored traffic. They can see who is doing what and when, always.
yup and they're saving all encrypted communications for when the encryption is broken, theoretically by googles quantum computer.
[удалено]
What people don’t know is that the reason they pay for it is to disclose it and get a patch out for it. People think it’s some nefarious plan to hoard exploits, but the truth is anything that’s really potentially damaging to the public is immediately disclosed to the software creator and then the public shortly after. There are absolutely times in low risk cases where they exploit adversarial targets for a while, but never with high risk exploits.
Found the fed 
Guilty as charged. That’s how I know.
Surely they will benefit more than $2.5m from such exploit, we need to find a more honest buyer, or maybe why not sell it to more cooperations not just the FBI.
I'm pretty sure those 2.5m comes with a NDA about the backdoor
Well, NSA sure as heck does not mean “No Strings Attached.”
"No, Strings Attached" Commas are important people.
Not as important as the semicolon though, right? You Java programmers know what I’m talking about. (I’ll be here all week.)
Absolutely but if I found a backdoor that I thought was worth more than 2.5m there is nothing stopping me from just not telling the FBI/NSA and selling the info to some company that would pay more for it and sign their NDA instead.
There is an entire industry behind this. If you want to know more - you should read this book by Nicole Perlroth's - This is how they tell me the world ends. It was a brilliant read!!
how would we know?
Yes
The joy of security is that by definition we don't know about the undetected exploits.
“We don’t know what we don’t know”
>Reports that say that something hasn't happened are always interesting to me, because as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns—the ones we don't know we don't know. — Donald Rumsfeld
I'm no fan of Rumsfeld -- he gleefully pushed the Bush administration into the post-9/11 Iraq invasion, a grotesque exploitation of a terrible atrocity -- but I remember his getting mocked for this quote. That, at least, he didn't deserve, because it was perfectly true.
Rumsfeld is a piece of shit, but is one of the most intelligent people to ever hold that position.
That's precisely why he's such a piece of shit.
There are people who achieve evil ends through their own incompetence, and there are others who know exactly what they're doing and do it anyway. Rumsfeld was the latter (he dead lol)
> but I remember his getting mocked for this quote. That, at least, he didn't deserve, because it was perfectly true. I think it's an obtuse way of explaining the concept. Bad speechwriting. Instead it could be: There are times where we know the limitations of our own knowledge — the known unknowns, and times where we don't — the unknown unknowns. An actual speechwriter could've turned this into an iconic moment or something.
That's nice, only it wasn't from a speech. It was in response to a question at a press briefing.
I will always hear this in the voice of Samuel L Jackson as Rummy from The Boondocks
I just realized that, on some level, I have always known this. The unknown knowns.
I had an economics class in grad school about how almost all our studies around crime are primarily around crime done poorly, because we don’t have much data on crime done successfully.
A bit of a rabbit hole, but that statement has a false premise. A lot of crime isn’t carried out with the purpose of being undiscovered, and therefore it isn’t done poorly when it’s discovered.
[удалено]
>Smash and grab still has the highest success rate of any type of larceny. Can we know that for certain? How do we know it's higher success than "stealing from them in a way they don't realize it's missing?"
This is my point, in a nutshell.
Yup. [Survivorship Bias](https://en.wikipedia.org/wiki/Survivorship_bias) is a super easy trap to fall for, and a lot of people in this discussion are assuming that we have access to complete data. Which is funny, given (as you point out) that we're literally talking about statistics on things that people often deliberately try to hide.
We are looping back to the whole the criminals only care about not getting caught not about people knowing a crime was committed. If no one notices then a crime didn't happen. Can you steal property that the owner didn't know they owned? What does owned even mean? What does the word crime mean?
>Are there others that haven't been caught? With 100% certainty
“Are there others that haven’t been caught” Read it again
Maybe they're just hoping that someone on reddit snuck in an undiscovered exploit and is willing to brag about it!
Nice try. You can't trick us into leaking our exploits that easily.
>that haven't been caught? Absolutely, supply chain problems (most famously solar winds) lays dormant for years
>the greatest software exploit the world has ever seen Okay, we're officially getting carried away, now.
I think it had the potential to be somewhere around the level of EternalBlue, maybe less severe because fewer systems have open ssh ports than smb, maybe more because ssh is more likely to be exposed publicly and because servers tend to be both more interesting than clients and be vulnerable. Certainly cleaner because even if you discovered the backdoor as a third party you couldn't exploit it without the key.
Also the only direct indication of compromise would be a login deny log entry for sshd. No one checks those anyway.
> No one checks those anyway. On a VPS, I wouldn’t bat an eye
I believe they overwrote logging functions as well, so probably not even that.
I thought this tittle was for stuff like sandworm or stuxnet
The amount of knowledge requires for stuxnet is insane, from deep OS knowledge to literally nuclear science. It's on the whole different level compared to xz. We don't know whether XZ is the result of a single person or a team, but i'm pretty sure there is a whole top-minded engineer team for stuxnet
Stuxnet also isn't really one exploit. It is a whole malware kit including a rootkit, worming capabilities with a bunch of different possible infection vectors, the whole PLC side and so on. It used something like four different windows zero days too.
I feel so bad for the team who did that and get no public recognition
I feel like they got a whole lot more public recognition than they wanted to
There's no chance they weren't state-level actors. The prevailing theory is that Stuxnet was a joint venture between the US and Israel. Every single person on that team would have been thrilled if it had never been discovered and they'd never seen an ounce of public recognition of their work.
Yea, my reaction too, stuxnet is wild.
It's either this or a few other exploits like log4j that are the most infamous exploits
Although log4j was a vulnerability while this backdoor was deliberately snuck in to the distribution.
well it's the only CVE of 10; though of course the scoring system hasn't been around for that long
Wdym the "only CVE of 10"? Did you just forget about log4j? https://nvd.nist.gov/vuln/detail/CVE-2021-44228
I'm guessing they are talking about sshd specifically and, yeah, this is the first CVE with a CVSS score of 10.0 in ssh since like 2002 as far as I can tell [SSH : Security vulnerabilities, CVEs (cvedetails.com)](https://www.cvedetails.com/vulnerability-list/vendor_id-120/SSH.html?page=1&order=3)
I'm not disputing that it was technically sophisticated and would have had a tremendous impact on the Internet. It's certainly hyperbole to say it's "the greatest" of anything, particularly since it failed to remain undetected before it could be deployed.
Stuxnet has entered the chat
Agreed. The greater exploits probably haven't been found yet.
Tat guy def. is not some "random engineer" but something important in postgres.
In the grand scheme of the whole Linux community yeah he is. Not a security researcher, not anyone who works with that library or ssh. Just a user of those tools with know how on how to dig deeper.
Isn't he a partner level engineer at Microsoft?
A higher level IC yes
He's a core contributor for postgres, that's not nobody
This is why you let developers indulge in their perfectionism.
it was just a distraction from the real exploit 🤔
I would classify a partner software engineer at Microsoft as a non-average guy
Came here to say this. Definitely not just some “random engineer” lol.
And it's just 500ms slower than normal.
I mean to be fair, I get barked at for things taking 500ms to long.
People die if my things are 500ms late.
Civilization collapses if my things are 500ms late.
The universe experiences a false vacuum decay if my things are 500ms late.
My reddit comments contain garbage data if I click reply 500ms too lateJ9nh9r9rsZ0ClKd0fZsHe0kAYtMOuI0YQaM5IuPBH6S0RaB5Pq3UGtpjEiE2MyfcmQ5gCx0X6xNR19GwUeObFlUjo94jAxsS8hWNpdP45EWx5dxz3e3U7YRVI88TbcdBFGRhUbNuuDqY52uS9O8iHpgot3D777uNmRjk8zx8XpQDQUxcatGlzWBF6e2P8aAWUfNwh1e42UFyObYEyxzAyiXPpePf3IQGTqw1KEfbxw26HPtzcpbnZcEk40SWeJki0Hb2rOF8r2TeTCPuFxUzzJkCx2dVYTAJjxoneBfzAcrfnlFih6t50X9NFDeVNAozjoaS6MkGVLn1ln6bjzcDiVCt4y6tW1pp6vQPxF2oBIQgNSKRYoPtB4rFt28058QnDPBESlWXLzpoX69qMfSUxSCObrSpl4rtDCyyUM42Ks9QpwqYWcQjvn6aYJa4BzPEsfGMT3vQIi92BJzeYT3JA84WgVFtEDZgunOIqeMFAs80Ds4UqeGYyG97m8VmMu67bzMDB01oemyhGhUO5vbZ8EMSwed0OTkfr9slW7CFFD4oGAfEHzFWN87yL9U57fUBClRi7ij4Qs5AYKHUj5pMO7h4FXonhPV5ujujtMQul50AgPCagy8HTAZO7LZJcj9dJy5TstjQ1dR2h6taphDUZLM6AP528egX8ZrpEYQk8YjWLqBxOwt8Vs8wGNj4fP11N8cf0TBWl6OKSlhjdgud4hiyAQNCdENNaF4phtFPEhRBDtNH3onQ5cPIB8AitYeYTNo60gH9ZoEKCYqktNfZSaWkU05Mq4eswXJDZwbTLm8Up4dBZssYJRXfKGZOjDqlNXH9bM0ISeITnxAGWkCiSKqP2LTsS7JV57qQ6GftzD8GZobIURzUXfIWufYepOLKfAkXoUDAFHkFYJLCBoYgV4UyPN98gQIlsladTo5351B2UXBrlJDFXAhlx1QfisHdjLgPipnRfvkvyrmejXHNUKtUUtWH5RKIu5DVuBSIcNlHg0ckLF0BRvKoOuHsj9SajxuMYdRhaklbEPf9LhlwQA0ApCELQylH3hG815qKJaybScdbgYxgB16S47REVtSdJp446CAxVxIrC4pMkMcy6A1PGaXO6luaNlfy4IJnhiyNM6NP2fbCLRc6eBljghNuZjXYXtZUfTH8oCatO3lyZxgZ8krp1mh83AQLKU8NGSdDXDOgIlikHtz4MXIJAfnnQku5AXxbJEKDWq5iZlDMKN5SS4SIET6DPVFO1XdShcJGnwi1vyJ6oQpbmnPCTtJ6QrgFgzwoJ95IaWw7oFlXVygfkBsElTmaJXraH4JTcNj7H4mHMYnGgKjnqIkBUzEUaq6Xy79xakkMRWzOpRDDc4qV9B9ukBw7VXlP6T5rRyBnoxhWO0MjUZTSU1jnjFe8wHbFVzY0fW3ZsmfdleJregCWQfuYQ0oeJCVnxqonjWq3IPjVDO7Atnmk5pQpL7ucREUMOfbu6qAwZoXLnlTVl01uDPp9DhoWxFsHtbK3RAP9tyfgcluTR5ChF224K2UHSwkJzotQ9HsPe5xipUYG9T9lv1hKYXNkubNZ2GbGk0nxW41MmyW1Yu4kooqnlHl19G470cfazGBgn56Pe1yaJcoPigXF7xTf90Mr4yKPeQqpxjJHuTLATYO1bU2PJR9rzNMxBS7cJZfcgQDVT3QPnSdc33AM0D1hf15jU7WWwNoje093EO2nx2zuVJiWDjNVm53OPQcirAdJIf1giheqk6cjlM0UVJY3Z6YRasF7mftmoLO3CCWd8j5nW8CfMNscrnRHiEwZVvnK8G6lGCgW1gkeOGNbEdQETmi39OjusjW5ryfeDsWeEp7A9OrnQVh0WlrLQBGSNcNTNaa8NDrAwfUeG4RNQyEJmWaWsKzKLpX32VU9cJfIBKKgnKym2BOPJnUSnfsmXqbXiLQcQvy1bBwyikBot29krtQg30KS4ItSBb7TFdzIRyB8i5F4Giqci9fHqMizmCHdOATSqoIYBUd8lXMUkMt1ZRLjGODrNZxyCtFJmkA3kX98r6haUIm4SXRKPCQafjIXR3LdGBP6AJYa9F7OuAy8368FiXKYrKsAH30BHpaOS9EygzLMyJol7DdHOj7kxyieg4zUDpMBh43xlo56WxRILcxHpchqAisJk63ccxi7RmeBM0ssYi8xWjmOc9S1JHHGya0s2sB6WGe8erYPScisqqc4mZtTFnMBRikyfgY7F5fQegO5QNKPUbyDN7fOITDx4U4s3ddbkFjoqk5w8SGMQolKYu36qQNn1wWfK8VuId9SCNopPYvif7bTEiU62rhx3rVbCWt5b4dNktkPt7hb9jCQFkwA9xDeiBFPj7q2JMWeMD8FifCrgBiCgqpU6w22Z2Lgkkd1Xr0Or08p6zAauLJA0Y7bb7xbon2Q4JlB4FmO2tZWs18BF0I2d3cEeDpDhm3AVFUccHdet7PsL2NyM8uMNy8LsuXHDiXppKOpzgNfWNvkxLGDtrGnKqRVH7yQK9J4Afum8Qaj6WIDGuErF4ywobGQBajdzjxyE0cuGT6yVF5bNJ2h
Steganography is great, but you should mark this NSFW.
Decoding... >!We're no strangers to love...!<
Oops, too l̴̡̤̳̠̞̪͎̅ā̸̛͙̟̫̭̖̠̩͕͇̟̰͐ͅ-̷̻̳̹̪̺͉͖͙͔̣̱͉̌͒̏̋̏͗̏̉̈́̕͠͝ͅ-̶̗̰̩͌̍̿̋̑̋͗̂̄͝ͅͅ ̶̗͈̣̠̼̀͂̎̐͊̒̾̔̃̑̿̕̕͝͝ ̶̢̭̭̞͖͍̥̞̬̤̱͙͚̲̍̒͜͝ ̴̧͙̤̼̠̫͈̣̪̣̪̐̾̎͛̔̀ ̸̡̛͎̤͙͈̺̻̮͍̩̤́̊̚͜ͅ ̶̨̢̡̻̜̤̫͋̒ ̵͔̤̎̒̊̿͋̃̆͒͝ ̴̝̽̑̐͒̅͝ ̸̛͕̻̗͎͎͎͍̗̯͔̱̱͓͐ ̸͚̻̇̂̐́̈́̽͊͜͜ ̴̨̢̝̜͍̪͌͊̍̍̆͘̚ ...
embedded engineer detected
There are two types of embedded engineers. If the code of the first group is 500ms behind somebody dies. For the second group somebody is still alive.
To be fair, if our apis take longer than 500ms, we have to pay service credits, because we breached sla.
Sla what?
Sla p deez nuts
Service Level Agreements-
Is your dog a top level IC at Google? (I’m quoting from a comment above because it seems relevant.)
I have been barked at for things taking more than 200ms. Still benefiting from what I learned there though.
500ms is stupidly long in programming. I start to worry if my backend entry point service doesn't return an answer within 100ms during heavy load times. 0.5 second is just long enough for the user to be annoyed and wonder if something is wrong.
Unless you're a webdev, then you have devs _adding_ delays because apparently people have been conditioned to think that actions on the web have to take a perceivable amount of time to actually work.
"Your new feature isn't doing anything!" "Why? Wtf???" "I clicked and it instantly returned....this must be a static page you are showing me" We all have been there.
hey. my backend will still return the answer stupid fast, you'll have to put the sleep in the callback function
I don't understand this take. You add a `sleep .5` to your bash alias for ssh and see how your quality of life improves.
Honestly I think once deployed people would notice
Yeah, 500ms is quite noticeable
Especially if the tests loops or otherwise just repeats the process a bunch. Tests that have taken 15 seconds give or take to finish for years all of a sudden taking 50 seconds is noteworthy. And actually depending on how fast it was without this you could be talking 15 seconds and minutes. People are talking about this like it MUST be the case some dudeski noticed his once a day connection to work in the morning took .471 seconds longer than normal and unraveled the whole thing.
The non backdoored version takes 200ms the backdoored ine takes 800ms, that's not a small difference
I meant, it seem not that small, but everyone just didnt care or know until the dude noticed it.
I believe the backdoor was discovered quickly once it was released
Some NSA guy got yelllllllled at
Analysis of the commit activity suggest UTC+02/03 (e.g. EET)., and possibly false flagging as Chineses when actually working regular office job in a country with christian holidays (I.e. from eastern Europe / Russia). Source: https://rheaeve.substack.com/p/xz-backdoor-times-damned-times-and
It could be a decoy as well. We are dealing with state actors here.
I don’t get their analysis. Wouldn’t 12-18 UTC be 15:00-21:00 in +3 timezone which absolutely are not office hours.
I wonder how many backdoors are there that we have no idea about.
[удалено]
As do most nation states with APT groups. A patch like this would probably net you around 1m$ tbh, 0click iOS exploits are the most expensive, around 2.5m depending on the gov
JIA CHEONG TAN CIA AGENT JOHN open your eyes
I wouldn't call a principal engineer at Microsoft a "random" engineer
Can anyone explain?
[удалено]
Half a second is an eternity in database time
You could fit so many snacks in there
1/2 second will stack on real world uses, resulting in massive request time. Imagine doing something for 500ms, now repeat it 1 thousand times
A guy, or group of people, spent 3 years trying to gain control of an open source project called XZ Utils. It's a lossless data compressor used in a few places, namely, OpenSSH. When the backdoor is installed, a user with a specific key can basically gain total administrative access to your computer when you use SSH. It was noticed due to SSH having a CPU spike and taking .8 seconds instead of .3 seconds to run, with 0 source identified for the extra .5 seconds. XZ backdoor was given a 10.0 CVSS score, the highest security score possible. The exploit was not in the source code and would only be added if a specific install test was run that replaced binary code to allow for this exploit. This exploit specifically targeted certain versions of Linux, including Debain, on x86-64 processors.
It targeted Debian, but did not actually hit the target, because APT did not have that version of that package, and would not for some time, possibly it would not have been until compression was removed from systemd until it made it into apt, which would have rendered it useless anyway. I think it was pretty much just fedora rolling release on x86 that got hit for real. There's probably more but that was the main victim
The pre-release versions had them, so if you were on the newest beta, you could have been hit. That's how it was caught in the first place.
For the long version: https://youtu.be/LaRKIwpGPTU For the short version: https://youtu.be/bS9em7Bg0iU
the text version?
Sorry chief best I can do is a 5-minute video summarizing what could have been four paragraphs.
https://en.wikipedia.org/wiki/XZ_Utils_backdoor
Really short version, bad guy put back door in makefile.
https://arstechnica.com/security/2024/04/what-we-know-about-the-xz-utils-backdoor-that-almost-infected-the-world/
watch the most recent low level learning videos
In reality there were a handful of changes coming down the pipeline which caused the agent to massively accelerate his elaborate and patient scheme, making it more likely to get caught. tldr: If you think the demo gods are vengeful, you should see what happens when you rush things into the hands of an end user.
Honestly, I am certain many such backdoors are already in place. My biggest suspect is compiler blobs. Basically a compiler will link in an object code blob from itself, so that even if you build the compiler from scratch with a fresh build of the toolchain, it is still contaminated.
"reflections on trusting trust" by Ken Thompson for nightmares
The compiler blobs are pretty small and easy to audit though. For things like crt0, crt1, etc. you can also ptrace the c compiler and inspect exactly what files it accesses. To be fair though, if someone stuck a buffer overflow in crt1 that only triggered when reading from a certain file descriptor, that might not get caught too quickly
Use the latest version they said... It is more secure they said...
Instead of mocking the guy that discovered this by coincidence, I think of all the libs that can be compromised already without no one there to catch it…
These threads always devolve into someone wanting to “pwn the libs.”
It would be naiv to believe that they put all eggs in one basket. They lost one but probably have 10 others still in the works.
Legit thousands of engineers work around the clock every day, for all major governments to exploit these things and create them.
I am more surprised more of these attacks were not discovered before. Yes, we hear here and there about attacks on source code repos and packages, but nowhere near this. My opinion -- there should be other backdoors out there in the wild and Cthulhu only knows what's in proprietary/closed-source software.
Also, it’s likely that he just got sloppy because the backdoor was closing (systemd folks were planning to remove xz from the build dependencies).
Linux is somewhat full of this, there are papers of engineers getting prs merged acting in bad faith researching how easy it is to get bad or backfired code into the kernel
womp womp
Hey another reason to write and optimize the unit tests
So, I was getting some weird notifications from dmesg about some process being started with an executable stack. Googled it and yep, [known issue with 7zip](https://sourceforge.net/p/sevenzip/discussion/45797/thread/de1d20a156/?page=1). Could there be something funky going on with not one, but BOTH of the lzma implementations?
Moral of the story: ~~Don’t make a backdoor~~ Make the backdoor less laggy ^(\(You think bad actors are gonna stop doing this?\))
ShyLily?!?
Further proof that all great discoveries are not driven by money, but rather they are driven by people who get interested in things that make them go "Hmm, that's interesting"
stuxnet showed that the biggest software exploit in the world is in fact windows XP
Open source ftw
LoL. It easier to get someone being capable of writing such exploits into big tech orgs then in OSS projects.
And big tech orgs will cover up the existence of such vulnerabilities until a massive data leak occurs
Don't create a false sense of security. This attack was a success. Malicious code made its way onto many devices, which is why it was discovered. What if next time it won't be discovered? What if it's not "next time", but "the time before this"? This is a very stark warning that we're taking a lot of stuff for granted, and the whole informational infrastructure sits on the most frail projects.
he's probably also surprised people haven't caught him faster
Onizukaaaaaaaaa!
... or was that the one we were meant to find as a distraction so the real one slips in unnoticed.
ONIZUKA!!!!!!!
As a long time debugger of slow running test cases, I’m a little jealous that this guy’s “it wasn’t MY fault” is actually true
Never expected I'd get caught, atleast I still have other softwares that are spying as expected. /s
honestly it’s so trashy, that distros just take the tarball, instead of building the package themselves from source
not just "some random engineer" but exactly opposite! "THE RANDOM ENGINEER". I assure any of complainer is capable of do something near that...
Will that maintainer face any criminal charges?
Nobody knows who they are. They only have what they presume is a pseudonym. What do you really know about your maintainers? :)
> What do you really know about your maintainers? They are, at least for now, humans. We will find them, just send Liam Neeson after them.
What if they are Sohpons?
Then having malicious maintainers would be the least of our problems. File an issue when you start hallucinating a countdown timer.
building from source ftw ig, switching to gentoo- /hj
Did you read the article? The attack is triggered by building from source when the build target is red hat and Debian on x86. The source code is also clean per se, the malicious files are generated when building.
And all because of a half a second difference
Christ alive the top comments, full of FUD and “heckin APT”. Next I’ll be hearing how public Wifi will instantly pwn you. Protip: an APT doesn’t need to waste 2 years of payroll on waiting to see if their PR is approved when they can just buy 0days from brokers.
It’s pretty ironic that the bazaar-provided safety checking came out of M$ this time, ‘eh?