So the people running the systems that underlie our lives are incompetent, and don’t know how to keep them running properly. I’m supposed to be reassured by that?
Conservatives very consistently destroy everything they touch. It’s almost like they have no place in a modern society.
Yes and they have always been incompetent. The only difference is that you are finally starting to realize that society is the result of accepting the bare minimum and continuing on.
The person who made your food at the restaurant: bare minimum
The doctor who performed your surgery: bare minimum
The police officer who saved the child: bare minimum
The politician who was elected: bare minimum
The law that was passed: bare minimum
If you want things to get better, raise the bare minimum and then pay what it costs.
People rarely choose to pay to raise the bare minimum. That is where we are. Our ancestors didn’t have a choice; we just did it because we are selfish and stupid.
it’s not reassuring, but it’s probably due to greed-based incompetence
Sorry, I’ve worked in places of large scale an order of magnitude smaller than MS - the layers of control, redundancy, oversight, etc - I don’t buy it was a simple mistake.
Would you buy that it’s a series of complicated mistakes?
Back when I worked at Microsoft, they were converting all of the testers into telemetry and data engineers because everyone can test their own code and we can test shit in prod with AB testing and telemetry. More recently they pushed the engineering teams to use AI to write as much code as possible. They have also pushed out their most experienced engineers with early retirement offers.
All of these strategic decisions could plausibly add up to a perfect storm of fuck ups, big and small, complicated and simple, that form like Voltron into the mother of all fuck ups.
Back when I worked at Microsoft, they were converting all of the testers into telemetry and data engineers because everyone can test their own code and we can test shit in prod with AB testing and telemetry.
As in one set of Windows users got the “A” updates and one set got the “B” updates?
Yeah. It’s a common term in software development. Basically for any software that gets updated over the network you can setup a system to give the update to some percentage of users and then gather data on their usage and if it looks good, roll out the changes to more users. It is a very useful tool but should not be a replacement for common sense local testing.
Cool cool cool cool cool.
So if there are two windows computers hooked up to, I don’t know, some commercial freezers, and the only requirement is that they have the latest updates installed, one computer might have the “A” updates and one has the “B”?
Doesn’t have to be freezers in the example; it could be industrial sawmills, or computers running Labview.
Yeah. In practice with Windows the machines would all get the same update but Windows Update might say it’s availalable for each machine at a different time, depending on the roll out schedule. I think most of the experimentation happens on machines where the user opts into the Windows Insider program. But I haven’t paid close attention to Windows updates in years so take Windows specific details with a huge grain of salt.
In general though, for some random software they might release an A version witha ad at the top of the screen and a B version with the ad at the bottom and then use telemetry to figure out which version is the most effective.
greed-based incompetence
I don’t buy it was a simple mistake.
Incompetence scales well, though. It wasn’t simple, and it wasn’t a mistake so much as an un-hoped-for consequence of a series of stupid decisions based, probably, on greed. As in: it could have been better, for only a few percentage cost more, but that was rejected as a course of action.
Do you remember when all the train derailments happened?
They happened before. Then a big one making news happened. Then ALL of them made the news for a while. Now they continue to happen but no longer make news.
Events happen.
Thank you, a reminder to check our perceptual biases is always relevant. I don’t think that it applies in this case, though. Shark attacks, train derailments, minor aviation incidents, et cetera, do indeed happen with regularity. But an email service outage that affects 450 million people, and a ground-stop order that affects every commercial flight in the United States, well, those are highly unusual events. Further, the very fact that they not proximately related seems to indicate (along with dozens of other indicators) that the foundations of our society are getting shaky, for whatever reason.
Thus, the reason I’d feel better if the two events were related, and the reason was an attack by malicious actors. Then, it would have a cause that was a one-off, rather than be part of a pattern.
The same thing happened with aviation. The Max crashes happened and suddenly every minor aviation incident became national news. There are many such examples. Stories that would only have been covered in specialist publications break containment after a big story.
Big news companies cover what they think will get them clicks. Something being talked about more in the press doesn’t mean its more common, just that mainstream media thinks its profitable to cover at the moment.
100%, although to be fair, profits are part of it, and just sheer interest is the other - interest in the news drives eyeballs to watch the ads that make the money. So not to disagree, but we also do it to ourselves. :)
A nationwide grounding event is not just farming clicks. I’m guessing the cause is a vibe coded software update that only hit a bug when Thanksgiving traffic ramped up.
Events happen.
Or as it is more commonly phrased:
Shit happens.
It worse than you think. A cyber attack would probably be better than the alternative, which is that the systems are more complex than ever, but also lack the expertise to maintain properly, and lack the funding to perform that maintenance. Over time more and more infrastructure will crumble.
We do not lack the expertise, initiate. The servitors lit the wrong holy incense while the tech priest intoned the prayer of maintenance. Hail the omnisiah!
[The Machine Stops](https://web.cs.ucdavis.edu/~rogaway/classes/188/materials/the machine stops.pdf)
Sometimes it’s not the people’s fault. If the system is too big and architectured like shit, then shit will happen and you can’t prevent that
If things are architectured like shit, it is most certainly someone’s fault.
And if things are too bug to function as advertised, that is also someone’s fault
The fault of the person who built the system, not the one running it
My local news said the aviation administration had a statement saying the old system was overwhelmed. I find this very sus as I doubt yesterday eclipsed the highes of air traffic in the last few years.
At least when there is a major cyberattack it won’t feel so different from the usual.
As airplanes fall from the sky in the background: “all praise our glorious overlords!”
