Have a sneer percolating in your system but not enough time/energy to make a whole post about it? Go forth and be mid - welcome to the Stubsack, your first port of call for learning fresh Awful you’ll near-instantly regret.
Any awful.systems sub may be subsneered in this subthread, techtakes or no.
If your sneer seems higher quality than you thought, feel free to cut’n’paste it into its own post — there’s no quota for posting and the bar really isn’t that high.
The post Xitter web has spawned so many “esoteric” right wing freaks, but there’s no appropriate sneer-space for them. I’m talking redscare-ish, reality challenged “culture critics” who write about everything but understand nothing. I’m talking about reply-guys who make the same 6 tweets about the same 3 subjects. They’re inescapable at this point, yet I don’t see them mocked (as much as they should be)
Like, there was one dude a while back who insisted that women couldn’t be surgeons because they didn’t believe in the moon or in stars? I think each and every one of these guys is uniquely fucked up and if I can’t escape them, I would love to sneer at them.
(Credit and/or blame to David Gerard - both for starting this, and for covering the previous week. We should get a bot to automate this…)
There is something about this picture, a warehouse full of books to be destroyed by Anthropic for training data, that just makes me sick. Knowing that they are destroying books in the abstract is bad enough, but seeing the sheer scale of it makes me viscerally angry.
Karen Hao called her book Empire of AI for a very good reason. The empire has all these noble narratives of bringing civilization and progress to the colonies, but in reality the empire only sees resources to be extracted, repackaged, and sold back to their subjects at a fee. Kudos to those who saw this immediately and fought against it. Shame on those who believed in the narrative of progress, to the point where many of them are collaborators with the empire.
Whenever a smarmy fellow on Linkedin posts about how AI is making so much progress in coding or math, I want them to see this picture. I want them to see the videos of families in tears after a data center takes away their home with eminent domain, and other families who have to deal with the resulting pollution. Go beyond the bloodless math of tokens and instead see for yourself what this all truly costs. I hope it was worth it.
What kills me is that there are so many obvious ways to be less wasteful about this. Nondestructive book scanners exist. They’re expensive but it’s not like AI companies are averse to throwing money down a fucking hole. Even if that’s not an option, it’s possible to rebind the pages and return the books to the market. And regardless of what happens to the physical books, the scans they create could be archived in a format that could be available as a digital library rather than just being fed into the statistical meat grinder to keep the trough topped off with slop. Even if they’ve got to fight with copyright holders it would cost them basically nothing to leave the option open and given the PR battle they’ve been losing it feels like doing so would be incredibly obvious. Hell, even Google Books was able to navigate this in a way that was less cartoonishly evil than this because they could point to their digitization effort as a public good in ways that Anthropic here just fucking can’t.
More insane cult stuff has emerged - this time its MAPLE. https://www.insidemaple.com/p/the-high-control-dynamics-at-maple
AI Crisis Flowchart spotted on linkedin
spoiler
and the actual article: https://www.lawfaremedia.org/article/the-ai-that-hacked-its-way-out-and-the-hype-that-followed-it
quoted zvi haha :|
To be fair, the safety community can plausibly claim vindication here. At worst, an autonomous agent that breaches containment and attacks a real third party is precisely the scenario researchers have warned about for years, which enthusiasts have routinely dismissed as speculative. And at best, the company in charge of maintaining safety failed to construct an adequate containment system.
are you fucking kidding me, they have been DDoSing sites for months, everybody knows about it, and the “safety” community has done jack shit the entire time and has offered zero solutions. I know multiple companies that woke up one day to 10-20k netlify bills until they could stand up an anubis.
Just let hugging face sue the idiots, it isn’t like law enforcement will do anything even though there’s already laws on the books for unauthorized access, industrial espionage, etc etc etc.
and the “safety” community has done jack shit the entire time and has offered zero solutions.
Occasionally I see a sane relatively workable idea on lesswrong (like making stricter laws on liability and transparency for everything AI companies do, I’ve also seen this idea put forward on lawfare a few times), but that is like 1 idea out of 20, with the other 19 being absurd, like the absolutely unworkable fantasies of AI: 2040.
Google shut down the AlphaFold team, trying to reassign the people behind the super interesting problem of using ML to predict protein structure from sequence onto chatbot development. A whole bunch left the company.
https://www.engadget.com/2225849/google-shuts-down-alphafold/
The Business Idiots are truly running the asylum.
One of the defenses I see of LLMs on place like /r/singularity (although recently /r/singularity has started wising up and the committed true believers have shifted to /r/accelerate), is that they are going to cure cancer or some other incredibly valuable thing, conflating AI, ML in general, DNN, vs. LLMs specifically. Technology like AlphaFold was a lot more on the “cure cancer” track (although there is still a huge gap between better protein folding predictions and successful drug discovery) than anything LLM related… this is sad.
This has always been a huge red herring. Llms are built on top of the transformer architecture which does text autocomplete (and yes we can combine text embeddings with other inputs like images). They have some interesting properties where they seem to be able to do text autocomplete in a bunch of different scenarios that they weren’t explicitly trained for, but they were never designed for precise dna analysis. It is their architecture that prevents them from other long horizon tasks like playing chess and the way that they represent text is why they can never count the letters in strawberry (most have this specific question hard-coded in their training data now).
Anyone who believes that LLMs are going to solve cancer either has no idea how they work or has been one-shotted from talking to Claudia
I am continually amused at people not quite understanding what AlphaFold is actually doing, too.
Yes, a bunch of its performance comes from it learning rules about how proteins fold. But not a majority of its performance. MOST of its performance is it effectively acting as a translator of what evolution knows about protein folding into a form we can understand.
A key part of the system is not just cooking the sequence into a structure. A system running alphafold has a database of terabytes of curated sequence information from all over the tree of life. You put in the sequence you care about, and it first searches that database for anything with homology, and builds a “covariation matrix” - wherever theres anything with even vague sequence relatedness, build a matrix of every position in your sequence and the correlation between variation at position X and variation at position Y. This covariation matrix represents implicit information from the evolutionary process about what parts of a sequence are functionally connected to each other, which has a correlation to positional information, and these correlations are in turn learned by the ML system.
You put in de novo designed proteins or orphan proteins without homologs in the curated dataset and performance does not go away, but it drops precipitously. A bunch of what is going on is finding an evolutionary signal, and translating that evolutionary signal into structural information. So still, evolution knows much much more about protein folding than we do or any machine does, and once again a ML system is revealed to essentially be an information channel that takes in information from an interesting source on one end and turns it into a different form of information on the other.
Those things are barely related, no wonder the staff voted with their feet. It would be a bit like taking a team of seasoned architects who design amazing skyscrapers and telling them they have to spend all their time on low density suburban houses from now on… Come on guys it’s still architecture. Where are you going?
no wonder the staff voted with their feet
Rather than rebelling against google, it seems like they where just poached by anthropic a short while before the project shut down:
[Alphahold VP and nobel winner] Jumper announced in June that he was leaving DeepMind to join Anthropic. A couple of his colleagues jumped ship to Anthropic, as well. DeepMind has confirmed to the Times that it also moved staff members internally to Gemini-focused projects. Other former staff members were reassigned to Alphabet’s drug-discovery company Isomorphic Labs, which is a DeepMind spin-off.
Depends on what Anthropic wants to do with the poached people.
Oh… Well that’s disappointing.
I was an academic in computer science in the last 10 years or so (keeping it vague to avoid doxxing myself) and it has been so depressing seeing so many of my colleagues selling out to OpenAI, Anthropic, Meta and even Google (for some reason the latter often gets a reputational free pass because people associate them with the golden days of big tech 10+ years ago)
I am actually in the middle of both trying to advance my career and a project about information theory in evolutionary biology making a bunch of explicit parallels to machine learning. Someone where I work suggested that given the connections I was making I should look to a ‘frontier AI lab’ as an employer.
He did not see the instant flashbacks to chasing these weirdos across the internet for almost two decades, watching in horror as the religious psychosis gained national prominence and great destructive power. All he got to hear was my instant intonation of “I’m sorry Dave, I’m afraid I can’t do that.”
a stock-slobber posts a cri de cœur about the coming automation of maths and HN says it’s no biggie, think how productive the field will become
https://kirwinhampshire.substack.com/p/the-dark-night-of-mathematics
https://news.ycombinator.com/item?id=49048681
btw the language in the post is bad enough to be entirely human-generated
For me, the affective quality of learning mathematics is empathetically tethered to an act of discovery and creation.
Before you a channel through which humans have accessed the ineffable and sacred for thousands of years is being sealed for eternity.
The blogger simultaneously elevates mathematics to some mystical endeavor and fixates on novel theorem proving as the key element of that and believes LLM-based AI will replace human mathematicians at theorem proving in a matter of years… that is quite a combination. I can imagine how 1.5 of those things fit together (although I disagree with that view point obviously), but the whole package is really quite an odd combination.
Also, in the fantasy scenarios where AI really is capable of totally replacing mathematicians, aren’t we supposed to get post-scarcity abundance, freeing up your time to pursue mathematics out of pure desire for enlightenment? Maybe the blogger doesn’t believe that part? Or they are so attached to themselves personally getting to discover novel theorems first they don’t care that the post-scarcity era would on net free up a lot more people to pursue pure mathematics as a hobby.
how productive the field will become
Let’s just ship all these people to Macroeconomica
Missed this paper last week from Google
Funny how now that it’s widely known that the general public hate AI, companies producing AI are softening their “it’s going to take your job” stance. Surely just a coincidence
It sure is a zany coincidence that all the major American AI companies have come out either for or against open weight AI models with grand ethical and/or patriotic concerns all at the same time isn’t it?
Most recently our pal Zuck! https://www.wsj.com/opinion/the-ai-future-is-for-everyone-a0c24e20
I’m just imagining a WSJ reader reading this and thinking “what the heck is this man talking about”. At some point you need a little structure or text to balance out the subtext.
Edit: did I miss any?
OpenAI (discussed here the other week): https://nitter.net/deanwball/status/2078133895766114412 / https://xcancel.com/deanwball/status/2078619513575137330
Microsoft: https://www.microsoft.com/en-us/corporate-responsibility/topics/open-weight/
Anthropic: https://www.anthropic.com/news/position-open-weights-models
Nvidia: https://images.nvidia.com/pdf/Open-Weights-and-American-AI-Leadership.pdf
Google: https://xcancel.com/sundarpichai/status/2081026488158040181
AmazonThe Washington Post who I’m sure wrote this for reasons completely unrelated to Mr. Bezos: https://www.washingtonpost.com/opinions/2026/07/02/right-answer-chinas-open-weight-ai-models-is-build-our-own/That one guy I hate: https://xcancel.com/elonmusk/status/2080672505660834163
It’s going to be really interesting to see if the foss zealots who switched over to the AI side will handle heavy handed regulation taking away their local models.
they’re all getting stiffies from the idea of black market models
As intelligence becomes abundant, the most important question will be how we direct it. Some argue that superintelligence itself, or a small set of experts who control it, should decide what is best for humanity. I disagree. The history of democracy and economics has shown that there is no single objective answer to how people define the best life, and therefore the best approach is letting people decide what matters to them. Rather than centralizing this power, we believe that delivering personal superintelligence to everyone is the way to answer this question.
Healthy societies also recognize that safety requires checks and balances. If only a handful of institutions have superintelligence, they will inevitably exercise a controlling influence over economics, science and politics. Even with the best intentions, that concentration would limit people’s ability to choose their own future.
Famous enemy of software monopolies and tech oligarchies that have an outsized influence over politics, champion of distributed ownership of the means of computation Mark Zuckerberg
As an interesting follow-up to the ai-does-maths-using lean4 stubstack comments on Sunday, an llm accidentally uncovers a bug in the lean4 kernel.
Summary by Meven Lennon-Bertrand:
https://lipn.info/@mevenlennonbertrand/116997917683191056
To summarize:
- an AI agent let loose provides a sorry-free proof of the Collatz conjecture
- the proof is detected as actually being a kernel bug
- the bug is related to (nested) inductive types, for which there is no clear theoretical specification: the kernel’s code is the reference
- external checkers (lean4lean and nanoda from a week ago) reproduce the bug, because they essentially copied the reference kernel implementation
Eta: “sorry-free” in this case means a complete proof with no trust-me-bro steps or TODOs… the
sorrytactic in lean “proves” a theorem to be correct even if it is garbage or incomplete. More programming languages should make developers apologise for half-arsing their work.And so
- AI raises the bar for kernel correctness by a lot
- without a clear type-theoretic understanding of what is actually implemented, we’re toast
- external checkers help to catch implementation bugs, but without a clear specification they can’t catch logic bugs
How bad this is, is unclear just yet… probably not the sky actually falling, but not great. Interesting though.
A little bit of plot thickening spotted by abadidea: https://infosec.exchange/@0xabad1dea/117002106099986943
tl;dr, the timeline looks like this:
- “proof” of collatz conjecture released
- bugs identified in lean kernel
- proof demonstrated to use these bugs
There was only a day between the first two events, and the non-proof was not where the bugs were discovered. So maybe it was just a coincidence that the chatbot found the bug at the same time, or maybe it’s training data included previous investigations into those bugs which it then built upon and that would be a bad thing for other llm generated proofs.
The collatz conjecture is sufficiently famous that enough third-party checking was done to spot the problem. I wonder how much checking would have been done on proofs of less famous and interesting things.
And a follow-up by talia ringer, who observes that there have always been gaps between the type-theoretic underpinnings of things like the lean prover and their actual implementation, and this hasn’t been so much of an issue til now because theorem provers haven’t had the attention of people in high places, and the type-theoreticians have been able to catch up in due course.
https://mathstodon.xyz/@TaliaRinger/117005740997367321
My big worry right now is that if organizations continue to fund the crap out of Al for formal proof research (and to generally support implementation and maintenance of proof assistants like Lean as part of that effort) but don’t bother funding the type theory side of things, those gaps will grow larger and will be exploited more often by Al tools via reward hacking. Whereas people tend to only exploit kernel bugs to make a point that the bug exists. Thus proof assistants will grow less trustworthy over time.
Anyone want to place any bets on whether or nor the big llm companies are going to fund academic research that isn’t obviously mechanisable right now and won’t yield any clickbait headlines?
Lean was already known to be untrustworthy and bad, although people refuse to internalize the situation because they’re caught up in Buzzard’s hype machine. This is extremely funny but I don’t see any signs of people waking up and realizing that Lean sucks.
Max Kennerly sneers the New York Times:
This opinion piece, written by someone with no relevant expertise, cites two sources, both of them blogs by people who also have no relevant expertise, one of whom is employed by a sham think tank funded by the “Effective Altruism” propaganda network.
Truly embarrassing decision to publish it.
The op-ed writer is Adam Mastroianni who is a regular at Lighthaven. https://gwern.net/interview-inkhaven
He cites Andy Masley who is yet another autodidact with a cushy job in the Effective Altruism grantland. https://www.andymasley.com/ and Brian Potter who is an engineer at a think tank called the Institute for Progress https://ifp.org/author/brian-potter/ (“The overwhelming priority of energy policy must be making it easier to build things” which things angryDuckMeme.png)
LLM bros actively force themselves everywhere, part infinity:
Newgrounds moderation had to deal with someone (probably a scammer) posting slop “art advice” recently - exactly why, nobody fuckin’ knows:
Transcript/desciption
I’ve already passed it off to folks that could nuke the account from orbit, but I guess as a public service announcement, be on the lookout for replies in the forums that sound a little too LLM-y.
An account (unsure if a bot or just a weirdo) recently attempted to infiltrate the art forum, giving plausible-sounding AI-generated list-based advice, complete with preambles and summations, exactly the kind of syntax and format one would expect if they’ve dealt with enough clanker slop text. It fooled a small handful of people, but it was extremely easy to confirm my suspicions as soon as I smelled the clanker stank on it. One click made it clear that they had a generated PFP and profile linking to some image slopification scam.
It raises the question, what would anyone possibly have to gain from posting slop on the forums? It seems like a really stupid thing to do, but I guess braindead behavior tracks, given who we’re dealing with. Make sure “helpful” new users aren’t just trying to Dead Internet us or whatever.
zerohedge sponcon so obviously a grift but wtf is this AI Black Paper by Jim Rickards
https://www.zerohedge.com/sponsored-post/sobering-july-29th-announcement-set-crush-ai-stocks
Have the goldbugs been reading Zitron? or just an extremely late pivot from btc to ai? or flooding the zone?
Anyway who the hell is Jim Rickards? Another week, another guy.
https://en.wikipedia.org/wiki/Talk:James_Rickards
“AI Black Paper” is catchy enough though, should be a goth band.
or flooding the zone?
With Zerohedge I think this is most likely. Maintaining truthiness by predicting 30 out of the last 2 recessions.
The AI Race With China Is a Lie Told by Big Tech to Justify the Data Center Invasion via naked capitalism
choice quotes:
More to the point, LLM development in China is incidental to the country’s real goal for AI. Its focus remains on products embedded with AI and robots. Or, as AI policy researcher Liang Zheng says, in China, “The first priority is to use it to benefit ordinary people” (debatable, but indicates the kind of AI they are pursuing).
In the US, the first priority is to exploit people so that the tech oligarchs can profit. It’s chatbots all the way down.
Yet, the “AI race” story is a convenient lie for Big Tech and its political protectors. It is a neat political argument designed to insulate the industry from criticism and regulation.
If every environmental review, every question raised by a community, every issue around water usage and electricity prices can be dismissed or lessened as “helping China,” then meaningful political debate can be silenced. Real regulation—if even possible—can be avoided. Fear becomes a substitute for policy.
At the same time, data center developers and their minions in Washington have tried to weaponize false claims about foreign ties to the anti-data center movement. Kevin O’Leary of Shark Tank fame has explicitly said that our movement is being funded by China. He has no evidence because evidence of a falsehood can’t exist. In fact, Fox News has been forced to retract its coverage of his claims.
We understand why O’Leary and the other Tech Broligarchs don’t understand the grassroots opposition to data centers building across the country. It’s hard to spot the grassroots from the window of a private jet.
I need one of those UFO posters that are like “I want to believe” but it’s a picture of the PRCs flag lol.
As always though pointing at the
SpanishgermansRussiansChinese remains an evergreen excuse as to why you can’t be anything other than wholely on board with madness or you’re a traitor.
So it’s been 7-8 months since the software industry decided that AI agents are good now and that you can do a year’s work in 1 hour
So… Where are all the amazing and wonderful AI generated software products? Where are all the companies using agents to leave the rest of us behind?
All I see are one-shotted maintainers, big marketing stories with little substance and a pile of detritus made up of abandoned vibe coded projects.
It’s almost like AI agents make you feel more productive but actually slow you down (paywalled sorry)
The AI shills would say: Are you still using the models from six months ago? Because the new models are sooo much more powerful.
I once listened to a talk by a guy from german OSHA that hit a lot of the good points. Perceived productivity opposed to actual productivity from the METR studies. Mental load of alien code review. (This was a company-internal IT-expert exchange thingy.)
A question in Q&A (submitted anonymously on slido like a sniveling coward) was deadass “Have you ever coded with a frontier model?”
Have you ever tried meth, tho?
yea you just don’t have enough experience, I do bath salts every week and I promise you productivity gets a huge boost after week 5
The government agents who live in my walls and plant microphones in all my appliances impose entirely different productivity metrics, which are simultaneously more and less attainable
you just know one of these guys has microdosed bath salts out of sheer contrariness
Funnily enough they said that back then too. What a weird coincidence!
Yeah. Behind all the hype about percieved productivity etc., there’s really only one benchmark - will GenAI help produce better software, faster, and at less cost? So far this doesn’t seem to be happening.
The major shift of not needing SaaS middleman products because AI can spin them up instantly at scale never happened. Instead we see lay-offs to pay for AI tokens while quality plummets and services offered do not increase.
Clicked on their profile to see if the Plissken thing was a reference to Snake Plissken, and (cw: rape jokes)eurhg what a fucking horrible guy. 160k+ likes 66m views, what a far right hellhole twitter has become.
The actual worst man, and also just fucking sad. Imagine knowing that not only did you peak in high school, but that you peaked specifically in the moment when you fucked up and lost the state championship. Like, you can just smell the empty beer cans and tears piled up around this guy’s posts.
Ow god he is Al Bundy, if Al didn’t score four touchdowns in one game.
I’m sure this guy uses the ai quite a lot (and so would the guy in Married With Children)
