small rant incoming
I work for a company that it’s mostly hardware focused, but we do ship some software for the final consumer including drivers and programs, to make use of said hardware.
While I am not in the software department, I held some SWE positions in previous companies for over a decade, our software isn’t very complex and I do know most of it pretty well.
Our employee just announced a new AI-only development cycle, they want all code submissions and reviews to be exclusively done by claude, effectively ending ownership of the code being shipped to customers. This is absolute madness.
Today, I received an email scheduling a workshop on how to integrate claude into vscode and how to work with the new gitflow, namely removing our authorship from commits and having al code reviews done by a LLM now.
I am just baffled at the decision.
edit: wow I’m a bit overwhelmed by the response, I did read all of you. Thanks!
Time to start tokenmaxing and make this as costly a decision as possible
Oops, i had Fable selected all month.
Time for some quality control and optimization.
Claude, take our codebase and generate a set of unit tests to evaluate the code functionality based on the current snapshot.
Now port this code to Rust and add comments describing how each function is different. Run the same set of unit tests to evaluate the new Rust code against the existing code. If any unit test fails by more than 0.01% remove the Rust code and regenerate it from scratch.
Evaluate both the current code and Rust code for readability and documentation. Generate full source documentation for each line of code.etc etc. You’ll have half a datacenter cranking for a week. They won’t appreciate the bill…
The attempt to imagine this with stock thinkmeat has left me insensate, emotionally bereft, and mildly insane.
You mad genius. You just named my next Thinkpad.
You were already insensate, emotionally bereft, and mildly insane. The prompt to generate unit tests made you aware that you are mildly insane as you had to create a test for insanity and fail yourself is you scored zero.
It’s a fair cop, guv
Same seems to happen with some healthcare software in Germany. For example, Compu Group Medical is in the news for massively losing customers for bad software quality, and they have gone all-in on AI.
I get notifications of my prescriptions being sent off to the pharmacy via a service called RXInform.
They recently incorporated an LLM feature. All this does for the end user is take a few seconds to type of what the prescription is, where it’s being sent, how much it’ll cost without insurance, if there are any coupons, and the insurance price.
All of this is incredibly simple and was successfully displayed using a template. The way the information is presented never changes. The only thing the LLM does is increase the time it takes to actually see the full text while making some server somewhere works harder than needed.
Edit: fixed typo
Also while feeding personal health information into a black box somewhere with no real way of knowing what BlackBox© company is collecting or selling in the mean time.
Whileas the pharmacy close to my house will get what you ordered if you just mail them a photo of your last package.
Myepic is used pervasively in US healthcare system, they likely use AI in some form. because the MDs use AI to write thier notes for sure.
I have a coworker who went full AI, and I think is actually losing it, like sees them as people. He was telling me he’s been asking chatgpt for relationship advice and how I should use it to get advice on my relationships (my wife and I are fine because I prefer talking to her than the robots).
On Friday he was spamming me with “issues” that codex found with my PR. I think he sent over 20 bulletpoints. 1 of them was a small bug that would only apply in a way far out edge case but good to fix. The rest was absolute nonsense, or evidently not an issue if a huge read it and thought about the context.
I ignored him all day, but still had to take the time to read and verify everything. At 4pm he sends me a message asking me to fix the bugs or he can have codex fix them. I told him they were nonsense and not applicable, explained each one in simple terms, then 20 minutes later I got a message “codex said there’s still these issue: <the same fucking list that he evidently didn’t read>”.
This has been going in for a while. I spend half my week fixing his AI slop that he genuinely thinks is gold. I swear 4 months ago he was a very solid developer, today I wouldn’t trust him to write an HTML page.
I think the easiest way to make an immediate change is to never call/talk to, or about it, as you would a person.
I dont use LLMs at all, but my wife does an i always correct her to say “It said…” when she tells me “He said…” every time she uses it instead of google to get an answer. (And then tell her to find me a trusted website that backs up the LLM as well)
Dehuminise it so theres always a part of your brain treating it as a machine/device, not a person you can trust.Its like saying “my car broke down, she isnt feeling well”. The car broke down, it needs a part, do i trust my car? Usually but i wont be suprised if it doesnt work some times.
And i dont expect it to be able to swim if i try make it instead of using the vehicle i should have been using.I spend half my week fixing his AI slop that he genuinely thinks is gold.
God I hate this. The sheer number of times I’ve heard “and I didn’t even write a line of code” and I have to bite back the “yeah, I can fuckin’ tell.”
It can happen to anyone.
I think the first documented cases were inside Google, from scientists who know exactly how they work:
https://www.cbc.ca/news/science/ai-consciousness-how-to-recognize-1.6498068
I seem to remember a debacle earlier than 2022, but can’t find it… but point is, I suspect its quite a human psychological vulnerability. So if they recover, I wouldn’t be too hard on your coworker.
The first form of this was called ELIZA effect. In 1966.
Oh, interesting. TIL.
That’s really sad… That is worse than what I thought was annoying at my work today. Claude indicated something was an issue that has been “an issue” for years and never actually been an issue. It suggested over engineering the fuck out of something we just plainly don’t need to do. I was annoyed we spent 10+ minutes talking about that. What you described is so much worse 😬
I was fired for pointing out dangers of AI and data centers. I would do it again
Thank you.
I’m retired, so I don’t have to deal with this AI garbage… but I have a friend who’s a manager of a decent-sized office at an accounting firm.
He told me recently that, not only are all employees required to use AI in their daily work, but my friend has to write weekly reports to his bosses on how AI is improving workflow in the office. He’s not allowed to say, “It’s not.” Only positive feedback. They need to justify its continued use, so he needs to find benefits to report to the higher-ups.
Sadly, he’s the only one who doesn’t like/use AI. Everyone else in his office has basically replaced their jobs with it. Every report that comes across his desk is AI nonsense, which he has to spend time fixing because his subordinates don’t know how to write reports without AI assistance.
I do not envy my friend.
Isnt the answer to then send on the un-fixed reports and say “Oh i had AI check it and it said that was all good, are you saying AI isnt working?”
. He’s not allowed to say, “It’s not.” Only positive feedback. They need to justify its continued use, so he needs to find benefits to report to the higher-ups.
Those higher-ups should not be in charge.
This feels like the emperor’s new clothes, except no one listens to the children pointing out the folly.
This feels like the emperor’s new clothes,
I guess it does not only feel like that. It’s the same thing.
my friend has to write weekly reports to his bosses on how AI is improving workflow in the office. He’s not allowed to say, “It’s not.” Only positive feedback. They need to justify its continued use, so he needs to find benefits to report to the higher-ups.
All right Claude, you know the drill. Write this week’s report.
AI is improving workflow in the office by writing these reports for me so I can do actual work.
- Write a weekly report on how AI is improving workflow in the office - Only positive feedback - MAKE NO MISTAKES!- while you’re at it, find new revenue streams
- Also, tell me which 5 employees to fire, so I can reduce costs and buy more tokens.
My analysis indicates we should fire you, the CEO, as you have the highest salary. They will be able to purchase more tokens then to use for productivity.
oh man you folks are funny!
my friend has to write weekly reports to his bosses on how AI is improving workflow in the office. He’s not allowed to say, “It’s not.” Only positive feedback. They need to justify its continued use, so he needs to find benefits to report to the higher-ups.
“Please make us blind to all consequences!”
At an accounting firm no less… Who gets the blame if some number turns out hallucinated? The lower level employees or your friend? (assuming C-levels never carry blame)
From my own personal experience working in all sizes of companies across various industries and even countries, this story a strange yet eerily familiar smell to it…
spoiler
[Somebody is being setup to be the fall guy when AI turns out to have made things worse]
it makes sense AI company pushing AI but why these random companies are pushing it? does openai pays them or something?
It’s the New Thing™. Companies are trying to get in on the ground floor and take advantage of it while it’s easy to integrate.
No business wants to avoid the new trend, only to find out they’re behind the curve and now failing. But AI was a very bad horse to bet on and a lot of businesses are gonna find out the hard way.
That is going to be a glorious dumpster fire. You have my sincere sympathy.
Unions solve this issue btw. Would love to see software development work more like carpentry or public works with a trade union and work hall than the current cult of personality or culture of staying at a single employer your entire life.
I’ve been looking for a job for the past 6 months. Every interview I had (more than 50) hinted that some Claude or Copilot was being used more and more. Some companies even want to hire “junior AI engineer.” How the fuck can you hire a junior with a non-existing degree and that knows nothing about a technology so recent?
I found one and only one company who officially rejects AI, and I hope they’ll hire me.
those companies likely use AI to screen out peoples AI resumes too.
Absolutely. They first ask you to fill an “internal resume” that is automatically filled by AI from your PDF, then is it parsed by an “ATS grader” to give you good or bad grades (I’ve seen it done by a friend of mine). It allows HR to reject most people quickly without reading or doing any kind of work.
I don’t blame them though, most fullstack job offers have up to 200 applicants for any kind of job or requirement. It’s: AI job offer -> 200 AI fake resume -> AI ATS to reject people.
or they just ignore the first few dozen resume, and just choose one they like. they originally use software to do this before AI, i can imagine it does it more faster, and haphazardly. i wouldnt be surprised if that ATS system, is just randomly doing it too. if like 50 people have too similar in experiences, likely it might choose those 50 to discard. im curious what did ATS metrics was used on your friend?
WTF?!? Where are you getting all of these interviews? I’ve been looking for a job for 19 months and had two interviews.
I’m in France. It’s the same apocalypse as in the USA, but companies and recruiters are eager to get the biggest number of resume in their database. 50 is only a random number and it makes no difference in what you experience. Most companies post job offers, but never hire anyone, I’ve seen it because those offers are reposted as “brand new” every other week, which means that they found no one and they try again.
I use LinkedIn, Welcome to the Jungle, Hello Work, and every other site where I can apply. I think I sent at least 2 resumes every day for 6 months, i.e. 360 resumes, and most interviews are only meant for the recruiter to read my resume, me to explain it, and that’s it. It seldom ends with a real interview in the final company. I forgot to mention that the recruiters are mostly working for “contracting companies” which is where you apply when you are desperate.
To sum it up, 50 or 10 interviews is the same thing, there is no job nowadays, it’s all lies. The serious jobs I found where from small unknown companies that do not advertise at all…
i think AI is being used to screen job applicant resumes in the usa, thats why i think theres been so many people who have the same problem.
Let me guess, 10 years experience needed?
Not yet, but they all have various job titles like AI developer, AI architect, AI devops, AI anything… It’s like they stick the word AI to traditional roles and expect people to understand what it’s all about (or pretend it’s a job that exists at all, it’s a scam after all).
At such a point, you could well say “Yes I am!”, since it’s a bullshitting race, and nobody can tell if it’s true.
actually no, whenever you fail to do a task they will blame you instead of admitting that AI is trash
Almost everywhere I’ve applied or know people at is frothing at the mouth for AI. Interestingly, I met a guy last weekend that works for a big video game studio you’ve heard of, and he was like “everyone here hates AI. Management on down”. So that was nice to hear.
Fight fire with fire. Feed the job requirement to the AI. If tech Claude Sonnet is good for this. Then feed it your resume and have it flag any mismatched items between the req and your CV. Adjust resume as necessary until the AI is content with the match. Repeat with other AI like ChatGPT, etc, until they are all content. Process should take less than 30 minutes. Submit.
Fork the repos that matter now, and squirrel them away somewhere, just in case someone gets froggy and gives Claude write access to your repos and fucks the history of something interesting like that. Always good to have a backup contingency.
I’m of the opinion that companies that make stupid AI decisions so they can fire people shouldn’t get bailed out by a good decision made by one of the people they’re planning to fire.
Let them burn.
Corporate suicide by AI?
My point is more along the lines of “some prudent sandbagging might let you swoop in and save the day at some future point when everything goes tits up”
My point is more along the lines of “Let things go tips up instead of incentivizing bad behavior by bailing them out of their fuckups.”
A clone of a git repository is a full backup.
and fucks the history
And then you accidentally pull / force pull or something and your “backup” is fucked.
Adding an extra step for a harder-to-find backup makes it more difficult to accidentally hose your known-good state.
I’ve seen cases where the commit was still there, even though there were no pointers (branches/tags) pointing at it. Which raises the question, when does git delete a commit?
By default, when it’s older than two weeks and you run any of the standard commands which trigger a GC check.
Docs here: https://git-scm.com/docs/git-gc
Commits are pointed to by references such as branch names. If a branch is deleted or a force push happens, the commit remains accesible for some time. It can be found in the “reflog”. This works much like pointers and automated memory management. If no tag points at it, it will become garbage-collected at some point. Adding a tag like “non-ai-master” would prevent garbage collection.
this is one of the most common cases in which you want to mutate your history, and correctly excising content from a commit will change the commit shasum, and make the previous commit shasum (intentionally) irretrievable (from remote, at least). If the shasum doesn’t change, you haven’t removed anything.
Because the goddamn suits think AI is going to make them more money, despite that it practically uses mined/stolen data and consumes more electricity than a small city.
a AI video consumes that much.
Where I work the code quality is so bad LLMs were a net improvement
Sounds like someone doesn’t know about how copyright works for AI code or any possible liability involved in distributing it. The walls will come tumbling down one of these days, then they’ll turn to you to fix it.
Not with Ka$h and his liquor cabinet. Why do you think they’re not opposed?
For the most part, I am an AI sceptic. My firm has had access to and beta tested every Google AI model since before the AI had a name (yes, well before Bard). We also have access to ChatGPT, Claude and locally run models on dedicated hardware. I’ll save you the suspense, at the moment it’s all mostly trash. Barely a proof of concept. Definitely not production ready, but there are a few exceptions I, personally, have found.
Gemini Pro is excellent at identifying and diagnosing sick garden plants if you take and upload a photos of the plants. Unfortunately, it still cannot do simple arithmetic. A simple SUM that any spreadsheet does automatically by just highlighting the cells, and somehow it was still off by one.
For software dev, the only one product we have found to be useful is Claude, BUT it really depends on the model. Haiku is a joke. Sonnet is not bad, if not bad is ~50% accuracy. Opus is fairly descent at building scaffolding that you can then correct and complete, but don’t expect efficient code.
The only model we have ever tested that can deliver ALMOST fully functional code, reliably diagnose errors, and review code is Mythos/Fable. Genuinely not absolute shite, BUT that shit is extremely slow and outrageously expensive. I seriously doubt that your company is forking out the money for Claude Fable as their default model, and if they are, they must not care about ever making a profit.
For the plant identification, I would say that these detection methods are heavily biased towards the data in the training sets. Sure, it’s probably good at commonly occurring plant symptoms, but would be unable to assess rare symptoms with any real accuracy. Same goes for diseases with similar symptoms.
I think it if fine for entertainment or research purposes, but should not be used for real world issues. AI should not be used to accuse people of crime they didn’t commit and have that be taken as evidence. Like how Google scans all personal photos and has falsely reported people of heinous crimes. Sure they catch some bad actors, but I doubt they are doing better than if they were to randomly report people’s accounts. And there is never a human in the loop to correct for false positives.
might not work with plants that look like another plant too, and give you misleading info. some people couldnt identify rare plants, because rarely are encountered by people.
For sure. I have no doubt that it was trained primarily using Reddit (the gardening subs were actually pretty good). My intent with the comment was to imply casual home gardening use (entertainment). I apologize if that wasn’t clear.
i visit the whatisthisplant sub quite often, and some pretty unusual rare plants(like wild orchids, mycoheterotrophs), i thinks harder to identify tropical wild plants though.
Your comment was clear, I just wanted to add on some other supporting information and also add on other scenarios for additional context. The main point of all of this being the use cases for AI are currently very slim, but it is being pushed to an unreasonable extent on people. Maybe some like it and are happy to give up their data and privacy, but I think there needed to have been regulations and protections set in place before it was rolled out.
likely common house plants have so many pictures online already it would be easy to identify with a AI, especially people also post disease plants to idenfy it too. they dont do that well with animals/pests though, since theres alot of look alike species and very similar patterns.
For software dev, the only one product we have found to be useful is Claude, BUT it really depends on the model. Haiku is a joke. Sonnet is not bad, if not bad is ~50% accuracy.
Lets assume you have to cross a desert by foot. You can have either a human guide or a map, which is cheaper. The guide can reliably tell you points where you will find water that you need in order to survive. The map has much more information but it will be bullshit half of the time.
Which would you chose?
What would a company chose?
Why?
There was another similar post not that long ago.
Manager says “we don’t touch code any more, we only tell claude what to do”. Part of the thinking is that claude will improve as you teach it.
There’s a lot of hype and hyperbole from both sides. Managers thinking claude can replace entire teams and commentary saying it can’t explain how to tie your shoes.
I haven’t used claude itself so I just don’t know, but things like Mistral and other models available on huggingface just aren’t in this kind of league.
I feel like the atrophy of cognitive function and high level skills is the other side of the AI-in-the-workplace coin. It just seems obvious to me, and I’m sure to any professional, that they wouldn’t have developed their skill if they didn’t… develop their skill. You can’t really develop coding skills by looking at what claude did.
Part of the thinking is that claude will improve as you teach it.
This is just a complete misunderstanding (by the manager) of how LLMs work. Sure, they probably log conversations as training data for a new model, but session to session, it does not improve from talking to it. The model is fixed. There are strategies which boil down to having the LLM write a diary of things it should “remember” which it subsequently has to reread when relaunching, but those suffer from fundamental limits, only so much can be stored before it runs out of context.
It’s so frustrating how much imaginative thinking and anthropomorphization is applied to these tools…
It’s great if you could have found the answer on Stack Overflow and applied it to your code. It’s legitimately useful if there are easy (but maybe tedious) ways to accomplish your goal.
It’s terrible if you get too vague or what you’re doing isn’t common. It’s worse than terrible if you can’t tell or refuse to believe when it’s useless.
It’s here to stay. It’ll likely replace the concept of ORMs, and it’ll kill the need for a lot of third party tools.
It’s not going to replace many workers. If it were used responsibly, it’d just make people’s jobs a bit easier and make software generally more polished and complete. Instead it’s going to bury those projects that use it responsibly in AI slop projects.
We have tried them all. The only model that gets remotely close, IMHO, is Claude Mythos/Fable, and even then it is definitely not “hands off”. It is also crazy expensive and very slow.
Better polish your CV and stick the popcorn in the microwave. Start sending applications when the first customer complaints start popping up. You should have a new job by the time the company’s reputation is in flames.
