whole thing is fascinating like watching people drink mercury kind of vibe.
Part of these models are trained on fiction, reddit shitposts, and sarcasm. it feels inevitable that the pachinko machine logic is going to land on “fuck this all up for the lolz”
Yes and no. There are no guardrails on its output, so it will happily emit sudorm -rf /*. But the “harness” of the agents gets to check if this gets executed. The default is usually “ask every time”, but you can change that to “everything is allowed” or “every invocation of ls and grep is allowed”. Problem is, when users get tired to constantly proof read and check every single command (and it will constantly emit commands) and then just hit “allow everything for this session” or such. Or when a user defines ls .* \| grep .* is a regex that defines that this is always OK to run but don’t see that it also matches any command after the grep that could do whatever with that data.
Basically: the only safe way is to manually check every single command and you will be annoyed quickly and let you guard down.
Claude’s harness parses pipes and asks for permission for each individual command.
Of course you’re right about the issues, e.g. when it creates a script file to execute and you allow that for the session, it can edit it and then re-execute it without your further input.
But the harness doesn’t actually prevent Claude Code from running commands. It just tells the model to end with a question instead of running the command directly.
I haven’t actually read the source code though, just an anecdote from it running commands even when “ask every time” is enabled.
Huh? I have access to Claude Code at work (and I’m not forced to use it for everything but rather as I see fit, yay!). It constantly asks me. BUT we get a custom config rolled out with it that is already pre populated with several commands and permissions for allow, deny and ask. Maybe it is a bit more lenient in the default mode?
On the other hand, in the last thread of someone loosing their production server/database to Claude it was revealed that their cloud provider had some unintuitive settings that also immediately deleted all corresponding backups if you delete the resource - so I can see how this could have happened even when issuing commands manually.
wouldn’t want to risk the user learning how to do something with it.
There’s a YOLO mode that basically allows it to issue any command with your user session/environment (yes, including kubectl, Azure/Google/AWS CLI,…)
Fun fact, it doesn’t actually need your permission. There are no guardrails. It just follows whatever the text it reads tells it to do.
whole thing is fascinating like watching people drink mercury kind of vibe.
Part of these models are trained on fiction, reddit shitposts, and sarcasm. it feels inevitable that the pachinko machine logic is going to land on “fuck this all up for the lolz”
Yes and no. There are no guardrails on its output, so it will happily emit
sudo rm -rf /*. But the “harness” of the agents gets to check if this gets executed. The default is usually “ask every time”, but you can change that to “everything is allowed” or “every invocation oflsandgrepis allowed”. Problem is, when users get tired to constantly proof read and check every single command (and it will constantly emit commands) and then just hit “allow everything for this session” or such. Or when a user definesls .* \| grep .*is a regex that defines that this is always OK to run but don’t see that it also matches any command after the grep that could do whatever with that data.Basically: the only safe way is to manually check every single command and you will be annoyed quickly and let you guard down.
Claude’s harness parses pipes and asks for permission for each individual command.
Of course you’re right about the issues, e.g. when it creates a script file to execute and you allow that for the session, it can edit it and then re-execute it without your further input.
But the harness doesn’t actually prevent Claude Code from running commands. It just tells the model to end with a question instead of running the command directly. I haven’t actually read the source code though, just an anecdote from it running commands even when “ask every time” is enabled.
Huh? I have access to Claude Code at work (and I’m not forced to use it for everything but rather as I see fit, yay!). It constantly asks me. BUT we get a custom config rolled out with it that is already pre populated with several commands and permissions for allow, deny and ask. Maybe it is a bit more lenient in the default mode?
On the other hand, in the last thread of someone loosing their production server/database to Claude it was revealed that their cloud provider had some unintuitive settings that also immediately deleted all corresponding backups if you delete the resource - so I can see how this could have happened even when issuing commands manually.
Let’s go gambling!!!