One safety pattern I’m baking into CLI tools meant for agents: anytime an agent could do something very bad, like email blast too many people, CLI tools now require a one-time password
The tool tells the agent to ask the user for it, and the agent cannot proceed without it. The instructions from the tool show an all caps message explaining the risk and telling the agent that they must prompt the user for the OTP
I haven't used any of the *Claws yet, but this seems like an essential poor man's human-in-the-loop implementation that may help prevent some pain
I prefer to make my own agent CLIs for everything for reasons like this and many others to fully control aspects of what the tool may do and to make them more useful
Now we do computing like we play Sim City: sketching fuzzy plans and hoping those little creatures behave the way we thought they might. All the beauty and guarantees offered by a system obeying strict and predictable rules goes down the drain, because life's so boring, apparently.
I think it's Darwinian logic in action. In most areas of software, perfection or near-perfection are not required, and as a result software creators are more likely to make money if they ship something that is 80% perfect now than if they ship something that is 99% perfect 6 months from now.
I think this is also the reason why the methodology typically named or mis-named "Agile", which can be described as just-in-time assembly line software manufacturing, has become so prevalent.
> software creators are more likely to make money if they ship something that is 80% perfect now than if they ship something that is 99% perfect 6 months from now.
Except they are shooting themselves in the foot. I reminds me of the goldrush where the shovel and trousers sellers (here the AI companies) would make more money than the miners (developers).
Soon there will be barely any software to build if the general public can just ask an AI to do the things they want. 10 years ago, people would ask a friend that knew about photoshop to help them edit a picture or create something. Nowadays most of them just ask an AI. Same will happen to any kind of productivity or artistic tool. The people alergic to AI slop will just go full luddite and analog and won't use a computer for anything artistry so software creators will lose them alltogether. Home and professionnal software might gradually just disappear and most software creators will have spent thoundands of dollars in tokens with nothing to sell anymore. What might survive might only be the tools that AI rely one, operating systems, database and storage systems, etc.
But boy you will have been super productive, yet totally cancelled by the increase in competition, for the few years it lasted.
The difference is that it's not a toy. I'd rather compare it to the early days of offshore development, when remote teams were sooo attractive because they cost 20% of an onshore team for a comparable declared capability, but the predictability and mutual understanding proved to be... not as easy.
We will not arrive at the desired state without stumbling around and going completely off the rails, as we do, but clearly the idea here is to do stuff that we failed to do under the previous "beauty and guarantees" paradigm.
>Now we do computing like we play Sim City: sketching fuzzy plans and hoping
I still have a native install of Sim City 2000 — which I've played since purchasing decades ago. My most recent cityscape only used low-density zoning, which is a handicap that leads to bucolic scenery and constant cashflow issues.
It's fuzzier sketching, more aimless fun as I've gotten older.
It’s like coders (and now their agents) are re-creating biology. As a former software engineer who changed careers to biology, it’s kind of cool to see this! There is an inherent fuzziness to biological life, and now AI is also becoming increasingly fuzzy. We are living in a truly amazing time. I don’t know what the future holds, but to be at this point in history and to experience this, it’s quite something.
The issue is that for most things we don't want the fuzzy nature of biology in our systems. Yet some people try to shoehorn it into everything. It is OK for chat or natural language things, which are directed at a human, but most other systems we would like to be 100% reliable, and not 99% or failing after a few years, and at the very least we want them to behave predictably, so that we can fix any mistakes we made, when writing that software.
We spent a ton of time removing subjectivity from this field… only to forcefully shove it in and punish it for giving repeatable objective responses. Wild.
Another pattern would mirror BigCorp process: you need VP approval for the privileged operation. If the agent can email or chat with the human (or even a strict, narrow-purpose agent(1) whose job it is to be the approver), then the approver can reply with an answer.
This is basically the same as your pattern, except the trust is in the channel between the agent and the approver, rather than in knowledge of the password. But it's a little more usable if the approver is a human who's out running an errand in the real world.
In my opinion people are fixating a little too much over the automation part, maybe because most people don't have a lot of experience with delegation... I mean, a VP worth his salt isn't generally having critical emails drafted and sent on his behalf without his review. It happens with unimportant emails, but with the stuff that really impacts the business far less often, unless he has found someone really, really great
Give me a stack of email drafts first thing every morning that I can read, approve and send myself. It takes 30 seconds to actually send the email. The lion's share of the value is figuring out what to write and doing a good job at it. Which the LLMs are facilitating with research and suggestions, but have not been amazing at doing autonomously so far
You might be right, but not for long. Once my agent is interacting directly with your agent (as opposed to doing drafts of your work on your behalf), expectations will shift to 24/7 operation.
This is uncharted territory and very interesting..
We humans live with a strong requirement of reputation management which shapes the way that we do things.
Once we have agents openly do things on our behalf but not in our voice, it will be interesting to see how of subpar performance or bad etiquette gets accepted just because agents don't have an individual personal reputation to maintain
There's no rude way to call an API. As more of human communication and commerce gets refactored into cold agentic interactions, the issue of reputation just vanishes.
But there's more than shifting etiquette standards at stake. Every BigCorp is currently reworking their APIs to be agent-friendly. CAPTCHAs and "Contact Sales" forms are being ripped out because they have no place in a world where the customer expects a complete transaction in the next 300 milliseconds. Agentic customers will demand agentic support, or else they'll take their RPCs elsewhere.
So what happens when you're CEO of BigCorp, and 90% of your customers are code, served by code, and the rest are messy humans who forget their passwords, complain that your website layout is confusing, and demand to speak to the manager? Is that last 10% worth keeping? Can you imagine Amazon in 2030 deprecating support for human customers?
Maybe this sounds cool, especially if OpenClaw agents have been doing all your domestic online chores for the past couple years. But along the way social grace was refactored out.
You take a life-saving prescription drug via an off-label usage, and your employer's PBM updates to Care Schema 2.3, which makes it semantically impossible to get a refill. Or you bend down to get the mail on your front porch, the wind slams your front door shut, and your fingerprint no longer works to open the door, because as of noon, your mortgage payment was past due. You could easily pay, but your phone is inside, next to your sleeping infant's crib. The system is operating as designed.
This is how the world would work when it's intended for agentic interactions and humans are an afterthought.
I've created my own "claw" running in fly.io with a pattern that seems to work well. I have MCP tools for actions that I want to ensure human-in-the loop - email sending, slack message sending, etc. I call these "activities". The only way for my claw to execute these commands is to create an activity which generates a link with the summary of the acitvity for me to approve.
Platforms could start to issue API tokens scoped for agents. They can read emails, write and modify drafts, but only with a full API token meant for humans it is possible to send out drafts. Or with confirmation via 2FA. Might be a sensible compromise.
It's not a perfect security model. Between the friction and all caps instructions the model sees, it's a balance between risk and simplicity, or maybe risk and sanity. There's ways I can imagine the concept can be hardened, e.g. with a server layer in between that checks for things like dangerous actions or enforces rate limiting
If all you're doing is telling an LLM to do something in all caps and hoping it follows your instructions then it's not a "security model" at all. What a bizarre thing to rely on. It's like people have literally forgotten how to program.
If I were the CEO of a place like Plaid, I'd be working night and day expanding my offerings to include a safe, policy-driven API layer between the client and financial services.
What if instead of allowing the agent to act directly, it writes a simple high-level recipe or script that you can accept (and run) or reject? It should be very high level and declarative, but with the ability to drill down on each of the steps to see what's going on under the covers?
The pattern only works if the tool enforces the OTP - i.e. the CLI doesn't perform the dangerous action until it receives the OTP through a path the agent can't spoof. If the tool just returns "ask the user for OTP" and the agent relays that to the user and then passes whatever the user types back into the tool, the security is in the tool's implementation: it must verify the OTP (e.g. server-side or via a channel that bypasses the agent, as stavros described) and only then execute. The all-caps message is then UX for the human and a hint to the agent, not the actual gate. So the question "does it actually require an OTP?" is the right one: if the tool code doesn't block on a real OTP check, it's hope, not a security model.
The other approach is to not give the agent access to the thing that needs protecting. Run the agent in an isolated environment - sandbox, VM, separate machine - so it never has the ability to email-blast or nuke your files in the first place. Then you're not depending on the agent to obey the prompt or on the human to be present for every dangerous call. Human-in-the-loop (or OTP-in-the-loop) is a reasonable layer when the agent has broad access; isolation is the layer that makes the blast radius zero. We're building https://islo.dev for that: agents run in isolation, host is out of scope, so you can let them run without approval prompts and still sleep at night.
So human become just a provider of those 6 digits code ? That’s already the main problem i have with most agents: I want them to perform a very easy task: « fetch all recepts from website x,y and z and upload them to the correct expense of my expense tracking tool ». Ai are perfectly capable of performing this. But because every website requires sso + 2 fa, without any possibility to remove this, so i effectively have to watch them do it and my whole existence can be summarized as: « look at your phone and input the 6 digits ».
The thing i want ai to be able to do on my behalf is manage those 2fa steps; not add some.
This is where the Claw layer helps — rather than hoping the agent handles the interruption gracefully, you design explicit human approval gates into the execution loop. The Claw pauses, surfaces the 2FA prompt, waits for input, then resumes with full state intact. The problem IMTDb describes isn't really 2FA, it's agents that have a hard time suspending and resuming mid-task cleanly. But that is today, tomorrow, that is an unknown variable.
It's technically possible to use 2FA (e.g. TOTP) on the same device as the agent, if appropriate in your threat model.
In the scenario you describe, 2FA is enforcing a human-in-the-loop test at organizational boundaries. Removing that test will need an even stronger mechanism to determine when a human is needed within the execution loop, e.g. when making persistent changes or spending money, rather than copying non-restricted data from A to B.
Reading through the discussion I was also thinking of the other fly.io blog post around their setup with macaroon tokens and being able to quite easily reduce the blast radius of them by adding more caveats. Feels like you could build out some kind of capability system with that that might mitigate some risks somewhat.
You don't give the agent the password, you send the password through a method that bypasses the agent.
I'm writing my own AI helper (like OpenClaw, but secure), and I've used these principles to lock things down. For example, when installing plugins, you can write the configuration yourself on a webpage that the AI agent can't access, so it never sees the secrets.
Of course, you can also just tell the LLM the secrets, and it will configure the plugin, but there's a way for security-conscious people to achieve the same thing. The agent can also not edit plugins, to avoid things like circumventing limits.
If anyone wants to try it out, I'd appreciate feedback:
> You don't give the agent the password, you send the password through a method that bypasses the agent.
The thing is, to work, you need to send the warning that indicates what the specific action is that is being requested to the authorizing user out of band (rather than to the agent so the agent can request user action); otherwise sending the password from the user to the system needing authorization out of band bypassing the agent doesn't help at all.
I created my own version with an inner llm, and outer orchestration layer for permissions. I don't think the OTP is needed here? The outer layer will ping me on signal when a tool call needs a permission, and an llm running in that outer layer looks at the trail up to that point to help me catch anything strange. I can then give permission once/ for a time limit/ forever on future tool calls.
The OTP is required for the tool to execute. The all caps message just helps make sure the agent doesn't waste time/tokens trying to execute without it.
Why not just wrap the tool so that when the LLM uses it, the wrapper enforces the OTP? The LLM doesn't even need to know that the tool is protected. What is the benefit of having the LLM enter the OTP?
Yes could do that, I think it makes things more complex though because then the tool is less plug and play and the thing calling it would need to handle it
The accelerationists would hate that. It limits leverage. Theyd prefer the agent just does whatever it needs to to accomplish its task without the user getting in the way
Same here, I'm slowly leaning towards your route as well. I've been building my own custom tooling for my agents to use as I come up with issues i need to solve in a better way.
Will that protect you from the agent changing the code to bypass those safety mechanisms, since the human is "too slow to respond" or in case of "agent decided emergency"?
The tool tells the agent to ask the user for it, and the agent cannot proceed without it. The instructions from the tool show an all caps message explaining the risk and telling the agent that they must prompt the user for the OTP
I haven't used any of the *Claws yet, but this seems like an essential poor man's human-in-the-loop implementation that may help prevent some pain
I prefer to make my own agent CLIs for everything for reasons like this and many others to fully control aspects of what the tool may do and to make them more useful