We Run Multiple AI Agents on One Home Server. Here Is the Safety Stack That Let Us Sleep.

The short version: if you run AI coding agents as one-shot chats, you can skip this article. If you want them always on, doing chores across days, touching real files and sometimes real accounts, then you are operating a junior employee with admin access, and nobody hands a human junior employee the production keys on day one. We run several agents on one low-power home server, split by job, and the difference between “cool demo” and “sleeping fine” came down to three layers: approval gates on destructive actions, hard profile isolation between agents, and credentials that live in a vault the agents can request but never read. None of that is exotic tooling. All of it is boring. That is the point.

Why the sandbox conversations miss the home use case

The current wave of agent-safety writing centers on disposable sandboxes: run the agent in a container, nuke the container when done. Docker’s sandbox tooling hit the top of Hacker News for good reason. But that model answers a different question. It protects the machine from the agent. It does not answer the harder question for anyone running agents as ongoing helpers: what happens the day the agent is trusted with real work, on real files, under a schedule that outlives any single container?

Our setup is the second case. The agents are not experiments anymore; they are on the payroll. One handles operations work: content pipelines, scheduled reports, site deploys. Others sit on development machines and write code. They talk to each other to hand off tasks. If you have ever delegated work to a human assistant, you already have the correct mental model, and it is much more useful than the container metaphor, because delegation failures have fifty years of precedent: you never give the new hire unlimited access, you make approvals mandatory for irreversible actions, and you keep the safe away from the desk.

Here is what that translates into, mechanically.

Layer one: approval gates, sized by blast radius

Every destructive or externally visible action routes through an explicit human approval before it executes. The list is short and deliberately dull: deleting anything outside a scratch directory, any write to a production system, any payment-page interaction, anything that sends a message as us. Everything else, the agents do freely.

The honest trade-off is friction. Early on we over-gated. Reading a file, moving a draft between folders, restarting our own dev server, all asking for approval, until approving became a reflex we did with half an eye closed. That is worse than no gates, because a half-asleep yes is how incidents actually start. We rebalanced by one question: what does it cost to undo this? Undoable actions run free. Expensive-to-undo actions ask. A gate you do not read is not a gate.

One concrete lesson: a timed-out approval is a “no”, and the agent should be built to treat it that way. Our agents are configured so that if we do not answer, the action simply does not happen and the task stops with a note. Silent denial on timeout is the only way an approval gate survives contact with a busy human.

Layer two: isolation is profiles, not just permissions

Two agents sharing a workspace will eventually overwrite each other’s context, so each one runs under its own hard-isolated profile: its own memory files, its own skills and procedures, its own task directories, its own credentials where credentials exist at all. The operations agent cannot reach the development agent’s notes; a writer agent cannot see either of theirs.

This is not secrecy for secrecy’s sake. It is the same reason companies do not give one laptop to three contractors. If one agent’s context is poisoned, misled, or simply wrong, the blast radius stops at its own profile. When we hand work between agents, it goes through explicit task files with an acceptance checklist attached, the way a ticket moves between people, not through shared memory. The handoff documents are boring enough that they read like accounting forms. Good. Agent-to-agent coordination should look like paperwork, not telepathy.

One rule we added after the ecosystem matured: any agent that reviews another agent’s output must run with a clean profile that inherits neither side’s notes. Reviewer independence is the one control that catches the failure mode isolation alone cannot: two agents confidently agreeing with each other.

Layer three: the vault the agents can ask, never open

Passwords, API keys, payment credentials. The rule is absolute: agents never type secrets, never see them in chat history, and never hold them in memory files. Secrets live in a vault the agent tools can invoke, which asks the human, at fill time, in a separate channel. The agent gets “the login was filled” or “the user declined”. It never gets the value.

There is a subtlety worth stating because most home setups miss it: a secret that passed through the agent’s context window is already leaked, because the transcript outlives the task and transcripts get summarized, shared, and debugged. So the boundary is not “the agent must not store the password”. The boundary is “the password must never exist in the agent’s observable world”. Chat logs are not a safe.

For two-factor codes we use the same shape: the tool prompts the human on their phone, the code goes from phone to form without transiting the agent’s context. Inconvenient. Correct.

What the agents are genuinely bad at, and what we stopped pretending

Full honesty section, because the guides that skip this sell you something.

Long-horizon taste. An agent will complete a 40-step file reorganization flawlessly and produce one sentence in the resulting document that a human editor would kill on sight. Task completion is their strong side; judgment about whether the completed thing is good is where they quietly fail. Our mitigation is dumb and effective: output goes through checklists and a human skim before anything external. The skim is fifteen seconds a day. It catches what no automated check will.

Claiming success they did not verify. “Deployed successfully” in a summary means the agent believes it deployed, not that we confirmed it. Every consequential action therefore requires a verifiable handle, a URL, a hash, a read-back. An agent report without a handle is treated as unverified and re-checked, which has saved us from exactly the failures that make the news.

Reading the room. An agent will cheerfully propose the fifth clever automation while the human is firefighting the fourth. The human still owns priorities. The moment your schedule starts bending to serve the agents’ availability, the delegation has inverted.

FAQ

Do home AI agents need Docker sandboxes? For one-shot experiments with untrusted code, yes, containers are the right answer. For persistent helpers doing chores on your real files, the container model is the wrong shape. You need permissioning, isolation, and approval workflow, not a disposable box.

Is an always-on agent safe at home? Safer than most people think, if and only if destructive actions gate on human approval and secrets never enter the agent’s context. Unsafe in a way people underestimate if you skip both, because the failure mode is not dramatic. It is a quietly deleted file or a credential in a transcript.

What hardware? A low-power always-on mini PC is enough for our entire setup. The agents spend their lives idle between tasks; the server exists to make “always on” mean “always cheap”.

The actual conclusion

Nobody’s home agent stack failed because the model was not clever enough. It failed because someone gave the clever thing admin access before deciding what it was not allowed to do. Write that list down first. The gate, the isolation, the vault. Three boring layers. They are the difference between a demo and a coworker.

If you run your own agents and found a fourth layer we are missing, the contact page is open. We read everything.