My first software factory was arguably the humble Ralph loop, 14 months ago.
Then I wrote IssueOps, a factory that ran entirely in GitHub Issues and Actions, so I could offload my laptop and keep building while travelling. That cost too much. API billing, amirite?
So I brought the build farm home and Anneal was born. It ran on my Claude subscription locally and wrapped Ralph loops with outer loops, guardrails and tickets. But it lived inside each project, and every copy soon drifted.
About 12 weeks ago I started Holodeck. Instead of putting the factory in the project, I build the projects in the factory. The first project it built was Holodeck itself.
What I've learned:
1. Deterministic gates are the real quality mechanism. The model isn't trusted to judge its own work. Hooks and tests run outside the agent loop. Cheaper models fail and retry instead of committing bad code.
2. The same agent never writes and grades its own work. Test author, implementer and reviewer are separate agents, on different models.
3. Detailed documents first, fast execution after. The spec is now the hard part.
4. Fresh contexts. Always. Tickets keep a log of what was tried and failed so the next loop doesn't make the same mistake.
5. An orchestrator with subagents. The top model coordinates and doesn't do grunt work.
6. Isolation by default. Every agent that edits code gets its own worktree. Runners are throwaway Docker containers with no Docker socket, no source mount and no host ports.
7. Agents get references to secrets, never the values. Enforcement sits in the tooling and hooks, not in the prompt. (I have a product for this coming.)
8. Store a detailed event log. Agents can interact with the API to emulate usage, see real-time logs, and self-improve. Even self-heal.
9. Tools are disposable. Every model release makes some tools redundant, so refactor your stack.
Everyone is suddenly talking about building their own "software factory".
Have you built one? If so, how? What does it do?