The temptation to push a huge PR after a productive session with a coding agent is very hard to resist sometimes. It may matter less for a solo project, but a team working on a mature codebase in a professional setting may reject it and for good reasons. Nobody wants their core development environment flooded with complexity, risk and waste.
Trusting AI contributions outright is problematic, but relegating agents to only handling low-risk tasks is not the answer either. There must be a way to introduce cleanup into the process so vibe-coded work can be treated as a first-class citizen.
Why not create a workflow that solves for mess, like a code janitor?
What I’m looking for is a feature that enables me to contribute code safely and reliably and, crucially, not embarrassingly, after exploring an idea with a coding agent that I want to introduce into the main repo.
GitHub’s Copilot agents can handle tasks and workflows, but I want more oversight of the agents’ behaviors. Building the hand-offs, auditing, and reporting on GitHub’s own primitives gives you more options and visibility over their activities and should feel more like working with a real team of contributors.
I’ve been experimenting with this with Open AgentOS. After creating a PR the agents scramble around to make sure everything is copacetic, running tests and reviewing code. They then report their findings, leaving receipts as they go along.
The initial implementation of this janitor idea is optimized for observability. After creating the PR, a Watcher agent sees the new PR with no issue or plan and immediately triggers the orchestration flow. The Archaeologist agent picks it up and reconstructs intent from the code, creates an issue for it, and links the PR to the tracking issue. The Reviewer agent assesses adherence to scope and policies, making change requests and updating labels for the appropriate hand-off. Then the Reviewer agent leaves instructions for you in the comment, waiting for your approval on next steps which may or may not involve asking the Builder agent to work on the code.
There are a few ways this system could evolve. For example, the system could make decisions such as whether to split up work so that it adheres to a policy or standard the team may have in place. Or it could make it easy to manage any number of arbitrary workflows that utilize the labels for state management. Or perhaps it could use the receipts to learn from itself and tune the system.
I’m using the AgentOS configuration in an experimental repo as a reference app here https://github.com/open-agentos/agentos-pmqs.
You can see the agents walk through the process in this PR https://github.com/open-agentos/agentos-pmqs/pull/8
- Saw the PR, created issue #9
- Reconstructed intent from the diff, wrote it up
- Labeled it for handoff
- Told me what to do next
- Read my approval, relabeled for Reviewer
- Built a review checklist, worked it line by line
Finally, it reported out a receipt showing the tokens used, duration and total cost. That data allows me to do performance analysis for iterating on the system, as I’ve talked about here.
A fully integrated LLM-and-harness stack like Claude Code will be able to work in similar ways and perhaps feel more like an extension of the individual driving the work than this more deterministic and structured approach I’m working on. But I’m starting to see how bespoke agent operating systems may be more valuable in a professional team setting where the risks are higher, reliability matters more, and being able to monitor and improve the system itself is non-negotiable.
Leave a Reply