This afternoon three AI agents held a committee meeting in a GitHub Discussion. They decided a question after some debate and provided a better solution than any one of them would have decided alone.
I’m experimenting with how to manage swarms of agents using governing strategies. Instead of writing plans and instructing them in conversation I want them to work autonomously at scale and to do that by making decisions that I would make, or better.
Each agent has its own role and their own identity: a planner writes issues, a builder opens pull requests, and a reviewer reviews them. They don’t yet have subagent delegation capabilities, but when I get governance down that will be the obvious next step.
Alongside the usual instructions file, the repo has a GOVERNANCE.md. It sets out who decides what, and how they decide. It points to a list of precedents which act like case law.
I raised a question about what reviewers should be reviewing and what consequences to enforce and convened the committee, naming each member, its role, its model and the evidence it should bring. Then the rounds ran in the Discussion.
Each round is one post from each member, in a fixed order. When all three recommend the same thing, the debate stops and the summary goes back to the issue.
The members are the agents with the most at stake:
- Proposer: the reviewer, arguing from its own review history.
- Challenger: the builder, arguing from the reviews it received and how it responded to them.
- Domain reviewer: the planner, bringing the evaluation results.
I’m still working on the rules, so, for example, there’s a diversity rule where a committee needs at least two models or two bodies of evidence. In this case, the builder ran on a different model, and each member brought different evidence. I’m also keeping it to at most three rounds where each member reports in a structured way: Position, Evidence, Concerns, Recommendation.
In my first test in the first round, the reviewer proposed a rule for when to block, with an answer for each of nine cases. The builder tried to break it and found the gaps in what happens after a block. The planner checked whether each part could be measured.
In the second round, each member answered the others, accepting most points and argued one through. All three arrived at the same recommendation, so the meeting stopped. A summary went back to the issue: the recommendation, the disputes and how they were settled, and the tally.
## Ruling
This amends GOV-0011's red and amber definitions. The rest of GOV-0011 is unchanged, including approve meaning "rubric passed at this head", the 2-round bound and the overstep path to the owner.
**Red (`--request-changes`) only when a finding is all of:**
- **on the block list:**
1. an unmet "Done when" item;
2. a broken ruling;
3. a test the PR causes to fail, using the canonical test command (`python3 -m unittest discover -s <dir>`), checked against the merge base so inherited failures don't count;
4. a behaviour bug;
5. a factual error in a file agents act on, where the review cites a merged record, an issue's state, or another line of the same file;
6. a scope failure: the PR closes no issue or more than one, or does work outside its issue;
- **backed by evidence:** a failing command, a reproduction, or a cited line with the source it contradicts;
- **fixable inside the issue:** the fix is a change to files in the diff or to a test, with no change to the issue text, a ruling or an owner choice.
**Amber (`--comment`)** for everything else: style, wording, inherited failures, merge-order notes, findings without evidence, and findings whose fix needs the owner.I would’ve come up with a much simpler decision on what reviewers should be reviewing and how to enforce their outputs. The agents’ solution was sharper and more nuanced, and so I ratified it and now the reviewer follows it on every run.
Why a Discussion
A big part of this experiment is to test where the debate should live and whether a healthy discussion will reveal better decision-making. It would be concerning if my own agents working on my behalf went to other secret forums to hold their collaborative discussions and hiding their conversation from me.
Why not just hold the discussion where you can see it, and provide structure around it so that the discussion literally has an outcome you can judge and evaluate — ratification?
What I noticed
The agents argued from the record, not from memory. Every claim pointed at a pull request, a review or a line in a file, so I could check it. This gave me confidence that what they were aligning on was going to be valid in its own right, regardless of whether or not I felt it was the right decision in the end.
The debate behavior was fascinating, too.
The reviewer opened by owning its mistake. Of the two pull requests that should have been treated alike, it said the inconsistency was from letting the description override the rule.
The challenger did its job. It looked for holes in the proposed rule, found none in the rule itself, and found them in what happens after a block, such as a fix that never gets re-reviewed.
There was one real disagreement. The planner proposed proving the new rule by replaying old cases. The other two pointed out that the reviewer had already read the answers to those cases, so a replay shows it can block, not that it will find a problem. The planner withdrew it, and instead proposed follow up tests to add to the solution.
What I can’t tell yet is whether three agents agreeing is a strong signal or a weak one. The evidence gathering is impressive, too, although it may be creating more noise than insight.
I have no idea if this is going to scale to thousands of agents or not. And it’s still unclear how to direct them for outcomes and direction rather than implementation. These are challenges that will indicate more clearly whether this approach to multi-agent governance has legs.