Missing from the agent loop discussion is the product view of the system doing the looping. Where is the insight into how a project is progressing? Anything happening above the agent loop level is the responsibility of an orchestrator function, but what does that workflow look like? How do you steer the orchestrator and monitor that the agent loops responding to it are achieving the larger goals of the project?
After exploring ways to use GitHub-as-AgentOS (Part 1, Part 2) at the agent loop level, I figured the Project Board feature might solve this problem, and I think it does. It gives you something closer to a command center with a view across the work that is in play.
With webhooks triggering on issue label changes you can assign agents to do reviews and implement change requests and push code or whatever you allow them to do. GitHub Apps let you scope each agent’s identity and permissions independently, so a “reviewer agent” can’t accidentally merge, and a planner can’t push code. A “planner” agent can then spin up new sub-issues in a larger feature and label them appropriately to trigger getting the work done.
And here’s where it gets really useful from a higher level… the collection of issues for a feature can be monitored via a Kanban-style view or a Roadmap scheduling view. What are my agents working on right now? And I don’t mean “which tool call did they just make?”. I mean, “what is the status of my feature?” Are there a lot of issues to process? Which ones are done? Which are coming up next? Which are stuck? It can make it clear that, for example, three issues merged overnight but one got blocked and is awaiting a decision from the orchestrator agent.
The tool stream data breadcrumbs that AI agents leave along the path from start-to-outcome can shine a light on things like which models are achieving the best balance between cost and result, indicators of token wastage, effects of pipeline changes, etc. That’s the foundational data this builds upon.
We can aggregate the data at the project level. That data may show that certain workflows give better outcomes for certain types of features. I can A/B test processes as well as models, such as a planning-led process with acceptance criteria or a short-burst implementation with code-cleanup agents or a documentation-heavy investigation informing a PRD and so on.
This feature-level observation layer over AI coding is starting to give me confidence as a product person that I can use agents to drive an outcome that has more moving parts and pieces. I trust it to build things, but without oversight of what’s actually happening and what’s been done and not done, it’s hard to trust it to build important things. GitHub-as-AgentOS is gradually proving to me that the solution is in the tools we already use every day.
Leave a Reply