All articles

CI in 2027: Why the future of coding needs instant back-pressure

Matt Moore Co-founder and CTO

This is the first post in a three-part series discussing how CI is changing in the AI era. Parts two and three are coming soon.

On August 6, GitHub Actions was degraded for nine hours and nine minutes. At the incident’s peak, 71% of workflow runs failed outright. Self-hosted runners failed too because the Actions control plane broke, not the runners themselves.

To put that in perspective, a 99.9% availability target allows about eight hours and 46 minutes of downtime in an entire year. That one incident blew through all of it. Even with perfect uptime for the rest of the year (which it hasn’t had), Actions can not hit three nines. GitHub’s published availability for Actions is 99.33%, and GitHub has been open about its recent struggles.

For us, the downtime meant the pipeline that patches the images the world runs on stopped. Nobody could merge, build, or ship a Common Vulnerabilities and Exposures (CVE) fix. Every time GitHub has a bad day, we have a terrible day. We recently blogged about Chainguard Checks, the solution we built to prevent this from happening again.

But the outage was the trigger, not the root cause. Continuous Integration (CI) was designed in 2015 for humans who push a few times a day. That isn’t how software gets written in 2026.

Agents need back-pressure

The key to a successful agent loop is strong back-pressure. Coding became a killer application of AI because software is where we’ve invested the most in making failure objective. We compile, test, lint, and fuzz. An agent loop is only as good as the “no” it receives.

The catch is that an agent can choose not to use those tools, or claim it did. Anyone running agents at scale has seen a report that all tests pass for a change whose transcript contains no test run at all. A model’s report on its own work isn’t back-pressure. It’s what back-pressure exists to check. And an agent that runs compilers and test suites is executing untrusted code, so it also needs a safe place to run them.

Our first answer was the simplest thing that could possibly work: We pointed our agents at CI. The agent drafts a change, we push a pull request, and when CI fails, we turn the failures into structured findings and invoke a fresh agent to fix them. Repeat until green, up to five times before a human takes over. It worked, because CI runs on every change and can’t be hallucinated.

What that loop cost us

While it worked, this loop was expensive across almost every dimension. Every cost of the cycle compounded as our fleet grew, and each pain point directly informed the architectural requirements for what we had to build next.

Slow iteration. Every “no” was painfully slow because it required a full pull-request roundtrip. Waiting tens of minutes just to learn that a change had a trivial syntax error or a failing test completely shattered the tight feedback loop an agent needs to converge. Much of that time, and billable compute, was wasted on cold runners re-downloading the exact same dependencies, pulling heavy container layers, and compiling unchanged packages from scratch. Instead of giving the model instant back-pressure, we were forcing it to wait in an asynchronous queue, stretching what should have been a 30-second fix into an hour-long multi-PR ordeal.

Lost context. When a run failed, our mitigation was to spin up a fresh agent, hand it the failure logs, and hope for the best. But a summary of a conversation is not the same thing as the conversation itself. Each retry began as a cold start, with a new model instance forced to read its predecessor’s lossy notes and piece together the intent behind the diff. We were burning tokens just rebuilding context between tries, paying multiple times over for the model to re-orient itself and re-derive conclusions the previous agent had already made. The continuous chain of thought that makes iterative debugging effective was wiped clean every time the CI gate turned red.

GitHub in the loop. Tying our agent feedback directly to pull requests meant every single step rode GitHub’s API. Every git push, status poll, check-suite event, and log fetch ran straight into API quotas, aggressive rate limits, and flaky status reads where the API lagged behind the actual run state. Worse, it tied our core operational tempo to someone else’s platform reliability. We felt it keenly whenever GitHub struggled, culminating in days like August 6 when their control-plane degradation halted our patch queue entirely. We had placed a third-party API right in the middle of our inner loop.

Secrets beside untrusted code. CI runners routinely pair long-lived credentials alongside code from strangers. CI vendors call this “Remote Code Execution as a Service,” and they are not wrong. When an agent is writing patches, compiling third-party dependencies, or testing external fixes, it is executing untrusted code by definition. Giving that process access to an environment containing deployment keys, internal tokens, or cloud credentials is a massive security liability. Traditional CI relies on fragile configuration to keep secrets away from test steps, but if an agent can execute arbitrary shell commands, that boundary becomes paper-thin.

Cost. We were paying every single day for GitHub-hosted runners just to service two image-maintenance bots. What made the bill particularly maddening was that we weren’t paying for actual computation or novel progress; we were paying for repetitive setup work that produced nothing new. Over and over, runners would spin up, pull gigabytes of identical toolchains, run for two minutes, and discard the disk. We were effectively funding a distributed caching failure, burning significant budget on runner minutes just to watch VMs set up the same environment over and over again.

Laptop collisions. When engineers tried to work around CI latency by running agent sessions locally, the friction simply migrated to their laptops. An engineer might run five or six agent sessions in parallel to tackle multiple issues at once, only to watch them descend into chaos. The agents would fight over the same local ports, kill each other’s processes, corrupt shared build caches, and trample development daemons. Even worse, those agents were executing arbitrary commands within the engineer’s local environment, putting local SSH keys, cloud CLI configurations, and environment secrets directly within the blast radius of untrusted code. 

We had to build our way out of this 

Relying on standard pull request pipelines to provide agent feedback was a dead end. We needed a system where the execution environment is completely isolated, failure is structurally impossible to fabricate, and feedback arrives in seconds, not after a 20-minute CI queue.

In Part 2, we’ll dive into the low-level systems architecture of how we iterated on what we learned. We’ll explore how we adapted our SLSA L3 build technology into ephemeral microVMs, engineered an out-of-guest network proxy to inject secrets in flight, and built a lazy virtual filesystem that boots gigabytes of toolchains faster than a network download, all in service of delivering real back-pressure at machine speed, without handing untrusted code the keys to our infrastructure.

Share this article

Start building with Chainguard