Sort failed CI runs and repair flaky tests

A QA engineer used OpenClaw to read failed builds, label the cause, alert teammates, and reduce test flakiness.

Done with OpenClawยท Intermediate

What they did
They set up OpenClaw to watch GitHub Actions failures, then inspect the logs and test output for each run. The agent decides whether the problem looks like a flaky test, a true defect, or an environment issue, then sends its findings to GitHub and Slack and creates issues when needed. They also added a second agent for self-healing locators that explores alternate selector strategies and recommends fixes when tests break.
How it went
The average diagnosis time fell from 45 minutes to 6 minutes. Flakiness dropped from 8% to 0.8%. In one case, the self-healing agent tried to force a test to fit the observed output instead of repairing the locator, and code review stopped it.
Tips
Use a confidence threshold before any automatic merge or code change is allowed. Keep the input tight: when they narrowed the agent to the most relevant 500-1000 tokens, monthly token spend fell from nearly $4,000 to $400.

Try it yourself with OpenClaw

Check failed GitHub Actions runs for [repo] from the last [7 days]. Read only the most relevant logs and test output, then sort each failure into one of three buckets: flaky test, real bug, or environment problem. Send a summary to [Slack channel] and open a GitHub issue for each real bug. For flaky tests, suggest likely locator or timing fixes, but stop and ask for my approval before editing code, changing assertions, posting anything, opening issues, or merging. Do not take any irreversible action without my explicit confirmation.

Fill in the [brackets]. Agents can make mistakes, so review anything before it is sent, paid or deleted.

Read the original on suneetmalhotra.com โ†—

Get 3 agent uses like this every morning

With the day's top AI news. Free, and one click to unsubscribe.