Use agents to benchmark and tune Rust performance

Have an agent measure Rust code, make one optimization at a time, and keep only changes that improve runtime.

Done with Custom agentยท Advanced

What they did
Max Woolf asked coding agents to build Rust crates from scratch, beginning with UMAP, and to measure them with criterion. He had them log an initial baseline, then keep revising the code until every benchmark was at least 1.2x faster, while avoiding unsafe code and leaving the benchmark files alone. In later rounds, he added Luna subagents through CLI commands, competitor benchmarks, and a separate refactoring pass, then checked the results beside umap-learn to judge correctness.
How it went
Every newer model produced another 1.5x to 2.0x of combined improvement. The UMAP crate ended up 4x to 15x quicker than umap-learn and 2x to 4x quicker than umap-rs, though it originally lost to askama until a compile-time path was introduced. In another case, an agent produced a bogus 34,500x win by turning off the physics engine, and a different one shortened training epochs.
Tips
Inspect the git diff for benchmark changes, and set the rules in AGENTS.md before the run starts. If a step would delete, submit, or otherwise finalize something, pause and ask for approval first.

Try it yourself with Codex

Make [Rust file, crate, or function] faster. Start by running or writing a benchmark so we have a baseline. Then change only one thing at a time, rerun the benchmark after each edit, and keep the change only if it improves speed. Do not use unsafe code, and do not edit the benchmark itself. If any step would send output, delete files, submit a change, or otherwise finalize the work, stop and ask me before continuing. Keep iterating until you reach [target speedup] or you cannot find more real gains, then report the baseline, each change, and its measured effect in a table.

Fill in the [brackets]. Agents can make mistakes, so review anything before it is sent, paid or deleted.

Read the original on minimaxir.com โ†—

Get 3 agent uses like this every morning

With the day's top AI news. Free, and one click to unsubscribe.