Pick models by testing them on the real job
A developer compared several models on a UAV software project and found that detailed plans still didn’t stop weaker ones from making serious mistakes.
- What they did
- The developer was working on software for UAV CAD, CFD, and DFM optimization. They prepared thorough requirement, design, and implementation documents, then ran smaller models and the strongest open-weight options through those materials to see how they handled the task. Based on the comparison, they chose Opus to shape and check the design, and Flash 3.8 to generate and build the code.
- How it went
- The open-weight models still introduced many bugs and major blunders, even with the written plans in place. Opus and Gemini 3.8, and now Argon, handled the work much better. The developer’s takeaway was that compact models can still be workable for simpler applications.
- Tips
- Don’t assume a detailed spec will prevent a model from failing on a niche engineering problem. Try the model on your own hardest examples before you commit to a lower-cost stack.
Try it yourself with Claude
I’m making a [kind of engineering tool] for [domain]. Start by outlining a brief product plan and a technical approach, then stop and ask for my approval before writing code. After I approve, build it in [folder/repo] with tests and a README explaining how to run it. When you finish, inspect your own work for mistakes and clearly mark anything that is still uncertain, approximate, or not fully checked. Do not take any irreversible step, such as sending, deleting, submitting, or paying, without asking me first.
Fill in the [brackets]. Agents can make mistakes, so review anything before it is sent, paid or deleted.
Get 3 agent uses like this every morning
With the day's top AI news. Free, and one click to unsubscribe.
Related agent uses
Have Claude take a bioinformatics challenge from paper to report
At a hackathon, Pedro Beltrao’s team used Claude agents to work through 13 bioinformatics prediction problems end to end.
Map ROM functions with Claude in Ghidra
A developer used Claude inside Ghidra to inspect an Atari River Raid ROM, identify key routines, and rename them.
Generate editable CAD models from a text prompt
An agent can draft a 3D part from plain language, then let you fine-tune key measurements before export.
Use an agent to hunt for smart contract bugs
A red-team benchmark let Claude search forked chains for contract weaknesses and draft exploit code in an hour.
Use Claude to verify code against crash bugs
Anthropic’s red team had Claude turn fuzz tests into formal specs and prove that selected RISC-V binaries would not crash on valid inputs.
Use Claude to probe hardware and write a driver
A user had Claude inspect a USB device and help build a new driver for hardware that broke on a newer macOS.