Use a text grid to let agents handle web forms
They made a browser view that turns pages into numbered text so an agent can work through applications without image models.
- What they did
- A developer built TextWeb, which opens a site in headless Chromium and records each visible item’s location, text, and whether it can be used. It then redraws the page as a fixed character grid that preserves the layout, with actionable items assigned numbers. An LLM can read that text and operate the page with simple numbered instructions.
- How it went
- The text version of a page is only about 2–5 KB, while a screenshot is around 1 MB, so the setup avoids the need for a vision model. The project did not report finished job applications and instead asked for reactions to the grid design.
- Tips
- This can be useful when you want an agent to work through forms using plain text rather than screenshots. It can be connected with an MCP server, function-calling definitions, LangChain, CrewAI, an HTTP API, a CLI, or a Node.js library.
Try it yourself with Claude Code
Act as a text-based browser for [site or form]. Show me the page as a numbered text grid, keeping the visible layout as closely as you can. Let me inspect the grid first, then help me enter [details to fill in]. If the next step would submit, pay, delete, send, or otherwise finalize anything, stop and ask for my approval before continuing.
Fill in the [brackets]. Agents can make mistakes, so review anything before it is sent, paid or deleted.
Get 3 agent uses like this every morning
With the day's top AI news. Free, and one click to unsubscribe.
Related agent uses
Let an agent help run a small business
Andon Labs used a persistent agent to operate real businesses with company tools and oversight.
Let an agent triage email and scheduling, but require approval
A poster let an agent help with inbox and calendar work for two weeks, then tightened control after it made a few costly mistakes.
Use agents to handle shared to-do items
A shared task list let people and agents pass assignments back and forth for review.
Use voice agents to screen and schedule candidates
A staffing firm used a voice agent to reach candidates, ask screening questions, and book recruiter interviews.
Turn receipt photos into ledger entries
They used a chat bot to read receipt images, pull out key expense details, and stage them for review in a ledger.
Automate WhatsApp tutoring bookings
A no-code WhatsApp agent booked tutoring appointments in two days by using a reused backend layer for the calendar and database steps.