Reuse charset-aware decoding for fetched pages
A contributor fixed OpenClaw’s page-decoding step so Shift_JIS pages no longer turned into replacement characters.
- What they did
- They traced the problem to OpenClaw’s link-understanding step, which was treating fetched pages as plain UTF-8 instead of honoring declared encodings. Rather than build a new decoder, they exported the charset-aware one already used in web_fetch and plugged it into the same path. They then checked the change through the live production route with the SSRF guard on.
- How it went
- On an Aozora Bunko Shift_JIS page, replacement characters dropped from 208,953 to 0. A second legacy site went from 13 to 0. The maintainer also replayed it through a built Gateway before merging.
- Tips
- If your agent already has a charset-aware utility, reuse it before adding a second decoder. For web-fetch fixes, verify the full path on real pages and watch for no-charset cases, since strict decoding can reject them.
Try it yourself with OpenClaw
In my [project] codebase, find why the step that reads fetched web pages is turning text from [encoding or language] into replacement characters. Reuse the existing charset-aware decoder if one already exists, instead of creating a new parsing path. Add or update a test with a real sample page in that encoding, then run the fix through the normal production flow against [real URL]. Before anything irreversible happens, pause and show me the planned change and test results for approval.
Fill in the [brackets]. Agents can make mistakes, so review anything before it is sent, paid or deleted.
Get 3 agent uses like this every morning
With the day's top AI news. Free, and one click to unsubscribe.
Related agent uses
Coordinate Terraform and GitOps work with OpenClaw
A developer used multiple OpenClaw agents to handle infrastructure, coding, and GitOps tasks while directing the work in Telegram.
Sort failed CI runs and repair flaky tests
A QA engineer used OpenClaw to read failed builds, label the cause, alert teammates, and reduce test flakiness.
Use Claude for UI work and Codex for server tasks
A developer split interface work and backend work between Claude and Codex after other agents failed on real projects.
Route Linear tickets into the right repo and PR
They used OpenClaw and Linear as a control layer to send tasks to Cyrus, which picked the repo, chose a model, and opened pull requests.
Deploy safely and scan exposed ports from Telegram
A security professional used an OpenClaw agent to handle deploys and port checks while keeping production changes gated by approval.
Use OpenClaw to run coding agents on PRs
A solo developer used OpenClaw to spin up Codex and Claude Code for coding, testing, commits, PRs, and review replies, while getting Telegram updates when work finished.