I built an open source i18n check that actually reads the translation, not just the key list

Most i18n checkers diff key sets. That catches the missing key and then stops.

The bug that actually ships looks like this. Every key present, placeholders intact:

en: "This will permanently delete your account" es: "Esto guardará tu cuenta de forma permanente" ("will save") 

Nothing structural is wrong with that file. You only catch it by reading it.

npx u/shipi18n/cli check ./locales -s en runs the structural pass with no API key at all: missing and orphan keys, placeholders dropped across eight syntaxes, plurals collapsed to one form, empty values, glossary violations. Add --semantic and your own LLM key and a judge reads each source/target pair and flags mistranslation, omission, or addition. Changed keys only, majority vote across 3 passes, advisory unless you opt into failing the build.

I tried to measure this instead of just asserting it. UIStringBench is 228 pairs, 54 with planted errors and 168 clean, committed before I wrote the judge, thresholds fixed up front. Every model I tried caught 54/54 (Claude, GPT-5.x, Gemini). False positives came in at 3.6% on the stronger ones (Sonnet 5, Opus 5, GPT-5.5) and 7.7% on the smallest. Structural checks catch 0 of the 54, which is sort of the whole point. Harness and corpus are in the repo, so you can run it against whatever model you'd actually use.

Formats: JSON, YAML, Flutter .arb, Apple .xcstrings. SARIF output so it annotates PRs. Apache-2.0, no account, no server. There's an MCP server too if you review translations from Cursor or Claude Desktop, and its validators don't need a key either.

The false positive rate is the number I care about most, so if you run it and it yells about something that's actually fine, I want to hear about it.

https://github.com/Shipi18n/shipi18n