fix: fall back to English per-key when locale is missing a msgid - #440
Open
jacalata wants to merge 5 commits into
Open
fix: fall back to English per-key when locale is missing a msgid#440jacalata wants to merge 5 commits into
jacalata wants to merge 5 commits into
Conversation
Previously, gettext.translation was called with a single-locale languages list. If the loaded catalog was missing a msgid the raw key was surfaced to the user instead of the English text. The chained locale_options list in set_client_locale only handled catalog *load* failures, not per-key lookups. Chain "en" behind the preferred locale so per-key fallback works. Also: - Add CI workflow to detect drift between committed .mo files and the .properties sources they are compiled from; regenerates and diffs on every PR that touches localization files. Uploads the regenerated .mo files as an artifact on failure so they can be downloaded and committed. - Fix wrong doit task name (combine_property_files -> properties) in bin/i18n/README.md, and update the English-fallback claim to match the new behavior. - Remove duplicate Localization section from contributing.md. - Fix NameError in dodo.py uniquify_file when no lines were discarded. - Gitignore the doit intermediate combined.tmp and codestrings.tmp files.
5 tasks
The check-mo-freshness workflow only installs .[localize], which doesn't pull setuptools_scm; the top-level import made 'doit properties/po/mo' crash with ModuleNotFoundError. setuptools_scm is only used by task_version, so import it lazily inside that task.
Two follow-on issues discovered when the workflow ran in a clean CI env: 1. prop2po.py imports click, which was not in the [localize] extra. Add click as a dep of [localize] so 'pip install .[localize]' is sufficient to run 'doit localize' from scratch. 2. task_properties invoked check_strings.py in build mode, which hard-fails on missing non-English keys. With the English fallback added in this branch, non-en gaps are handled at runtime, so blocking the build on them is counterproductive. Remove enforce_strings_present from the properties task; check-strings.yml already runs check_strings.py in dev mode on every PR as a separate check.
The committed .mo files were out of sync with their .properties sources: ~20 keys per locale were either orphan translations for removed code or missing translations for currently-used code. Regenerated via the check-mo-freshness workflow on Ubuntu and downloaded from the workflow artifact so the bytes match what CI produces.
.po files are gitignored across the repo, so the workflow's `git diff --exit-code -- '**/*.po'` was always a no-op and the `.po` path filter never triggered a run. Trim both. No behavior change; the .mo drift check is what has always been doing the actual work. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
If a locale catalog was missing a msgid, gettext surfaced the raw key
(e.g.
tabcmd.user.warning.local_auth_remapped) instead of the Englishtranslation. The fix is a one-liner in
localize.py-- chainenbehind the preferred locale in
gettext.translation(languages=[..., "en"]). Bundled with a CI workflow that catches.modrift on PRs sothis class of gap stops accumulating silently.
Behavior change
For users: untranslated msgids now resolve to English rather than
surfacing the raw key. No behavior change for anyone on a fully-covered
locale.
For contributors:
.github/workflows/check-mo-freshness.ymlonevery PR touching localization sources regenerates
.mo/.pofromcommitted sources and fails if the committed binaries drift. Failed
runs upload the regenerated
.mofiles as an artifact so contributorscan commit them back without re-running doit locally.
Also bundled:
NameErrorindodo.py'suniquify_filewhen no lines werediscarded (
uniques->unique_lines).combined.tmpandcodestrings.tmpdoit intermediates.combine_property_files->properties) inbin/i18n/README.md; removed a duplicateLocalization section from
contributing.md.Test plan
pytest tests/commands/test_localize.py -v— 8 passed, includingtwo new tests: fr load with en-only msgid returns en; fr load with
translated msgid still returns fr
check-mo-freshnessandwill (deliberately) fail: committed
.mofiles have drifted. Willdownload regenerated files from workflow artifact and commit back.
Related followup: #441 (stop committing generated
.mo/.poat all).🤖 Generated with Claude Code