os_helper.gettext_utils module

Message Catalogues (gettext)

The companion to os_helper.i18n_utils, for packages whose human-facing text lives in the source rather than behind a semantic key. Where Locales answers what is the text for this key, Catalog answers what is the French for this English sentence — the question gettext was built for.

Why both exist

Locales keys on meaning (submit_button), which survives a rewrite of the copy. Catalog keys on the English source string, which does not: edit the English and the French no longer matches it, so gettext falls back to the English and the page half-reverts with nothing raised. That is the known cost of the gettext shape, and it is why Catalog.problems() reports a msgid used by the code but absent from a catalogue. A test that calls it turns the silent half-revert into a red build, which is the only thing that makes this shape safe to live with.

What it gives in return is what the suite wanted: one mechanism, the translator sees a real sentence instead of NAV_CONVERT, and an untranslated string degrades to readable English rather than to its own key.

No build dependency

GNU gettext’s msgfmt would compile the catalogues, and requiring it would mean a translated interface could only be built where somebody had installed the gettext tools. The .mo format is a header, two offset tables and a blob of strings, so build_mo() writes it here instead. The thing is small; the dependency is not.

Layout on disk

The canonical shape, which Catalog.from_package() expects:

<package>/locales/<lang>/LC_MESSAGES/<domain>.po     # source, committed
<package>/locales/<lang>/LC_MESSAGES/<domain>.mo      # compiled, committed

English is not a directory: it is the msgid, so it needs no catalogue and cannot drift from one.

Usage example

>>> # cat = Catalog.from_package(Path(__file__).parent / "locales", "video_helper")
>>> # cat.gettext("fr", "Convert a document")
>>> # 'Convertir un document'

Author

[Warith HARCHAOUI](https://linkedin.com/in/warith-harchaoui/)

class os_helper.gettext_utils.Catalog(locales_dir, domain, default_language)[source]

Bases: object

Compiled message catalogues for one package, read through gettext.

Build one with from_package() rather than calling the constructor, so the directory layout is resolved once and in one place.

Parameters:

locales_dir (Path)
domain (str)
default_language (str)

locales_dir

The locales/ root. Used in error messages, so a failure names the directory to fix.

Type:: pathlib.Path

domain

The gettext domain, and therefore each .mo file’s stem.

Type:: str

default_language

The language served when the caller asks for one that has no catalogue. The suite’s GUIs default to French; English needs no catalogue because it is the msgid.

Type:: str

default_language: str

domain: str

classmethod from_package(locales_dir, domain, *, default_language='fr')[source]

Resolve the catalogue directory for a package.

Parameters:

locales_dir (str or pathlib.Path) – Conventionally <package>/locales, i.e. inside the importable package directory. Outside it, the catalogues do not travel in the wheel and every string raises at run time on an installed copy.
domain (str) – The gettext domain; by convention the package’s import name.
default_language (str, keyword-only) – Language for an unknown or absent request.

Returns:

A catalogue handle. Nothing is read from disk until a lookup.

Return type:

Catalog

gettext(language, message)[source]

Translate one English source string.

Parameters:

language (str or None) – Requested language; resolved through resolve() first, so an unknown code never reaches the catalogues.
message (str) – The English source string, i.e. the msgid.

Returns:

The translation, or message itself when the catalogue has no entry for it.

Return type:

str

property languages: tuple[str, ...]

Every language this package can render, sorted.

The source language is always present — it is the msgid — followed by each <lang>/LC_MESSAGES/<domain>.mo actually on disk. A .po with no compiled .mo is deliberately not listed: it cannot be served, and claiming it would offer a language that renders English.

locales_dir: Path

negotiate(header)[source]

Best language from an HTTP Accept-Language header, or None.

Honours q weights and falls back to a base language, so a browser asking for fr-CA;q=0.9, en;q=0.8 gets the fr catalogue.

Parsed here rather than with a header library: this is one header, the grammar is a list of tags with optional weights, and the suite does not take a dependency for forty lines of standard parsing.

Parameters:: header (str or None) – The raw header value.
Returns:: The language to serve, or None when the best match is the source language or when nothing offered has a catalogue — both of which mean “leave the source text alone”. Returning None rather than "en" lets a caller distinguish “the visitor asked for English” from “the visitor asked for nothing we have”.
Return type:: str or None

Notes

The source language counts as offered even though it has no catalogue, and that is not a technicality: without it, a visitor who asked for English first and French second would be served French, because the language they actually wanted was the one the loop could not see.

ngettext(language, singular, plural, n)[source]

Translate a message whose wording depends on a count.

Parameters:

language (str or None) – Requested language, resolved through resolve() first.
singular (str) – The two English source forms. The pair is the msgid: gettext stores it NUL-joined, which is what build_mo() writes.
plural (str) – The two English source forms. The pair is the msgid: gettext stores it NUL-joined, which is what build_mo() writes.
n (int) – The count that picks the form. Which form that is comes from the catalogue’s own plural rule, not from n == 1 — the point of asking gettext rather than branching at the call site.

Returns:

The translated form, or the matching English source when the catalogue has no entry.

Return type:

str

pgettext(language, context, message)[source]

Translate one source string in a given context.

The answer to a word that is one word in English and two in French: pgettext("month name", "May") and pgettext("verb", "May") are separate entries, because a translator cannot disambiguate them from the msgid alone.

Parameters:

language (str or None) – Requested language, resolved through resolve() first.
context (str) – The disambiguating label, as written in the catalogue’s msgctxt.
message (str) – The English source string.

Returns:

The translation, or message itself when this context has no entry — never the contextmessage key, which would put a control character in front of a reader.

Return type:

str

problems(*, used=None)[source]

Everything wrong with this package’s catalogues, as sentences.

Checks, in order:

a .po with no compiled .mo, or a .mo that no longer matches its .po — a translation edited and not recompiled;
an entry whose translation is empty, which would quietly serve English from a language that claims to be translated;
when used is given, a string the code asks for that no catalogue carries, and a catalogue entry nothing asks for.

That third check is the one that matters most under this shape. Because the msgid is the English copy, rewording a sentence in the template orphans its translation, gettext falls back to English, and the page half-reverts with nothing raised. Passing the set of strings the code actually uses turns that into a failure a test can see.

Parameters:: used (set of str, optional) – Every msgid the code looks up. When omitted, the two code-versus-catalogue checks are skipped.
Returns:: One sentence per fault, empty when the catalogues are sound.
Return type:: list of str

resolve(language)[source]

The language that will actually be served for a request.

Parameters:: language (str or None) – The requested code, typically straight off a ?lang= parameter.
Returns:: language when it has a catalogue (or is the source language), its base language when only that has one (fr-CA → fr), otherwise default_language.
Return type:: str

translator_for(language)[source]

The raw translator, for a caller that wants to bind its own _.

Parameters:: language (str or None) – Requested language, resolved through resolve().
Returns:: The translator whose gettext method does the lookup.
Return type:: gettext.NullTranslations

os_helper.gettext_utils.build_mo(entries)[source]

Serialise {msgid: msgstr} into the binary gettext reads.

Layout, in order: a 28-byte header, the originals’ (length, offset) table, the translations’ table, then the two blobs. Entries must be sorted by msgid — gettext binary-searches them. A plural entry arrives here already NUL-joined on both sides, which is how the format carries it.

Parameters:: entries (dict of str to str) – The catalogue, typically from parse_po().
Returns:: A complete .mo file.
Return type:: bytes

os_helper.gettext_utils.compile_catalogs(locales_dir, *, check=False)[source]

Compile every .po under locales_dir to the .mo beside it.

Parameters:

locales_dir (str or pathlib.Path) – The locales/ root holding <lang>/LC_MESSAGES/<domain>.po.
check (bool, keyword-only) – Recompile in memory and compare instead of writing. This is what CI wants: a translation edited without recompiling cannot then reach a release looking live.

Returns:

With check=False, the .mo files written. With check=True, the .mo files that are stale or missing — empty means in sync.

Return type:

list of pathlib.Path

os_helper.gettext_utils.extract_call_sites(*paths)[source]

Every msgid a Python source asks for, read from the syntax tree.

The companion to extract_sources(), which scans template markup. Some members keep their human-facing strings in _() calls spread across modules rather than in one template, and for those the template scanner finds nothing.

Why this matters beyond convenience: Catalog.problems() can only report a catalogue entry nothing uses — the orphan left behind when a source string is reworded — if it is handed the complete set of strings the code asks for. An incomplete set turns that half of the check into noise. On md2star, a first attempt that collected only bare _("…") literals reported sixty-nine dead entries of which none was dead, and the check had to be left unasserted.

The three shapes are collected in the form the catalogue stores them, so the two sides compare directly:

_("text") and gettext("text") yield "text".
ngettext("one", "many", n) yields "one\many" — the plural separator gettext itself uses, so a plural entry is not reported as an orphan just because the singular was collected alone.
pgettext("context", "text") yields "contexttext", likewise.

Only literal arguments are collected, because only those are knowable without running the code. A call like _(message) where message is a variable is invisible here, which is why this function makes the code-uses-it-but-the-catalogue-does-not direction sound and the orphan direction only as sound as the caller’s own coverage. Say which one you are asserting.

Parameters:: *paths (str or Path) – Files or directories. A directory is walked for *.py, skipping .venv, build and __pycache__ — a vendored dependency’s strings are not this package’s.
Returns:: The msgids, keyed as the catalogue keys them.
Return type:: set of str

Examples

>>> import pathlib, tempfile
>>> folder = pathlib.Path(tempfile.mkdtemp())
>>> code = [
...     'from .i18n import ngettext as _n',
...     '_("Save")',
...     '_n("%d file", "%d files", n)',
... ]
>>> _ = (folder / 'm.py').write_text(chr(10).join(code))
>>> found = extract_call_sites(folder)
>>> sorted(m.replace(chr(0), ' | ') for m in found)
['%d file | %d files', 'Save']

os_helper.gettext_utils.extract_sources(*texts)[source]

Every English source string marked for translation in texts.

The extraction step a gettext workflow normally gets from pybabel, scoped to the one marker this suite uses. It exists so that a member’s test can hand Catalog.problems() the strings the code actually asks for, rather than a hand-maintained list that drifts from the markup — which is the only way the reworded-source fault gets caught.

Before this, eight members each kept their own copy of the same four-line regex, which is exactly the duplication a shared loader is for.

Parameters:: *texts (str) – Anything carrying markers — typically a module’s HTML template. Pass several and the results are unioned.
Returns:: The msgids found. Empty when nothing is marked, which a caller should treat as suspicious rather than as “nothing to translate”.
Return type:: set of str

Examples

>>> sorted(extract_sources('<h1>{{_("Run")}}</h1><p>{{_("Done.")}}</p>'))
['Done.', 'Run']

os_helper.gettext_utils.language_candidates(tag)[source]

A language tag plus its base, so fr-CA can find the fr catalogue.

Normalised the way gettext names its directories (fr_CA, not fr-CA) and stripped of any .UTF-8 / @euro suffix a POSIX locale string carries. One normalisation here means an HTTP header and an environment variable can both be passed in.

Parameters:: tag (str or None) – A BCP 47 tag (fr-CA), a POSIX locale (fr_CA.UTF-8), or None.
Returns:: Most specific first, then the bare language: a Québécois browser gets the French catalogue rather than nothing when fr_CA does not exist. Empty when tag is empty.
Return type:: list of str

os_helper.gettext_utils.parse_po(path)[source]

Read a .po into the {msgid: msgstr} mapping a .mo stores.

Plural entries are kept: gettext stores them by NUL-joining the two msgid forms into one key and the msgstr[n] forms into one value, which is exactly what build_mo() then writes. Contexts (msgctxt) are kept the same way, joined to their message by — the format has no separate field for them, which is why Catalog.pgettext() needs no change to the layout.

An entry with an empty translation is dropped, which is what makes an unfinished catalogue fall back to its English source rather than serve emptiness. Use untranslated() to see those instead of losing them.

Parameters:: path (str or pathlib.Path) – The .po file to read.
Returns:: Every translated entry, ready for build_mo().
Return type:: dict of str to str

os_helper.gettext_utils.untranslated(path)[source]

Every msgid in a .po whose translation is empty.

parse_po() drops these on purpose, so that an unfinished catalogue degrades to English. This reports them, which is what a test needs in order to fail on a half-finished translation instead of shipping one.

Parameters:: path (str or pathlib.Path) – The .po file to inspect.
Returns:: The untranslated msgids, in file order. The header entry (msgid "") is not included — it legitimately carries metadata, not a translation.
Return type:: list of str

os_helper.gettext_utils.write_po(path, entries, *, language, project)[source]

Write a .po catalogue, sorted by msgid so diffs stay readable.

The header carries the charset, because a catalogue without one is read as ASCII by some tools and French is not ASCII.

Parameters:

path (str or pathlib.Path) – Destination .po file; parent directories are created.
entries (dict of str to str) – {english_source: translation}. An empty translation is written as an empty msgstr, which marks the entry as still to do rather than dropping it.
language (str) – The catalogue’s language code, recorded in the header.
project (str) – The package the catalogue belongs to, recorded in the header.

Returns:

The file written.

Return type:

pathlib.Path