Skip to content

Add locales.MN, a Mongolian pack reading a Cyrillic name's father's name as the family (#1237) - #1365

Merged
derek73 merged 13 commits into
masterfrom
feat/issue-1237-mongolian-pack
Nov 13, 2026
Merged

derek73 merged 13 commits into
masterfrom
feat/issue-1237-mongolian-pack

Conversation

@derek73

@derek73 derek73 commented Nov 12, 2026 •

Copy link
Copy Markdown
Owner

Closes #1237. Refs #1253.

Adds locales.MN, a Mongolian pack. It reads a name written wholly in Cyrillic with the father's name first, so the father's name is the family, as #1253's first statement has it. A romanized name keeps the caller's order.

from nameparser import parser_for, locales
mongolian = parser_for(locales.MN)
sukhee = mongolian.parse("Батын Сүхээ")
sukhee.family, sukhee.given            # ('Батын', 'Сүхээ')
mongolian.parse("Sükhee Batyn").given  # 'Sükhee': a romanized name keeps the caller's order

1.4.0 through 3.0.1, and the default still, read Батын Сүхээ as given Батын, family Сүхээ.

Changed since the first push (decided in coordination, 2026-11-12)

The first version declared FAMILY_FIRST_GIVEN_LAST for every name. Its own gate found that Mongolians write their own romanized names mostly given-first (65 of 87 two-word ORCID credits), so the order is now keyed to Cyrillic.

What the pack is

  • A plain choice of mn, plus a script_orders entry for Script.CYRILLIC, set to FAMILY_FIRST_GIVEN_LAST.
    • The choice changes no reading, because mn.toml's readings are all on by default.
    • _INERT_CHOICES exempts MN from the two tests that expect a choice to change a word.
  • Given name last, not second, so a word written before the father's name (a clan name, Боржигин Батын Сүхээ) leaves the given name in place. The clan name itself is not read as one.
  • Not read:

What the Cyrillic order needed

  • Script.CYRILLIC joins the enum and _policy._SCRIPT_RANGES: the Cyrillic block, plus Supplement and Extended-A, -B and -C.
    • The readers that meant "the East Asian scripts" when they read the whole table now read _EAST_ASIAN_SCRIPTS:
      • rules/T3's interpunct flank and the two-family-names run count (_classified_char);
      • the segmenter's neighbour test;
      • the CJK corpus builder;
      • the case table's CJK guard.
    • DEFAULT_SCRIPT_ORDERS and _NO_INITIALS gain no entry.
    • _classify asks the Cyrillic matcher only of a word that opens with a Cyrillic letter, so a non-ASCII Latin word costs nothing extra.
    • _assign._effective_order returns the declared order at the first word whose script, outside the kana licence, has no script_orders entry. Such a script resolves alone to itself or, beside any other, to None, so the answer cannot change. Under the default parser a Cyrillic name therefore costs what it did on master: Иван Петров 233 frames and Иванов Иван Иванович 353 (242 and 370 before this return), and John Smith 126 on both. A guard in tests/v2/test_benchmark.py pins one effective_script call per name from that walk.
  • effective_script reads a Cyrillic word with a hyphen or an apostrophe inside it (Оюун-Эрдэнэ, Мар'яна) as Cyrillic. The table stays letters only.
  • PolicyPatch.script_orders composes by script, where it was a scalar override.
    • A patch's entry replaces the base's entry for that script, and the other entries stand. So the pack adds its entry without erasing a caller's own: Policy(script_orders=()) with MN keeps only the Cyrillic entry.
    • parser_for warns where two packs set one script to different orders.
    • This reaches any caller's own PolicyPatch, not only MN's, so the release log carries it as a Change. In 2.1.0 through 3.0.1, Policy().patched(PolicyPatch(script_orders={Script.HAN: GIVEN_FIRST})) kept the Han entry alone; now it keeps the Hangul and hiragana entries too.
    • An empty patch no longer clears the table. Clear it with Policy(script_orders=()) on the base parser.
    • A malformed patch still reaches Policy's curated messages, an unhashable key included.
    • Nothing shipped patched script_orders before, which is why _policy.py's comment had said to revisit this when a pack needed it.
  • A lone word stays rules/O5's. The Cyrillic entry orders a longer name and settles no lone word's field: EffectiveOrder.by_script is set for an East Asian script alone. So Сүхээ under MN reads given and reports given-or-family. A caller's own katakana entry still settles a lone word, as Han's does; a test pins both.
  • No [rules] table. The table's keys are scalar Policy fields, and a mapping keyed by Script would need a value type that RULE_KEYS and _locale._RULE_VALUES lack. The pack carries its patch in code, as ZH and JA carry segment_scripts.
  • The licence, worded so no one reads "Cyrillic means family-first":
    • rules/W4's When and Then now name "a script a chosen locale pack gives an order". Its Then bullet says the pack's order is the caller's choice, not the script's, and that O1 and O2 stand down under it.
    • decisions/W4 records the decision.
    • Root AGENTS.md's design-philosophy paragraph says a pack's script order is never a default, ending "Never read it as 'Cyrillic means family-first'".

Measured (2026-11-12, master 2b39f23)

Source Script default FAMILY_FIRST the pack
State Great Khural (mn_parliament, from the cache) Cyrillic 0 / 126 126 / 126 126 / 126
JRC-Names, six Mongolian people, classified two-word spellings Cyrillic 0 / 9 9 / 9 9 / 9
same Latin 3 / 45 42 / 45 3 / 45, left to the caller
ORCID MN, 150 Latin free-text credits, against the holder's family box Latin 73 25 73
  • CLDR's mn sample renderings (all Cyrillic, rendered in every order) read 19 of 24 by default and 15 of 24 under the pack.
  • With no pack chosen, no name moves:
    • snapshot.py reports no moves; the snapshot only gains rows for the new rules examples.
    • snapshot.py --local <each> --base <master checkout> reports "matches the base" for all 16 local corpora.
    • Two reviews compared 47 mixed-script and edge names under 10 configurations against master, and readings, ambiguities, initials() and warnings were identical.

Pack unions (asked for plainly)

The pack reads every wholly-Cyrillic name its way, Russian, Kazakh and Kyrgyz ones included.

Name MN+RU MN+TR_AZ
Иван Сидоров family Иван family Иван
Сидоров Иван Петрович given Петрович, middle Иван, no patronymic given Петрович
Нурсултан Абишевич Назарбаев family Нурсултан, patronymic Абишевич family Нурсултан
Бекмырза Чолпонбай уулу given уулу given уулу, patronymic Чолпонбай уулу

Proposed precedence, not built: a patronymic shape the active rule reads (an O1 ending, an O2 marker) keeps its name from a pack's script order. None of the Khural's 126 members carries either, so the pack would lose nothing there. It is recorded in decisions/W4's Open:.

Decisions for the maintainer

  1. No genitive rule.
  2. FAMILY_FIRST_GIVEN_LAST against plain FAMILY_FIRST. Given-last protects the clan-name shape. It costs a three-word Russian name written family first: Сидоров Иван Петрович reads given Петрович. Neither shape is measured.
  3. The union precedence above.
  4. Accepted costs:
    • Every wholly-Cyrillic name reads the pack's way, so the pack is for Mongolian data.
    • Сүхээ Б. reads family Сүхээ.
    • A Cyrillic letter with a combining stress accent, which has no precomposed form, leaves its word unclassified, so that name keeps the declared order.
    • Ноён (Mr.) is no listed title, and neither are Доктор and the honorific гуай: vocabulary candidates, not filed.
    • A Latin letter that looks Cyrillic inside a Cyrillic word (Cүхээ Батын, U+0043) leaves the name to the caller's order.
    • Matching. The father's name written as an initial, the usual short form (Батын Сүхээ / Б. Сүхээ), is the family under the pack. So the pair reads family-initial and conflicts. The default matcher calls it consistent, but only by reading the father's name as the given name. Accepting family-initial restores it; docs/languages.rst shows the accept as a doctest.
    • A pickle holding Script.CYRILLIC does not load in earlier releases.

Also in this PR

Checks

  • uv run pytest with the ja extra, on the head (1743756): 46,080 passed; -m serial 35; sphinx-build -b doctest 840. On the rework commit: without ja, 46,034 passed; on Python 3.15 in its own environment, 46,034 passed.
  • pytest -n0 -m serial: 35 passed.
  • mypy, ruff and tools/vocab/build.py --check: clean.
  • sphinx-build -b doctest and -W -b html: clean.

Reviews

Each review ran as a subagent in its own detached worktree, on the pushed head of its round, with a read-only master checkout beside it.

Round Review Found Done
1 (b4aee8e) design-docs-reviewer Enkhtaivan Erdene cited as written family-first, though its boxes say given-first. #1237's option 1 misquoted (it named family-first). The JRC "9 of 9" and "Latin and Greek" overstated what the recipe classifies. "Three in four" taken from the wrong population. The spaced-initial and population counts did not reproduce (8, not 6). The traditional-script example had lost its U+202F. All corrected in f0bfd08, with each recipe's filter stated.
1 (b4aee8e) code-reviewer Missed roster entry in docs/contributing.rst. The inert choice is not recorded in decisions/deviates-registry. "snapshot unchanged" is literally false (one row added). The given-last order's cost to Russian names is unweighed. Plain-space traditional script is untested. docs/policy.rst names only vi. All fixed except policy.rst, which is another PR's file. The Russian-name cost is now Decision 3 above.
1 (b4aee8e) comment-analyzer The glued initial is divided by script_segment (rules/T8), not tokenize. The U+202F example. contributing.rst. A wrong NFD test comment. mn.py claimed a "gate re-take" the entry did not record. "yet" in locales.rst. A misordered clause in languages.rst, with a romanized example that was not the Cyrillic one's transliteration. The strict-mode comment and the DEVIATES "so" were imprecise. All fixed.
1 (b4aee8e) pr-test-analyzer No critical gaps; mutations confirm each test bites. The strict choice was untested. Readings the entry states were unpinned. The non-interference gate is vacuous for an all-declaring pack, as it already was for VI. Added a strict test, a table of the entry's stated readings, and seven more default-equal comma and title rows. The vacuous gate is VI's shape and left as is.
round 2 (f0bfd08) code-reviewer One fresh error: the new Russian-name bullet blamed O1 for the empty patronymic view, but O1 stands down under either declared order; the cause is that R9's East Slavic rule reads only a middle word, and the given-last order puts the patronymic in given. Everything else re-verified (ORCID and JRC counts over the stated filters, the U+202F, T8's division, the snapshot row, the doctest and prose, the new tests' mutations). Fixed in a3506a1.
rework (c9f11d2) code-reviewer Found no default reading moved through Parser() or any shipped pack, but three problems. A caller's own PolicyPatch(script_orders=…) now composes differently, with no Change bullet. An unhashable malformed key lost Policy's curated message. The Cyrillic matcher cost a frame per non-ASCII Latin word. All fixed in 33efcf5.
rework type-design-analyzer The same malformed-key regression and missing Change line. The composition's metadata had no constant, and no test pinned it. The by_script docstrings contradicted the code. The katakana vs Cyrillic lone-word asymmetry was undecided. Stale autodoc docstrings. segment_scripts accepted CYRILLIC with no word on what it does there. Fixed: a _BY_SCRIPT constant with a test, rewritten docstrings, a test pinning the katakana and Cyrillic rows, and a segment_scripts note.
rework design-docs-reviewer The pack clause was filed under rules/W4's Except, but it covers names outside W4's When. The composition change was undocumented. decisions/deviates-registry still named mn.toml's [rules]. The snapshot gains three rows, not one. O1 says nothing of the pack. W4's When and Then now state it, with a Then bullet saying O1 and O2 stand down under it. The rest is fixed.
rework pr-test-analyzer Seven of fifteen conditions survived mutation. Two were reachable without the pack: the interpunct flank (Гал·ла Иванова would split) and the segmenter neighbour test (山田太郎 Иван would stop dividing). The per-script warning's filter was untested, and so were the joiner edges. An unreachable empty check. Stale comments: the Cyrillic range, EffectiveOrder on katakana, the joiners, build.py. Glued initials (Б.Б.) and stress marks escape the pack's order. Fixed in 7ac557a: a test for each survivor (each checked to fail without its condition), the neighbour test routed through _EAST_ASIAN_SCRIPTS as decisions/W4 says, the empty check dropped in 33efcf5, the comments corrected, and the two escapes recorded as accepted.
fix (33efcf5) code-reviewer One fresh error: the new segment_scripts docstring called CYRILLIC inert there, but a configured segmenter divided Cyrillic words, and the construction warning asked for one. Everything else held: 13 malformed-patch shapes give master's messages, the joiner edges, the frame counts (José García 177 as on master), the release-log claim against 2.1.0, 2.8.2 and 3.0.1, and W4's citation. Fixed in 3301938: a Cyrillic word is no segmentation site, and the warning leaves CYRILLIC out, with a test that fails without the change.
fix (7ac557a) code-reviewer Nothing at confidence 80 or above. The neighbour gate reads exactly as master's is not None on 46 tokens, Cyrillic aside; every new test fails under its mutation; the decisions sentence holds when run. Cosmetic: two stray blank lines, and effective_script's docstring missing the joiner path. Tidied in e95d3d0.
merged (e944bb3) independent review (coordinator) No defect: with no pack chosen, 1,227,135 grid rows read as master. Three follow-ups. The default parser walked every word of a Cyrillic name (+9 and +17 frames). The pack's matching cost for a father's initial was undocumented. The user docs did not say the patronymic rotations stand down beside mn. Also: no direct test that the default keys no Cyrillic order; a homoglyph escape and two honorifics to record. All done in 1743756: the early return with a frame guard, a matching doctest in languages.rst, a composition doctest in locales.rst, a default test, and the #1237 entry records the rest.

🤖 Generated with Claude Code

derek73 and others added 2 commits November 12, 2026 12:25
…ily (#1237)

A plain choice of mn with mn.toml's [rules] order, family first with the
given name last, so the father's name a Mongolian name writes first is
the family, as #1253's decision states. The genitive is not read into
the patronymic view, and the choice of mn changes no reading. The
default does not move.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hoice

From review: the glued initial is divided by script_segment (rules/T8),
not tokenize; the traditional-script example holds its U+202F; the JRC
and ORCID figures name the population each recipe classifies; the
Russian-name cost of the given-last order is recorded; contributing.rst
and decisions/deviates-registry name the inert choice.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@derek73 derek73 added this to the 3.1 milestone Nov 12, 2026
@derek73 derek73 added enhancement non-latin-script Names written in a script other than Latin (rules/W) locale-packs Locale packs and language choices (nameparser.locales) name-order Name order, family-first and two family names (rules/O) labels Nov 12, 2026
@derek73 derek73 self-assigned this Nov 12, 2026
From the round-2 review of f0bfd08: O1 stands down under either
declared order; the view is empty because R9's East Slavic rule reads a
middle word, and FAMILY_FIRST_GIVEN_LAST puts the patronymic in given.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
derek73 and others added 3 commits November 12, 2026 13:13
…he in suggest_measure

local_parliaments.DATASETS gains mn_parliament (126 members, each in
Cyrillic, father's name first), so #1237's pack is measured from the
cache. suggest_measure.py read each local corpus's raw.jsonl directly,
which skipped local_corpora.load's checks and kept running on a
parliaments cache fetched from fewer datasets; it now reads through
load, and a stale or edited corpus stops the run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…script that settles none (#1237)

Decided in coordination: the pack reads a name written wholly in
Cyrillic family first with the given name last, and leaves a romanized
name to the caller's order, since Mongolians mostly write their names
given-first in Latin letters (65 of 87 two-word ORCID credits).

Script.CYRILLIC joins the enum and the script table; the readers that
mean the East Asian scripts (rules/T3's interpunct flank, the
two-family-names run count, the segmenter's neighbour test) read
_EAST_ASIAN_SCRIPTS, so a parser without the pack reads every name as
before (the snapshot and the 16 local corpora match master). A Cyrillic
word with a hyphen or apostrophe inside it classifies too.
PolicyPatch.script_orders composes by script, so the pack adds its entry
without erasing a caller's own; parser_for warns where two packs set one
script differently. The entry orders a name and settles no lone word
(rules/O5). rules/W4 gains the pack's Except, decisions/W4 the entry,
and root AGENTS.md says Cyrillic does not mean family-first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n change, and spare Latin the Cyrillic frame

From review of the rework: apply_patch's by-script merge lets an
unhashable key reach Policy's curated message; the composition change
reaches a caller's own PolicyPatch, so the release log carries it as a
Change and the PolicyPatch, Policy.patched and parser_for docstrings say
it; _classify asks the Cyrillic matcher only of a word opening with a
Cyrillic letter. rules/W4 states the pack's order in When and Then;
by_script's docstrings say it is East Asian; decisions text corrected.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@derek73 derek73 changed the title Add locales.MN, a Mongolian pack reading the father's name as the family (#1237) Add locales.MN, a Mongolian pack reading a Cyrillic name's father's name as the family (#1237) Nov 12, 2026
derek73 and others added 6 commits November 12, 2026 14:06
…ing says

From the review of 33efcf5: listed in segment_scripts, CYRILLIC made a
Cyrillic word a site a configured segmenter divided, and the
construction warning asked for that segmenter. A Cyrillic word is now no
site, and the warning leaves CYRILLIC out. _effective_order's citation
quotes both clauses of rules/W4's Then.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eview found stale

From the rework's test review: rules/T3's flank and the two-family-names
count read the East Asian scripts (a Cyrillic punt volat and a Cyrillic
run are pinned), the segmenter's neighbour test reads
_EAST_ASIAN_SCRIPTS and a Cyrillic neighbour no longer blocks the
consult (pinned), the per-script warning credits a pack only with the
scripts it set (pinned), and the joiner edges are pinned. Comments on
the Cyrillic range, EffectiveOrder, the joiners and build.py's
EAST_ASIAN_RANGES corrected; glued initials and a stress mark recorded
as accepted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… blank lines

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e pack's matching and composition costs

_assign._effective_order now returns the declared order at the first
word whose script, outside the kana license, has no script_orders entry:
such a script resolves alone to itself or beside any other to None, so
the answer cannot change. The default parser no longer classifies every
word of a Cyrillic name (Иван Петров 242 -> 233 frames, Иванов Иван
Иванович 370 -> 353, master's counts); tests/v2/test_benchmark.py pins
one call a name with its negative control, and decisions/W4's frame line
says what holds. docs/languages.rst shows that a father's initial reads
family-initial under the pack and the accept that restores it;
docs/locales.rst says a script order takes a Cyrillic name before the
patronymic rotations, with Сидоров Иван Петрович under MN + RU. The
#1237 entry records the matching cost, a Latin homoglyph that leaves a
name to the caller's order, and Доктор and гуай as vocabulary
candidates. A test holds that the default keys no order to Cyrillic.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…golian-pack

Keeps both sides' release-log bullets, case rows and suggest_measure
tests; corpus_rules.jsonl regenerated with build_rules_corpus.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@derek73
derek73 merged commit d3795e0 into master Nov 13, 2026
14 checks passed
@derek73
derek73 deleted the feat/issue-1237-mongolian-pack branch November 13, 2026 01:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement locale-packs Locale packs and language choices (nameparser.locales) name-order Name order, family-first and two family names (rules/O) non-latin-script Names written in a script other than Latin (rules/W)

Projects

None yet

1 participant