Skip to content

Name the licences of the shipped data in the package metadata - #1370

Open
derek73 wants to merge 3 commits into
masterfrom
fix/licence-metadata-familiar-data
Open

derek73 wants to merge 3 commits into
masterfrom
fix/licence-metadata-familiar-data

Conversation

@derek73

@derek73 derek73 commented Nov 13, 2026 •

Copy link
Copy Markdown
Owner

Summary

The package's licence expression read LGPL-2.1-or-later alone, while the wheel ships data derived from five third-party datasets under other licences. Two of those licences ask their text or notice to travel with copies, and the tree held neither text.

  • Licence expression: LGPL-2.1-or-later AND Apache-2.0 AND GFDL-1.2-no-invariants-or-later AND CC-BY-4.0 AND OGDL-Taiwan-1.0 AND Unicode-3.0. packaging canonicalizes it unchanged, and the wheel and sdist carry it as License-Expression (Metadata-Version 2.4).

  • LICENSE-nam_dict (new): nam_dict's copyright and licence notices as the file states them, then the GFDL 1.2 text. It covers the 3.1 familiar forms and the suggester's common given names.

  • LICENSE-unicode (new): the Unicode License v3 notice. It covers locales/zh.py's simplified surname forms, which have shipped since 2.5.0: 19 of the block's 324 forms come only from Unihan's variant table. It also covers the 3.1 suggester's pinyin surnames. zh.py's docstring now credits Unihan.

  • README.rst's License section lists every source and where its terms are met. This is the PyPI page.

  • tests/v2/test_licensing.py holds the licence files, license-files, the expression, the README and the modules crediting the attribution-only sources (CC BY 4.0, OGDL) to one another. Checked against deliberate breakages: dropping LICENSE-unicode from license-files, CC-BY-4.0 from the expression, and zh.py's README credit each fail it.

  • decisions/shipped-data-licences (new key) records:

    • each source, its first release, what its licence asks and how that is met;
    • what is left out and why: CC0 sources, corpora used only for counts, and census lists with no stated terms;
    • an open question: SUFFIX_ACRONYMS' Wikipedia origin;
    • relicensing to LGPL-3.0, weighed and not taken.

    AGENTS.md's Extension Patterns now says what a new data source needs.

The code's licence is unchanged. Releases 2.5.0 through 3.0.1 shipped OGDL and Unihan data, and 3.0.0 onward CC BY data, behind the LGPL-only expression. Debian's debian/copyright will need a paragraph per source when 3.1 is packaged.

The entry also records the rule for future sources (FACTS WE MEASURE): data the project derives by measuring (its own counts, cutoffs and filters) is its own, with sources credited as a courtesy. A curated list copied whole, one compiler's judgements of which entries belong, keeps its licence, and if that licence is a burden it goes in an optional extra package instead of the core. Whether the attribution-only terms stay in the expression under that rule is recorded as Open; they stay for now.

Follow-ups, each its own PR:

  • replace Unihan with Wikidata's pinyin;
  • derive the suggester's common given names from ORCID instead of nam_dict;
  • drop nam_dict's familiar-form pairs before 3.1, keeping the ODU list's;
  • rebuild the non-English familiar forms from CC0 data (ORCID, GND), reviewed by hand.

Each removal shrinks the expression and the files again, and the test flags any of them left behind.

Review

  • design-docs-reviewer on the decisions entry, the AGENTS.md paragraph and the licence-file headers. It found:

    • Unihan's 2.5.0 shipping in zh.py;
    • the census lists and the Wikipedia import missing from the entry;
    • a test claim wider than the test;
    • a per-release overstatement;
    • the "rest of nameparser" sentences in the two licence files.

    All are fixed in this commit. The missing PR number is added in a follow-up commit.

Test plan

  • uv run pytest: 46,722 passed, 38 skipped, 3 xfailed
  • uv run ruff check, and uv run mypy on the new test
  • sphinx-build -b doctest docs, python -m doctest README.rst, and readme_renderer on README.rst
  • uv build: the wheel's dist-info/licenses/ and the sdist carry LICENSE, AUTHORS, LICENSE-nicknames, LICENSE-nam_dict and LICENSE-unicode

🤖 Generated with Claude Code

The licence expression read LGPL-2.1-or-later alone while the wheel
ships data under five other licences. It now reads
LGPL-2.1-or-later AND Apache-2.0 AND GFDL-1.2-no-invariants-or-later
AND CC-BY-4.0 AND OGDL-Taiwan-1.0 AND Unicode-3.0.

- LICENSE-nam_dict: nam_dict's notices and the GFDL 1.2 text, which
  its terms ask to travel with the familiar forms and the suggester's
  common given names and which no file in the tree held.
- LICENSE-unicode: the Unicode License v3 notice for the Unihan data,
  in locales/zh.py's simplified surname forms since 2.5.0 and in the
  suggester's pinyin surnames; zh.py's docstring now credits it.
- README.rst's License section lists every source.
- tests/v2/test_licensing.py holds the licence files, license-files,
  the expression, the README and the crediting modules together.
- decisions/shipped-data-licences records the sources, what each
  licence asks and how it is met; AGENTS.md says what a new source
  needs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@derek73 derek73 added docs Documentation fixes and updates tests vocabulary Fixed by a word-list entry: per-word vocabulary and language packs labels Nov 13, 2026
@derek73 derek73 self-assigned this Nov 13, 2026
derek73 and others added 2 commits November 12, 2026 20:48
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Data the project derives by measuring is its own, its sources credited
as a courtesy; a curated list copied whole keeps its licence. Under
that rule nam_dict's familiar-form pairs are dropped before 3.1 and
rebuilt from CC0 data reviewed by hand.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation fixes and updates tests vocabulary Fixed by a word-list entry: per-word vocabulary and language packs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant