Skip to content

Coerce ClassLabel.names to plain str to fix invalid YAML in README.md - #8454

Open
RudrenduPaul wants to merge 1 commit into
huggingface:mainfrom
RudrenduPaul:fix-6919-classlabel-names-yaml
Open

Coerce ClassLabel.names to plain str to fix invalid YAML in README.md#8454
RudrenduPaul wants to merge 1 commit into
huggingface:mainfrom
RudrenduPaul:fix-6919-classlabel-names-yaml

Conversation

@RudrenduPaul

Copy link
Copy Markdown

Coerce ClassLabel.names to plain str to fix invalid YAML in README.md

Fixes #6919

Problem

push_to_hub can fail with:

ValueError: Invalid metadata in README.md.
- Invalid YAML in README.md: unknown tag !<tag:yaml.org,2002:python/tuple> (50:11)

This happens whenever a ClassLabel's names list contains elements that
are not exactly str — most commonly numpy.str_, which shows up whenever
names is built from numpy.unique(...) or read out of a pandas column
(a very common pattern for NER/token-classification labels, exactly as
described in the issue).

Root cause

ClassLabel.__post_init__ already normalizes the internal _int2str
mapping with str(name), but it left the public names list untouched.
When the dataset card YAML metadata is generated
(Features._to_yaml_list -> yaml.dump), PyYAML's default Dumper
selects a representer by exact type match, not isinstance. Since
numpy.str_ is a subclass of str but not str itself, it falls
through to the generic object representer, which serializes it with
unsafe !!python/object/apply:numpy... tags (including base64-encoded
binary state). The resulting README.md is technically written, but it
can no longer be parsed back with the strict yaml.safe_load used to
validate dataset card metadata, hence the "unknown tag" error.

Fix

Coerce every element of ClassLabel.names to a plain str in
__post_init__, mirroring what already happens for _int2str. This
guarantees only native Python strings ever reach the YAML dumper,
regardless of where names originally came from.

Testing

  • Added a regression test
    (test_class_label_names_are_coerced_to_str in
    tests/features/test_features.py) that builds a ClassLabel from
    numpy.str_ values, dumps its YAML representation, and asserts it
    round-trips through yaml.safe_load — this reproduces the exact
    failure from the issue and fails without the fix.
  • Ran the full tests/features/test_features.py, tests/test_info.py,
    and tests/test_metadata_util.py suites locally: 226 passed, 3
    skipped, no regressions.
  • Verified with a minimal repro matching the issue's steps (labels
    derived via numpy.unique, cast to
    Sequence(ClassLabel(names=list(labels))), then dumping the resulting
    DatasetInfo to YAML) — the dump now round-trips through
    yaml.safe_load cleanly instead of raising ConstructorError.

Scope

Only src/datasets/features/features.py (the fix) and
tests/features/test_features.py (the regression test) are touched. No
unrelated code or docs were changed.


Note: this PR was prepared with AI assistance (Claude Code) under my
direction — I reviewed the root-cause analysis, the fix, and the test
before submitting.

numpy.str_ (and similar numpy-backed) entries slip into ClassLabel.names
when built from numpy.unique(...) or a pandas column. PyYAML's default
Dumper only matches representers by exact type, so these fall back to
unsafe !!python/object/apply tags when the dataset card metadata is
dumped, producing a README.md that fails to parse back with
yaml.safe_load.

Fixes huggingface#6919
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Invalid YAML in README.md: unknown tag !<tag:yaml.org,2002:python/tuple>

1 participant