Skip to content

Fix flaky SAM-HQ integration tests #43211

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
engmohamedsalah wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Fix flaky SAM-HQ integration tests #43211

engmohamedsalah wants to merge 1 commit into from
+8 −0

Conversation

@engmohamedsalah
Copy link
Contributor

@engmohamedsalah engmohamedsalah commented Jan 10, 2026

Summary

Fixes #42890

SAM-HQ integration tests were failing due to non-deterministic positional embeddings. The positional_embedding parameter in SamHQPositionalEmbedding is:

  1. Randomly initialized using torch.randn() (line 418 in modeling_sam_hq.py)
  2. Excluded from checkpoint loading via _keys_to_ignore_on_load_missing (line 1236)

This meant every test run generated different random values, causing flaky test failures.

Solution

Added set_seed(0) in the setUp() method of SamHQModelIntegrationTest class to ensure deterministic random initialization across all integration tests.

This approach:

  • ✅ KISS: Single line in setUp() method
  • ✅ DRY: One fix applies to all 14 integration test methods
  • ✅ Clean: Follows existing transformers testing patterns
  • ✅ Minimal: Only 8 lines added (including comments)

Changes

  • Added from transformers.trainer_utils import set_seed import
  • Added setUp() method with set_seed(0) to SamHQModelIntegrationTest class
  • Added clear comments explaining why seed is needed

Test Plan

  • Python syntax check passed
  • Integration tests will be run by CI

@engmohamedsalah
Fix flaky SAM-HQ integration tests
99e95c4 
SAM-HQ's positional_embedding is randomly initialized with torch.randn()
and excluded from checkpoint loading via _keys_to_ignore_on_load_missing.
This causes non-deterministic test results. Fixed by setting seed in
setUp() to ensure reproducible integration tests.

Fixes huggingface#42890
@github-actions
Copy link
Contributor

github-actions bot commented Jan 10, 2026

[For maintainers] Suggested jobs to run (before merge)

run-slow: sam_hq

@engmohamedsalah engmohamedsalah mentioned this pull request Jan 10, 2026
Open
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

tests/models/sam_hq/test_modeling_sam_hq.py::SamHQModelIntegrationTest may fail since a lot of cases are lack of set_seed()

1 participant

@engmohamedsalah