Skip to content

Conversation

@McPatate
Copy link
Member

@McPatate McPatate commented Nov 7, 2025

I suspect some sort of error where we're not working with the correct up to date code of transformers in the benchmarks, which definitely is an issue.

EDIT: culprit seems to be:

working-directory: /transformers

and thinking of it indeed, it's not what we just checked out but transformers dir of the docler image

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@McPatate McPatate requested review from ArthurZucker, remi-or and ydshieh and removed request for remi-or November 7, 2025 17:48
Copy link
Collaborator

@ydshieh ydshieh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, nice catch, thank you!

For me, I would probably use something like

      - name: Update clone
        working-directory: /transformers
        env:
          commit_sha: ${{ github.sha }}
        run: |
          git fetch origin "$commit_sha" && git checkout "$commit_sha"

It's not a technical reason, but it's what I did for all workflow files where we use our CI docker images. But since you maintain the benchmark workflows, it's up to you to make the call :-)

@McPatate McPatate marked this pull request as ready for review November 12, 2025 15:39
@McPatate McPatate force-pushed the fix/ci_unexpected_keyword_arg_streaming branch from d84ce9a to afeb173 Compare November 12, 2025 15:40
@McPatate McPatate enabled auto-merge (squash) November 12, 2025 15:40
@McPatate
Copy link
Member Author

It's not a technical reason, but it's what I did for all workflow files where we use our CI docker images. But since you maintain the benchmark workflows, it's up to you to make the call :-)

I'd rather keep my workflow standard with the actions/checkout step! Will change if it is too confusing

@ArthurZucker ArthurZucker merged commit ba938fa into main Nov 12, 2025
20 of 24 checks passed
@ArthurZucker
Copy link
Collaborator

Ty all!

@ArthurZucker ArthurZucker deleted the fix/ci_unexpected_keyword_arg_streaming branch November 12, 2025 16:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants