Skip to content

fix: Fix warmup phase batch size out of range. #4986

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Jun 9, 2025
Merged

fix: Fix warmup phase batch size out of range. #4986

merged 3 commits into from
Jun 9, 2025

Conversation

@hyukn
Copy link
Collaborator

@hyukn hyukn commented Jun 6, 2025

This is due to the number of autotuning warmup requests exceeding the limit of max_batch_size. This PR will limit the batch size according to multiple size constraints.

@hyukn hyukn requested review from QiJune and yiqingy0 June 6, 2025 08:44
@hyukn hyukn requested a review from a team as a code owner June 6, 2025 08:44
@hyukn
Copy link
Collaborator Author

hyukn commented Jun 6, 2025

/bot run

@tensorrt-cicd
Copy link
Collaborator

tensorrt-cicd commented Jun 6, 2025

PR_Github #7885 [ run ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

tensorrt-cicd commented Jun 6, 2025

PR_Github #7885 [ run ] completed with state FAILURE
/LLM/main/L0_MergeRequest_PR pipeline #5696 completed with status: 'FAILURE'

@hyukn
Copy link
Collaborator Author

hyukn commented Jun 7, 2025

/bot run

@tensorrt-cicd
Copy link
Collaborator

tensorrt-cicd commented Jun 7, 2025

PR_Github #7943 [ run ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

tensorrt-cicd commented Jun 7, 2025

PR_Github #7943 [ run ] completed with state FAILURE
/LLM/main/L0_MergeRequest_PR pipeline #5746 completed with status: 'FAILURE'

@hyukn
Copy link
Collaborator Author

hyukn commented Jun 7, 2025

/bot run

@tensorrt-cicd
Copy link
Collaborator

tensorrt-cicd commented Jun 7, 2025

PR_Github #7953 [ run ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

tensorrt-cicd commented Jun 7, 2025

PR_Github #7953 [ run ] completed with state FAILURE
/LLM/main/L0_MergeRequest_PR pipeline #5755 completed with status: 'FAILURE'

@hyukn hyukn force-pushed the fix/fix_autotune_warmup branch from 0d78cca to ebf229e Compare June 7, 2025 05:10
@hyukn
Copy link
Collaborator Author

hyukn commented Jun 7, 2025

/bot run

@tensorrt-cicd
Copy link
Collaborator

tensorrt-cicd commented Jun 7, 2025

PR_Github #7955 [ run ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

tensorrt-cicd commented Jun 7, 2025

PR_Github #7955 [ run ] completed with state FAILURE
/LLM/main/L0_MergeRequest_PR pipeline #5757 completed with status: 'FAILURE'

@hyukn
Copy link
Collaborator Author

hyukn commented Jun 7, 2025

/bot run

@tensorrt-cicd
Copy link
Collaborator

tensorrt-cicd commented Jun 7, 2025

PR_Github #7958 [ run ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

tensorrt-cicd commented Jun 7, 2025

PR_Github #7958 [ run ] completed with state FAILURE
/LLM/main/L0_MergeRequest_PR pipeline #5760 completed with status: 'FAILURE'

@hyukn hyukn force-pushed the fix/fix_autotune_warmup branch from ebf229e to bd82516 Compare June 8, 2025 11:43
@hyukn
Copy link
Collaborator Author

hyukn commented Jun 8, 2025

/bot run

1 similar comment
@hyukn
Copy link
Collaborator Author

hyukn commented Jun 8, 2025

/bot run

@tensorrt-cicd
Copy link
Collaborator

tensorrt-cicd commented Jun 8, 2025

PR_Github #8039 [ run ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

tensorrt-cicd commented Jun 8, 2025

PR_Github #8039 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #5826 completed with status: 'FAILURE'

@QiJune
Copy link
Collaborator

QiJune commented Jun 9, 2025

/bot run

QiJune
QiJune approved these changes Jun 9, 2025
View reviewed changes
Copy link
Collaborator

@QiJune QiJune left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@tensorrt-cicd
Copy link
Collaborator

tensorrt-cicd commented Jun 9, 2025

PR_Github #8056 [ run ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

tensorrt-cicd commented Jun 9, 2025

PR_Github #8056 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #5840 completed with status: 'SUCCESS'

@hyukn hyukn merged commit 137fe35 into NVIDIA:main Jun 9, 2025
3 checks passed
@hyukn hyukn deleted the fix/fix_autotune_warmup branch October 31, 2025 09:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@QiJune QiJune QiJune approved these changes

@yiqingy0 yiqingy0 Awaiting requested review from yiqingy0

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@hyukn @tensorrt-cicd @QiJune