-
Notifications
You must be signed in to change notification settings - Fork 3.5k
Open
Labels
community-requestenhancementNew feature or requestNew feature or requestneeds-follow-upIssue needs follow-upIssue needs follow-up
Description
Is your feature request related to a problem? Please describe.
Enable VPP for vision language model training
Describe the solution you'd like
I can easily set vpp/pp config for vision language model training, especially vision module division, language module division in pp and vpp dimensions.
Additional context
I have checked a feature that related "--pipeline-model-parallel-layout", but not clear on vlm model. Is this supported?
--pipeline-model-parallel-layout "Et*3|(tt|)*29,m|L"
https://docs.nvidia.com/megatron-core/developer-guide/latest/user-guide/features/pipeline_parallel_layout.html
Metadata
Metadata
Assignees
Labels
community-requestenhancementNew feature or requestNew feature or requestneeds-follow-upIssue needs follow-upIssue needs follow-up