Skip to content

Enable VPP for vision language model training #2866

@robotsp

Description

@robotsp

Is your feature request related to a problem? Please describe.
Enable VPP for vision language model training

Describe the solution you'd like
I can easily set vpp/pp config for vision language model training, especially vision module division, language module division in pp and vpp dimensions.

Additional context
I have checked a feature that related "--pipeline-model-parallel-layout", but not clear on vlm model. Is this supported?
--pipeline-model-parallel-layout "Et*3|(tt|)*29,m|L"
https://docs.nvidia.com/megatron-core/developer-guide/latest/user-guide/features/pipeline_parallel_layout.html

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions