Skip to content

Allow discrete sequence lengths (enumerated input shapes) #10

Open
@pcuenca

Description

@pcuenca

When the Core ML model is converted using flexible shapes, it doesn't run on GPU or ANE. We are currently using a fixed sequence length, but a better solution (to explore) would be to:

  • Use EnumeratedShapes to allow several possible input lengths. This change can be tested by converting manually using coremltools, and then adapting the code in exporters.
  • Update the generation code to select the shortest sequence length from the enumerated set that is larger than the current sequence.

Metadata

Metadata

Assignees

No one assigned

    Labels

    modellingrelated to CoreML/Transformers

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions