Skip to content

Get rid of LazyModel #218

@juanmc2005

Description

@juanmc2005

Problem

LazyModel makes it rather complicated for someone to add their own model, especially when some changes need to be made to the input/output. The reason LazyModel exists is so that we can spawn new processes and share a PipelineConfig instance. This is not possible without lazy loading because copying weights between processes is expensive and could also fail.

Idea

To get rid of LazyModel without sacrificing multiprocessing we need to lighten the content of PipelineConfig, which currently needs to hold model objects. We could simply hold model names or paths and share that across processes. Then, each process will have to load the model given the path or name.

This is actually the same behavior that exists today, the difference lies in the fact that lazy loading will be left to the config/pipeline instead of the actual model.

Metadata

Metadata

Assignees

No one assigned

    Labels

    APIImprovements to the APIfeatureNew feature or request

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions