-
Notifications
You must be signed in to change notification settings - Fork 10
Description
The logic of fetching models and model json files in transformers.js is as follows:
For the scenarios of env.allowLocalModels=true and env.allowRemoteModels=true (default), transformers.js will first try to get the model resources (including the model and model json file) from the local model path, and then try to get it from HuggingFace.
Current implementation Hosting mode:
env.allowLocalModels=true;
env.allowRemoteModels=false;Remote mode:
env.allowLocalModels=false;
env.allowRemoteModels=true;We cloud unify current 2 modes, just configure env.allowLocalModels=true, and then fetch models from local model path first, and then fetch it from HF.
Another major reason is because of the models jsons files(configs.json, tokenizer.json). Usually, the path to fetch these files follows the model files. If the configuration is to fetch the model files from HF, then these configuration files will also be obtained from HF. Because we have added these files into the repo, we hope to load the model json locally.
Pros
- The model json files will be used
- Solved a tricky case of loading a model from the local system in
remotemode when HF is not accessible, the model json file could not be obtained, so even if the user had loaded theonnxmodel, the examples still did not work. - Code simplification
- No need to maintain the
USE_REMOTE_MODELSparameter inconfig.js(this file is also tracked by Git, so it may change for every time build production)
Cons
- Errors that fail to request local model paths will be reported in
Devtools -> Networks
Development work
For the examples integrated from transformer.js, there are no additional code modifications. For the stable transmission turbo -webgpu example, we should refactor the model fetching logic.
