Model Conversion Process

[Justinius]

Using Olive for model conversion - needs some clarification.

However, the other one is how to incorporate the converted model into an existing cache.

The guide tells how to set the cache directory to the output of the Olive process, how do we copy the files over to the existing cache?

e.g.
models stored in "foundry_cache"
olive outputs to "converted_models"

Do we just change olive to output to the cache directory, or is there a way to import.

[samuel100]

@Justinius you can copy or move files to the existing cache, using cp or mv. The cache location can be gleaned using foundry cache location

[Justinius]

so I can just take the llama-3.2 folder and just move it into the same cache, do I put it in the microsoft folder in the cache directory? Do I need to update any config files or anything so it shows up when doing foundry cache ls or the like?

[samuel100]

You can just move it into ./foundry/cache/models. No need to put it under Microsoft - when we pull models we put them under a publisher folder (this is to avoid duplicate naming from different publishers e.g. Hugging Face).

[Justinius]

This did seem to work, except it doesn't have an alias, just the model ID. So if there is a way to set alias, so those can be used appropriately, that would be good

[samuel100]

Currently, there is no mechanism to add your own alias (it just uses the folder structure). I'll tag this as an enhancement for adding custom models. I'll convert to a discussion.