Install large dependency from single local copy

[arainboldt]

Hi!

I'm often creating repos that depend on large libraries like Torch. Re-installing fresh copies of these in every env is not only time-consuming but memory-consuming. Is it possible to have poetry reference a local copy and symlink it?

Thanks!

Andrew

[Secrus]

Poetry already does that. We cache downloaded wheels.

[arainboldt]

ok, that's great to know. that solves the downloading time issue, does that mean that there is only one memory footprint for a given dependency for all venvs?

[abn]

Unfortunately no. While this might be a bit fragile (as locked versions might differ etc), you can make use of the virtualenvs.options.system-site-packages option at a per-project level. You can manually install the "large" dependency in your system site (via distro package manager or just pip install) then configure the option to true.

poetry config --local virtualenvs.options.system-site-packages true

This will mean Poetry will recognize the package as "already installed". But obviously, it is not a silver bullet.

https://python-poetry.org/docs/configuration/#virtualenvsoptionssystem-site-packages

Edit: You will have to recreate any existing virtual environments after modifying that.