Poetry can not resolve dependencies when priority = "explicit" is set for a private package registry

[simon-liebehenschel]

Versions

• poetry 1.5.0

Working 'pyproject.toml`

[tool.poetry.dependencies]
foo = {version = "*", source = "gitlab-private-package-registry"}
bar = {version = "*", source = "gitlab-private-package-registry"}

[[tool.poetry.source]]
name = "gitlab-private-package-registry"
url = "https://gitlab.com/api/v4/projects/<MASKED>/packages/pypi/simple/"
default = false
secondary = true

Broken pyproject.toml

Let's add priority = "explicit":

[tool.poetry.dependencies]
foo = {version = "*", source = "gitlab-private-package-registry"}
bar = {version = "*", source = "gitlab-private-package-registry"}

[[tool.poetry.source]]
name = "gitlab-private-package-registry"
url = "https://gitlab.com/api/v4/projects/<MASKED>/packages/pypi/simple/"
priority = "explicit"

Error output:

  Because no versions of foo match !=1.0.0
   and foo (1.0.0) depends on bar (*), every version of foo requires bar (*).
  So, because no versions of bar match *
   and compute depends on foo (*), version solving failed.

[dimbleby]

you didn't read the documentation about package sources that tells you this is working exactly as expected

[simon-liebehenschel]

I am reading repositories documentation carefully. The page says:

By default, Poetry discovers and installs packages from PyPI. But, you want to install a dependency to your project for a simple API repository? Let’s do it.
First, configure the package source as a supplemental (or explicit) package source to your project.

Done on my side. I have defined a private source, and I added priority = "explicit" to it:

[[tool.poetry.source]]
name = "gitlab-private-package-registry"
url = "https://gitlab.com/api/v4/projects/<MASKED>/packages/pypi/simple/"
priority = "explicit"

Explicit sources are considered only for packages that explicitly indicate their source.

Done on my side. I have defined

foo = {version = "*", source = "gitlab-private-package-registry"}

---

Either I still do not see something, or the migration from default = false secondary = true to priority = "explicit" is not described well.

Note that I am not trying to setup Poetry for the first time. I used to use it since 1.1. I just wonder how this new feature in the 1.5 version should work.

[dimbleby]

Hmm, maybe this is more interesting. Even though you have configured bar with an explicit source it looks as though poetry doesn't respect that when it is looking for bar as a dependency of foo. is that right?

Sounds like a bug if so, meanwhile a supplemental source should work

[simon-liebehenschel]

Even though you have configured bar with an explicit source it looks as though poetry doesn't respect that when it is looking for bar as a dependency of foo. is that right?

Exactly.

Sounds like a bug if so, meanwhile a supplemental source should work

Yes, defining "supplemental"

[[tool.poetry.source]]
name = "gitlab-private-package-registry"
url = "https://gitlab.com/api/v4/projects/<MASKED>/packages/pypi/simple/"
priority = "supplemental"

works as a temporary workaround for the not working "explicit".

---

By the way, Poetry still looks at PyPI even when the source is specified explicitly:

Source (PyPI): 0 packages found for <MY-PACKAGE-NAME> *

[dimbleby]

that is how supplemental sources are supposed to work

[simon-liebehenschel]

It you comment my this statement

By the way, Poetry still looks at PyPI even when the source is specified explicitly:
Source (PyPI): 0 packages found for *

then I want to clarify that it happened with

[tool.poetry.dependencies]
foo = {version = "*", source = "gitlab-private-package-registry"}

[[tool.poetry.source]]
name = "gitlab-private-package-registry"
url = "https://gitlab.com/api/v4/projects/<MASKED>/packages/pypi/simple/"
priority = "explicit"

[b-kamphorst]

Both of these issues are indeed not the intended behavior. I tried to write a test for the original issue, but I can't pinpoint the origin of the issue. I thought it would be in the solver, but the test passes: 2f7692b.

Hypothesis: the code that is responsible for finding the sources for the packages in the resulting Transaction is unaware of the explicit sources, and just queries PyPI -- rightfully ignoring the explicit source.

[simon-liebehenschel]

@b-kamphorst Please let me know if I can help with testing some beta changes, or if I need to provide full logs.

[b-kamphorst]

I'd gladly take on that offer once the PR is ready. For now, I need to understand the inner workings of the package retrieval (after solving the puzzle, in the installation process). I'll share my observations thus far in #7992.

[dimbleby]

The error message is surely coming from locking, not from installation. Someone with a repro could confirm by running poetry lock.

If you cannot (easily) reproduce this I would definitely keep in mind the possibility that the reporter is simply mistaken: and ask for a repro to prove it one way or another.

[b-kamphorst]

Well that does seem much more efficient than my semi-random attempts for reproduction... AIGeneratedUsername can you provide such a repo? E.g. one where you disable PyPI and add an explicit repository with the PyPI URL (to lose any dependence to e.g. a company registry index)?

[simon-liebehenschel]

I have solved the problem.
The solution was to comment out all private packages except one, then run poetry lock. Then uncomment one more private package, run poetry lock. Repeat until all private packages will be uncommented again.

I believe that in the 1.5 there are some resolving issues in rare edge cases.

Since I have no idea about how to provide a reproducible sample with private Package Registries, and I have a workaround for the initial problem, consider this discussion as closed.

[b-kamphorst]

Glad you solved it! I have tested several approaches and I was also not able to replicate the issue. I'll consider this solved as well, thanks for reporting back :)

[AuroraLantean]

poetry add --lock new_dependency_name