fix(pypi): update get_maintainers_of_package to avoid request blocking #1097
Conversation
oracle-contributor-agreement
bot
added
the
OCA Verified
Signed-off-by: Amine <[email protected]>
AmineRaouane
force-pushed
the
improve_pypi_registry
branch
from
7401155 to
7e41b96
Compare
| @@ -244,11 +244,10 @@ def get_package_page(self, package_name: str) -> str | None: | |||
| str | None | |||
| The package main page. | |||
| """ | |||
| url = os.path.join(self.registry_url, "project", package_name) | |||
There was a problem hiding this comment.
I'm not entirely sure why these changes resolve the issue? I tried printing url = os.path.join(self.registry_url, "project", package_name) and it gives the same output as the hardcoded result you've replaced it with. It also seems that response.content.decode("utf-8") and response.text seem to return the same results? Have you been able to avoid the request blocking with this? When I try it I'm still getting blocked.
There was a problem hiding this comment.
I can confirm that f"https://pypi.org/project/{package_name}/" works. It seems that the trailing / is essential. Without it, PyPI executes a JavaScript code that sends a POST request and then redirects to the URL with the trailing / appended, and Macaron is not able to follow the POST request.
| @@ -244,11 +244,10 @@ def get_package_page(self, package_name: str) -> str | None: | |||
| str | None | |||
| The package main page. | |||
| """ | |||
| url = os.path.join(self.registry_url, "project", package_name) | |||
There was a problem hiding this comment.
I can confirm that f"https://pypi.org/project/{package_name}/" works. It seems that the trailing / is essential. Without it, PyPI executes a JavaScript code that sends a POST request and then redirects to the URL with the trailing / appended, and Macaron is not able to follow the POST request.
| @@ -244,11 +244,10 @@ def get_package_page(self, package_name: str) -> str | None: | |||
| str | None | |||
| The package main page. | |||
| """ | |||
| url = os.path.join(self.registry_url, "project", package_name) | |||
| url = f"https://pypi.org/project/{package_name}/" | |||
There was a problem hiding this comment.
@AmineRaouane Please add a comment above this line to emphasize why / at the end is important.
| response = send_get_http_raw(url) | ||
| if response: | ||
| html_snippets = response.content.decode("utf-8") | ||
| return html_snippets | ||
| return response.text |
Summary
This PR improves the functionality of the pypi_registry.py by enhancing the reliability of requests made to the PyPI registry.
Description of changes
Related issues
Checklist
verifiedlabel should appear next to all of your commits on GitHub.