Getting HTTP Error 403: Forbidden without urllib header when accessing Auth0

[yogevyuval]

When using the PyJWKClient, suddenly responses from jwk.json from auth0, but it works when visiting the URL in the browser. Further debugging shows that when including some user-agent in the headers of the underlying request solves the issue.
I'm not sure if this is related to the CloudFlare incident that just happens - does anyone else expireince this?

Do we want to provide a user agent to requests?

[jasonkoverantos]

We are seeing the same issue as well, which just started happening today. Has anything changed recently?

[0xflick]

We think that as part of outage mitigation, Cloudflare began blocking requests with the python urllib default user agent (anything that begins with Python-urllib). We mitigated by changing the user agent used by pyjwt

[david206]

Might be related to this auth0 issue:
https://status.auth0.com/incidents/hdwf387t3nx7

This appears to have particularly impacted customers using non-standard user agents as part of their traffic requests.

We got 403 from https://host.us.auth0.com/.well-known/jwks.json until we set the user agent.

[micahjsmith]

+1 to setting a user agent, seems like a reasonable practice

[david206]

Since the package is using urllib, the request already has a sensible default User-Agent (Python-urllib/...):
https://github.com/python/cpython/blob/main/Lib/urllib/request.py#L397

[micahjsmith]

Right - not a big deal, but I think it's typical for sdks to identify themselves, rather than the low-level library they are using. see https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/User-Agent#examples_2 No hard and fast rule here, could consider something like

User-Agent: PyJWT/X.Y.Z

or

User-Agent: Python-urllib/3.11 (PyJWT X.Y.Z)