[bug] Exception: 401: Remote Inference Unauthorized. - on docker setup #1296
Description
Describe the bug
I get this error : Exception: 401: Remote Inference Unauthorized. even though I have specified the token
To Reproduce
I am running a guardrails server as a docker image:
FROM <my python image>
ENV PYTHONUNBUFFERED=1
# Create app directory
WORKDIR /app
COPY requirements-guardrails.txt .
RUN pip install uv
RUN uv pip install --system --prerelease=allow -r requirements-guardrails.txt
# Set the directory for nltk data
ENV NLTK_DATA=/opt/nltk_data
# Download punkt data
RUN python -m nltk.downloader -d /opt/nltk_data punkt
RUN guardrails configure --disable-metrics --disable-remote-inferencing --token <my_token>
RUN guardrails hub install hub://guardrails/profanity_free --quiet
RUN guardrails hub install hub://guardrails/toxic_language --quiet
COPY config.py config.py
EXPOSE 8000
# Create startup script
CMD gunicorn --bind 0.0.0.0:8000 --timeout=90 --workers=1 \
-k uvicorn.workers.UvicornWorker \
'guardrails_api.app:create_app(None, "config.py")'
Upon calling inference:
2025-07-16T14:59:59.328383167Z ERROR:guardrails-api:401: Remote Inference Unauthorized. Please run `guardrails configure`. You can find a new token at https://hub.guardrailsai.com/keys
2025-07-16T14:59:59.329324126Z Traceback (most recent call last):
2025-07-16T14:59:59.329888417Z File "/pyenv-2.6.4/versions/3.12.11/lib/python3.12/site-packages/guardrails_api/utils/handle_error.py", line 16, in wrapper
2025-07-16T14:59:59.329894042Z return await fn(*args, **kwargs)
2025-07-16T14:59:59.329895251Z ^^^^^^^^^^^^^^^^^^^^^^^^^
2025-07-16T14:59:59.329896126Z File "/pyenv-2.6.4/versions/3.12.11/lib/python3.12/site-packages/guardrails_api/api/guards.py", line 210, in validate
2025-07-16T14:59:59.329897251Z execution = guard.parse(
2025-07-16T14:59:59.329898084Z ^^^^^^^^^^^^
2025-07-16T14:59:59.329899001Z File "/pyenv-2.6.4/versions/3.12.11/lib/python3.12/site-packages/guardrails/hub_telemetry/hub_tracing.py", line 144, in wrapper
2025-07-16T14:59:59.329900042Z resp = fn(*args, **kwargs)
2025-07-16T14:59:59.329900834Z ^^^^^^^^^^^^^^^^^^^
2025-07-16T14:59:59.329901667Z File "/pyenv-2.6.4/versions/3.12.11/lib/python3.12/site-packages/guardrails/guard.py", line 973, in parse
2025-07-16T14:59:59.329902667Z return trace_guard_execution(
2025-07-16T14:59:59.329903501Z ^^^^^^^^^^^^^^^^^^^^^^
2025-07-16T14:59:59.329904334Z File "/pyenv-2.6.4/versions/3.12.11/lib/python3.12/site-packages/guardrails/telemetry/guard_tracing.py", line 206, in trace_guard_execution
2025-07-16T14:59:59.329905376Z raise e
2025-07-16T14:59:59.329906209Z File "/pyenv-2.6.4/versions/3.12.11/lib/python3.12/site-packages/guardrails/telemetry/guard_tracing.py", line 195, in trace_guard_execution
2025-07-16T14:59:59.329907209Z result = _execute_fn(*args, **kwargs)
2025-07-16T14:59:59.329908042Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-07-16T14:59:59.329908876Z File "/pyenv-2.6.4/versions/3.12.11/lib/python3.12/site-packages/guardrails/guard.py", line 799, in _execute
2025-07-16T14:59:59.329909834Z return guard_context.run(
2025-07-16T14:59:59.329910709Z ^^^^^^^^^^^^^^^^^^
2025-07-16T14:59:59.329911584Z File "/pyenv-2.6.4/versions/3.12.11/lib/python3.12/site-packages/guardrails/telemetry/common.py", line 100, in wrapped_func
2025-07-16T14:59:59.329912584Z return func(*args, **kwargs)
2025-07-16T14:59:59.329913459Z ^^^^^^^^^^^^^^^^^^^^^
2025-07-16T14:59:59.329914334Z File "/pyenv-2.6.4/versions/3.12.11/lib/python3.12/site-packages/guardrails/guard.py", line 778, in __exec
2025-07-16T14:59:59.329915209Z return self._exec(
2025-07-16T14:59:59.329915876Z ^^^^^^^^^^^
2025-07-16T14:59:59.329916584Z File "/pyenv-2.6.4/versions/3.12.11/lib/python3.12/site-packages/guardrails/guard.py", line 877, in _exec
2025-07-16T14:59:59.329917376Z call = runner(call_log=call_log, prompt_params=prompt_params)
2025-07-16T14:59:59.329918126Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-07-16T14:59:59.329926501Z File "/pyenv-2.6.4/versions/3.12.11/lib/python3.12/site-packages/guardrails/hub_telemetry/hub_tracing.py", line 144, in wrapper
2025-07-16T14:59:59.329928751Z resp = fn(*args, **kwargs)
2025-07-16T14:59:59.329929459Z ^^^^^^^^^^^^^^^^^^^
2025-07-16T14:59:59.329930084Z File "/pyenv-2.6.4/versions/3.12.11/lib/python3.12/site-packages/guardrails/run/runner.py", line 200, in __call__
2025-07-16T14:59:59.329931084Z raise e
2025-07-16T14:59:59.329931709Z File "/pyenv-2.6.4/versions/3.12.11/lib/python3.12/site-packages/guardrails/run/runner.py", line 170, in __call__
2025-07-16T14:59:59.329932501Z iteration = self.step(
2025-07-16T14:59:59.329933167Z ^^^^^^^^^^
2025-07-16T14:59:59.329933834Z File "/pyenv-2.6.4/versions/3.12.11/lib/python3.12/site-packages/guardrails/hub_telemetry/hub_tracing.py", line 144, in wrapper
2025-07-16T14:59:59.329934626Z resp = fn(*args, **kwargs)
2025-07-16T14:59:59.329935292Z ^^^^^^^^^^^^^^^^^^^
2025-07-16T14:59:59.329935917Z File "/pyenv-2.6.4/versions/3.12.11/lib/python3.12/site-packages/guardrails/telemetry/runner_tracing.py", line 104, in trace_step_wrapper
2025-07-16T14:59:59.329936751Z raise e
2025-07-16T14:59:59.329937542Z File "/pyenv-2.6.4/versions/3.12.11/lib/python3.12/site-packages/guardrails/telemetry/runner_tracing.py", line 96, in trace_step_wrapper
2025-07-16T14:59:59.329938376Z response = fn(*args, **kwargs)
2025-07-16T14:59:59.329939042Z ^^^^^^^^^^^^^^^^^^^
2025-07-16T14:59:59.329939709Z File "/pyenv-2.6.4/versions/3.12.11/lib/python3.12/site-packages/guardrails/run/runner.py", line 284, in step
2025-07-16T14:59:59.329940501Z raise e
2025-07-16T14:59:59.329941126Z File "/pyenv-2.6.4/versions/3.12.11/lib/python3.12/site-packages/guardrails/run/runner.py", line 269, in step
2025-07-16T14:59:59.329941876Z validated_output = self.validate(
2025-07-16T14:59:59.329942542Z ^^^^^^^^^^^^^^
2025-07-16T14:59:59.329943209Z File "/pyenv-2.6.4/versions/3.12.11/lib/python3.12/site-packages/guardrails/hub_telemetry/hub_tracing.py", line 144, in wrapper
2025-07-16T14:59:59.329944001Z resp = fn(*args, **kwargs)
2025-07-16T14:59:59.329944667Z ^^^^^^^^^^^^^^^^^^^
2025-07-16T14:59:59.329945334Z File "/pyenv-2.6.4/versions/3.12.11/lib/python3.12/site-packages/guardrails/run/runner.py", line 465, in validate
2025-07-16T14:59:59.329946167Z validated_output, metadata = validator_service.validate(
2025-07-16T14:59:59.329946876Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-07-16T14:59:59.330839709Z File "/pyenv-2.6.4/versions/3.12.11/lib/python3.12/site-packages/guardrails/validator_service/__init__.py", line 91, in validate
2025-07-16T14:59:59.330849334Z return validator_service.validate(
2025-07-16T14:59:59.330853542Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-07-16T14:59:59.330854501Z File "/pyenv-2.6.4/versions/3.12.11/lib/python3.12/site-packages/guardrails/validator_service/sequential_validator_service.py", line 460, in validate
2025-07-16T14:59:59.330855584Z value, metadata = self.run_validators(
2025-07-16T14:59:59.330856501Z ^^^^^^^^^^^^^^^^^^^^
2025-07-16T14:59:59.330857334Z File "/pyenv-2.6.4/versions/3.12.11/lib/python3.12/site-packages/guardrails/validator_service/sequential_validator_service.py", line 355, in run_validators
2025-07-16T14:59:59.330858376Z validator_logs = self.run_validator(
2025-07-16T14:59:59.330859251Z ^^^^^^^^^^^^^^^^^^^
2025-07-16T14:59:59.330860084Z File "/pyenv-2.6.4/versions/3.12.11/lib/python3.12/site-packages/guardrails/validator_service/sequential_validator_service.py", line 66, in run_validator
2025-07-16T14:59:59.330861209Z result = self.run_validator_sync(
2025-07-16T14:59:59.330862126Z ^^^^^^^^^^^^^^^^^^^^^^^^
2025-07-16T14:59:59.330862959Z File "/pyenv-2.6.4/versions/3.12.11/lib/python3.12/site-packages/guardrails/validator_service/sequential_validator_service.py", line 33, in run_validator_sync
2025-07-16T14:59:59.330864042Z result = self.execute_validator(
2025-07-16T14:59:59.330864834Z ^^^^^^^^^^^^^^^^^^^^^^^
2025-07-16T14:59:59.330865626Z File "/pyenv-2.6.4/versions/3.12.11/lib/python3.12/site-packages/guardrails/hub_telemetry/hub_tracing.py", line 144, in wrapper
2025-07-16T14:59:59.330866667Z resp = fn(*args, **kwargs)
2025-07-16T14:59:59.330867542Z ^^^^^^^^^^^^^^^^^^^
2025-07-16T14:59:59.330868459Z File "/pyenv-2.6.4/versions/3.12.11/lib/python3.12/site-packages/guardrails/validator_service/validator_service_base.py", line 70, in execute_validator
2025-07-16T14:59:59.330869501Z result = traced_validator(value, metadata)
2025-07-16T14:59:59.330870334Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-07-16T14:59:59.330871126Z File "/pyenv-2.6.4/versions/3.12.11/lib/python3.12/site-packages/guardrails/telemetry/validator_tracing.py", line 147, in trace_validator_wrapper
2025-07-16T14:59:59.330872167Z raise e
2025-07-16T14:59:59.330872917Z File "/pyenv-2.6.4/versions/3.12.11/lib/python3.12/site-packages/guardrails/telemetry/validator_tracing.py", line 117, in trace_validator_wrapper
2025-07-16T14:59:59.330873917Z resp = fn(*args, **kwargs)
2025-07-16T14:59:59.330874709Z ^^^^^^^^^^^^^^^^^^^
2025-07-16T14:59:59.330875792Z File "/pyenv-2.6.4/versions/3.12.11/lib/python3.12/site-packages/guardrails_grhub_toxic_language/main.py", line 160, in validate
2025-07-16T14:59:59.330876751Z return self.validate_each_sentence(value, metadata)
2025-07-16T14:59:59.330877626Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-07-16T14:59:59.330878459Z File "/pyenv-2.6.4/versions/3.12.11/lib/python3.12/site-packages/guardrails_grhub_toxic_language/main.py", line 122, in validate_each_sentence
2025-07-16T14:59:59.330882376Z sentence_predictions = self._inference(sentences)
2025-07-16T14:59:59.330883167Z ^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-07-16T14:59:59.330883834Z File "/pyenv-2.6.4/versions/3.12.11/lib/python3.12/site-packages/guardrails/hub_telemetry/hub_tracing.py", line 144, in wrapper
2025-07-16T14:59:59.330884626Z resp = fn(*args, **kwargs)
2025-07-16T14:59:59.330885209Z ^^^^^^^^^^^^^^^^^^^
2025-07-16T14:59:59.330885876Z File "/pyenv-2.6.4/versions/3.12.11/lib/python3.12/site-packages/guardrails/validator_base.py", line 272, in _inference
2025-07-16T14:59:59.330886626Z return self._inference_remote(model_input)
2025-07-16T14:59:59.330887292Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-07-16T14:59:59.330887959Z File "/pyenv-2.6.4/versions/3.12.11/lib/python3.12/site-packages/guardrails_grhub_toxic_language/main.py", line 198, in _inference_remote
2025-07-16T14:59:59.330888792Z response = self._hub_inference_request(json.dumps(request_body), self.validation_endpoint)
2025-07-16T14:59:59.330889542Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-07-16T14:59:59.330890292Z File "/pyenv-2.6.4/versions/3.12.11/lib/python3.12/site-packages/guardrails/validator_base.py", line 404, in _hub_inference_request
2025-07-16T14:59:59.330891126Z raise Exception(
2025-07-16T14:59:59.330891751Z Exception: 401: Remote Inference Unauthorized. Please run `guardrails configure`. You can find a new token at https://hub.guardrailsai.com/keys
If I run this without docker it works fine. I have tried several new tokens and they all give the same error
Expected behavior
Validation should work
Library version:
requirements-guardrails.txt
guardrails-ai==0.6.6
guardrails_api
gunicorn==23.0.0
Flask-Cors
nltk
torch
Additional context
This is only a problem in the docker version