Skip to content

[bug] ToxicLanguage fails on empty input. #1293

@miscenko

Description

@miscenko

Describe the bug
This issue is basically already described in: #619.
Seems like it was not fixed (stack trace provided below).

To Reproduce

API call : /guards/{guardName}/validate call fails with sample body:

{
  "llmOutput": ""
}

Response body:

{
  "message": "Internal Server Error",
  "status": 500
}

Stack trace (from container but the behavior is the same everyewhere):

guardrails-1  | ERROR:guardrails-api:Value cannot be empty.
guardrails-1  | Traceback (most recent call last):
guardrails-1  |   File "/app/.venv/lib/python3.12/site-packages/guardrails_api/utils/handle_error.py", line 13, in decorator
guardrails-1  |     return fn(*args, **kwargs)
guardrails-1  |            ^^^^^^^^^^^^^^^^^^^
guardrails-1  |   File "/app/.venv/lib/python3.12/site-packages/guardrails_api/blueprints/guards.py", line 316, in validate
guardrails-1  |     result: ValidationOutcome = guard.parse(
guardrails-1  |                                 ^^^^^^^^^^^^
guardrails-1  |   File "/app/.venv/lib/python3.12/site-packages/guardrails/hub_telemetry/hub_tracing.py", line 154, in wrapper
guardrails-1  |     return fn(*args, **kwargs)
guardrails-1  |            ^^^^^^^^^^^^^^^^^^^
guardrails-1  |   File "/app/.venv/lib/python3.12/site-packages/guardrails/guard.py", line 1056, in parse
guardrails-1  |     return trace_guard_execution(
guardrails-1  |            ^^^^^^^^^^^^^^^^^^^^^^
guardrails-1  |   File "/app/.venv/lib/python3.12/site-packages/guardrails/telemetry/guard_tracing.py", line 198, in trace_guard_execution
guardrails-1  |     raise e
guardrails-1  |   File "/app/.venv/lib/python3.12/site-packages/guardrails/telemetry/guard_tracing.py", line 187, in trace_guard_execution
guardrails-1  |     result = _execute_fn(*args, **kwargs)
guardrails-1  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
guardrails-1  |   File "/app/.venv/lib/python3.12/site-packages/guardrails/guard.py", line 858, in _execute
guardrails-1  |     return guard_context.run(
guardrails-1  |            ^^^^^^^^^^^^^^^^^^
guardrails-1  |   File "/app/.venv/lib/python3.12/site-packages/guardrails/telemetry/common.py", line 100, in wrapped_func
guardrails-1  |     return func(*args, **kwargs)
guardrails-1  |            ^^^^^^^^^^^^^^^^^^^^^
guardrails-1  |   File "/app/.venv/lib/python3.12/site-packages/guardrails/guard.py", line 835, in __exec
guardrails-1  |     return self._exec(
guardrails-1  |            ^^^^^^^^^^^
guardrails-1  |   File "/app/.venv/lib/python3.12/site-packages/guardrails/guard.py", line 944, in _exec
guardrails-1  |     call = runner(call_log=call_log, prompt_params=prompt_params)
guardrails-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
guardrails-1  |   File "/app/.venv/lib/python3.12/site-packages/guardrails/hub_telemetry/hub_tracing.py", line 154, in wrapper
guardrails-1  |     return fn(*args, **kwargs)
guardrails-1  |            ^^^^^^^^^^^^^^^^^^^
guardrails-1  |   File "/app/.venv/lib/python3.12/site-packages/guardrails/run/runner.py", line 234, in __call__
guardrails-1  |     raise e
guardrails-1  |   File "/app/.venv/lib/python3.12/site-packages/guardrails/run/runner.py", line 199, in __call__
guardrails-1  |     iteration = self.step(
guardrails-1  |                 ^^^^^^^^^^
guardrails-1  |   File "/app/.venv/lib/python3.12/site-packages/guardrails/hub_telemetry/hub_tracing.py", line 154, in wrapper
guardrails-1  |     return fn(*args, **kwargs)
guardrails-1  |            ^^^^^^^^^^^^^^^^^^^
guardrails-1  |   File "/app/.venv/lib/python3.12/site-packages/guardrails/telemetry/runner_tracing.py", line 85, in trace_step_wrapper
guardrails-1  |     raise e
guardrails-1  |   File "/app/.venv/lib/python3.12/site-packages/guardrails/telemetry/runner_tracing.py", line 77, in trace_step_wrapper
guardrails-1  |     response = fn(*args, **kwargs)
guardrails-1  |                ^^^^^^^^^^^^^^^^^^^
guardrails-1  |   File "/app/.venv/lib/python3.12/site-packages/guardrails/run/runner.py", line 328, in step
guardrails-1  |     raise e
guardrails-1  |   File "/app/.venv/lib/python3.12/site-packages/guardrails/run/runner.py", line 313, in step
guardrails-1  |     validated_output = self.validate(
guardrails-1  |                        ^^^^^^^^^^^^^^
guardrails-1  |   File "/app/.venv/lib/python3.12/site-packages/guardrails/hub_telemetry/hub_tracing.py", line 154, in wrapper
guardrails-1  |     return fn(*args, **kwargs)
guardrails-1  |            ^^^^^^^^^^^^^^^^^^^
guardrails-1  |   File "/app/.venv/lib/python3.12/site-packages/guardrails/run/runner.py", line 595, in validate
guardrails-1  |     validated_output, metadata = validator_service.validate(
guardrails-1  |                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^
guardrails-1  |   File "/app/.venv/lib/python3.12/site-packages/guardrails/validator_service/__init__.py", line 91, in validate
guardrails-1  |     return validator_service.validate(
guardrails-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
guardrails-1  |   File "/app/.venv/lib/python3.12/site-packages/guardrails/validator_service/sequential_validator_service.py", line 459, in validate
guardrails-1  |     value, metadata = self.run_validators(
guardrails-1  |                       ^^^^^^^^^^^^^^^^^^^^
guardrails-1  |   File "/app/.venv/lib/python3.12/site-packages/guardrails/validator_service/sequential_validator_service.py", line 355, in run_validators
guardrails-1  |     validator_logs = self.run_validator(
guardrails-1  |                      ^^^^^^^^^^^^^^^^^^^
guardrails-1  |   File "/app/.venv/lib/python3.12/site-packages/guardrails/validator_service/sequential_validator_service.py", line 66, in run_validator
guardrails-1  |     result = self.run_validator_sync(
guardrails-1  |              ^^^^^^^^^^^^^^^^^^^^^^^^
guardrails-1  |   File "/app/.venv/lib/python3.12/site-packages/guardrails/validator_service/sequential_validator_service.py", line 33, in run_validator_sync
guardrails-1  |     result = self.execute_validator(
guardrails-1  |              ^^^^^^^^^^^^^^^^^^^^^^^
guardrails-1  |   File "/app/.venv/lib/python3.12/site-packages/guardrails/hub_telemetry/hub_tracing.py", line 154, in wrapper
guardrails-1  |     return fn(*args, **kwargs)
guardrails-1  |            ^^^^^^^^^^^^^^^^^^^
guardrails-1  |   File "/app/.venv/lib/python3.12/site-packages/guardrails/validator_service/validator_service_base.py", line 70, in execute_validator
guardrails-1  |     result = traced_validator(value, metadata)
guardrails-1  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
guardrails-1  |   File "/app/.venv/lib/python3.12/site-packages/guardrails/telemetry/validator_tracing.py", line 137, in trace_validator_wrapper
guardrails-1  |     raise e
guardrails-1  |   File "/app/.venv/lib/python3.12/site-packages/guardrails/telemetry/validator_tracing.py", line 107, in trace_validator_wrapper
guardrails-1  |     resp = fn(*args, **kwargs)
guardrails-1  |            ^^^^^^^^^^^^^^^^^^^
guardrails-1  |   File "/app/.venv/lib/python3.12/site-packages/guardrails/hub/guardrails/toxic_language/validator/main.py", line 158, in validate
guardrails-1  |     raise ValueError("Value cannot be empty.")
guardrails-1  | ValueError: Value cannot be empty.

Expected behavior
Empty input should be "passed", ToxicLanguage validator shouldn't fail on it.

Library version:
Found in 0.5.12 but 0.5.15 behaves identically.

Additional context
IMPORTANT: The validation-failing message gets added to a request history in guardrails. Guardrails tries to load this history on each API call, and fails with the same unhandled pydantic validation exception, returning a 500. In other words, once any content: "" has been sent, all RCA proxy requests will fail.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions