-
Notifications
You must be signed in to change notification settings - Fork 427
Description
Describe the bug
This issue is basically already described in: #619.
Seems like it was not fixed (stack trace provided below).
To Reproduce
API call : /guards/{guardName}/validate
call fails with sample body:
{
"llmOutput": ""
}
Response body:
{
"message": "Internal Server Error",
"status": 500
}
Stack trace (from container but the behavior is the same everyewhere):
guardrails-1 | ERROR:guardrails-api:Value cannot be empty.
guardrails-1 | Traceback (most recent call last):
guardrails-1 | File "/app/.venv/lib/python3.12/site-packages/guardrails_api/utils/handle_error.py", line 13, in decorator
guardrails-1 | return fn(*args, **kwargs)
guardrails-1 | ^^^^^^^^^^^^^^^^^^^
guardrails-1 | File "/app/.venv/lib/python3.12/site-packages/guardrails_api/blueprints/guards.py", line 316, in validate
guardrails-1 | result: ValidationOutcome = guard.parse(
guardrails-1 | ^^^^^^^^^^^^
guardrails-1 | File "/app/.venv/lib/python3.12/site-packages/guardrails/hub_telemetry/hub_tracing.py", line 154, in wrapper
guardrails-1 | return fn(*args, **kwargs)
guardrails-1 | ^^^^^^^^^^^^^^^^^^^
guardrails-1 | File "/app/.venv/lib/python3.12/site-packages/guardrails/guard.py", line 1056, in parse
guardrails-1 | return trace_guard_execution(
guardrails-1 | ^^^^^^^^^^^^^^^^^^^^^^
guardrails-1 | File "/app/.venv/lib/python3.12/site-packages/guardrails/telemetry/guard_tracing.py", line 198, in trace_guard_execution
guardrails-1 | raise e
guardrails-1 | File "/app/.venv/lib/python3.12/site-packages/guardrails/telemetry/guard_tracing.py", line 187, in trace_guard_execution
guardrails-1 | result = _execute_fn(*args, **kwargs)
guardrails-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
guardrails-1 | File "/app/.venv/lib/python3.12/site-packages/guardrails/guard.py", line 858, in _execute
guardrails-1 | return guard_context.run(
guardrails-1 | ^^^^^^^^^^^^^^^^^^
guardrails-1 | File "/app/.venv/lib/python3.12/site-packages/guardrails/telemetry/common.py", line 100, in wrapped_func
guardrails-1 | return func(*args, **kwargs)
guardrails-1 | ^^^^^^^^^^^^^^^^^^^^^
guardrails-1 | File "/app/.venv/lib/python3.12/site-packages/guardrails/guard.py", line 835, in __exec
guardrails-1 | return self._exec(
guardrails-1 | ^^^^^^^^^^^
guardrails-1 | File "/app/.venv/lib/python3.12/site-packages/guardrails/guard.py", line 944, in _exec
guardrails-1 | call = runner(call_log=call_log, prompt_params=prompt_params)
guardrails-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
guardrails-1 | File "/app/.venv/lib/python3.12/site-packages/guardrails/hub_telemetry/hub_tracing.py", line 154, in wrapper
guardrails-1 | return fn(*args, **kwargs)
guardrails-1 | ^^^^^^^^^^^^^^^^^^^
guardrails-1 | File "/app/.venv/lib/python3.12/site-packages/guardrails/run/runner.py", line 234, in __call__
guardrails-1 | raise e
guardrails-1 | File "/app/.venv/lib/python3.12/site-packages/guardrails/run/runner.py", line 199, in __call__
guardrails-1 | iteration = self.step(
guardrails-1 | ^^^^^^^^^^
guardrails-1 | File "/app/.venv/lib/python3.12/site-packages/guardrails/hub_telemetry/hub_tracing.py", line 154, in wrapper
guardrails-1 | return fn(*args, **kwargs)
guardrails-1 | ^^^^^^^^^^^^^^^^^^^
guardrails-1 | File "/app/.venv/lib/python3.12/site-packages/guardrails/telemetry/runner_tracing.py", line 85, in trace_step_wrapper
guardrails-1 | raise e
guardrails-1 | File "/app/.venv/lib/python3.12/site-packages/guardrails/telemetry/runner_tracing.py", line 77, in trace_step_wrapper
guardrails-1 | response = fn(*args, **kwargs)
guardrails-1 | ^^^^^^^^^^^^^^^^^^^
guardrails-1 | File "/app/.venv/lib/python3.12/site-packages/guardrails/run/runner.py", line 328, in step
guardrails-1 | raise e
guardrails-1 | File "/app/.venv/lib/python3.12/site-packages/guardrails/run/runner.py", line 313, in step
guardrails-1 | validated_output = self.validate(
guardrails-1 | ^^^^^^^^^^^^^^
guardrails-1 | File "/app/.venv/lib/python3.12/site-packages/guardrails/hub_telemetry/hub_tracing.py", line 154, in wrapper
guardrails-1 | return fn(*args, **kwargs)
guardrails-1 | ^^^^^^^^^^^^^^^^^^^
guardrails-1 | File "/app/.venv/lib/python3.12/site-packages/guardrails/run/runner.py", line 595, in validate
guardrails-1 | validated_output, metadata = validator_service.validate(
guardrails-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^
guardrails-1 | File "/app/.venv/lib/python3.12/site-packages/guardrails/validator_service/__init__.py", line 91, in validate
guardrails-1 | return validator_service.validate(
guardrails-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^
guardrails-1 | File "/app/.venv/lib/python3.12/site-packages/guardrails/validator_service/sequential_validator_service.py", line 459, in validate
guardrails-1 | value, metadata = self.run_validators(
guardrails-1 | ^^^^^^^^^^^^^^^^^^^^
guardrails-1 | File "/app/.venv/lib/python3.12/site-packages/guardrails/validator_service/sequential_validator_service.py", line 355, in run_validators
guardrails-1 | validator_logs = self.run_validator(
guardrails-1 | ^^^^^^^^^^^^^^^^^^^
guardrails-1 | File "/app/.venv/lib/python3.12/site-packages/guardrails/validator_service/sequential_validator_service.py", line 66, in run_validator
guardrails-1 | result = self.run_validator_sync(
guardrails-1 | ^^^^^^^^^^^^^^^^^^^^^^^^
guardrails-1 | File "/app/.venv/lib/python3.12/site-packages/guardrails/validator_service/sequential_validator_service.py", line 33, in run_validator_sync
guardrails-1 | result = self.execute_validator(
guardrails-1 | ^^^^^^^^^^^^^^^^^^^^^^^
guardrails-1 | File "/app/.venv/lib/python3.12/site-packages/guardrails/hub_telemetry/hub_tracing.py", line 154, in wrapper
guardrails-1 | return fn(*args, **kwargs)
guardrails-1 | ^^^^^^^^^^^^^^^^^^^
guardrails-1 | File "/app/.venv/lib/python3.12/site-packages/guardrails/validator_service/validator_service_base.py", line 70, in execute_validator
guardrails-1 | result = traced_validator(value, metadata)
guardrails-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
guardrails-1 | File "/app/.venv/lib/python3.12/site-packages/guardrails/telemetry/validator_tracing.py", line 137, in trace_validator_wrapper
guardrails-1 | raise e
guardrails-1 | File "/app/.venv/lib/python3.12/site-packages/guardrails/telemetry/validator_tracing.py", line 107, in trace_validator_wrapper
guardrails-1 | resp = fn(*args, **kwargs)
guardrails-1 | ^^^^^^^^^^^^^^^^^^^
guardrails-1 | File "/app/.venv/lib/python3.12/site-packages/guardrails/hub/guardrails/toxic_language/validator/main.py", line 158, in validate
guardrails-1 | raise ValueError("Value cannot be empty.")
guardrails-1 | ValueError: Value cannot be empty.
Expected behavior
Empty input should be "passed", ToxicLanguage validator shouldn't fail on it.
Library version:
Found in 0.5.12 but 0.5.15 behaves identically.
Additional context
IMPORTANT: The validation-failing message gets added to a request history in guardrails. Guardrails tries to load this history on each API call, and fails with the same unhandled pydantic validation exception, returning a 500. In other words, once any content: "" has been sent, all RCA proxy requests will fail.