Fix target-scoped single-agent tool validation#145
Conversation
|
Thank you @rockyzl for the PR. This is a tricky failure mode, and it is great that you went for a deterministic guardrail with real routing tests. The graph wiring reads cleanly. My one bigger-picture worry is the design: the decision rests on re-parsing the free-text query, guessing the molecule from the filename, and string-matching the two. Even once polished, that base tends to drift, e.g. "dipole moment of the water molecule" gives target "water molecule" ≠ species "water", and the default molecule.xyz resolves to a generic "molecule". Each miss usually becomes another special case on the same guessing layer. The sturdier version is probably to track the active target molecule as explicit graph state (set when a molecule is resolved) and give coordinate files provenance, so run_ase checks identity structurally rather than from strings. One thing I'd fix regardless: when a blocked run_ase shares a message with other parallel tool calls, only the blocked id gets a ToolMessage, so the siblings go unanswered and the next turn can "400 Bad Request". Worth handling all tool_call_ids. |
Summary
ToolNodeexecutesrun_ase.ToolMessagerepair instruction so it can generate/reuse coordinates for the current target instead of running on stale artifacts.Why
This addresses the smallest validation/routing slice discussed in #135: a follow-up such as
glucose dipolemomentshould not executerun_aseon a coordinate file produced earlier for another molecule, such as methane.Demo scenario
Run the Streamlit UI:
Then use a single chat session:
Ask for a property that creates a coordinate artifact for one molecule, for example:
Follow up with a different molecule target:
The routing risk is when the model tries to call
run_asewith the old file, for example:{ "name": "run_ase", "args": { "params": { "input_structure_file": "/tmp/methane.xyz", "driver": "dipole" } } }Without this guard, that stale tool call can reach
ToolNode. After this PR, it is blocked and the agent receives feedback to resolve/generate coordinates for the current target,glucose, before callingrun_ase.sequenceDiagram participant User participant Agent participant Validator participant ToolNode User->>Agent: What is the IR spectrum of methane? Agent->>ToolNode: smiles_to_coordinate_file -> /tmp/methane.xyz User->>Agent: glucose dipolemoment Agent->>Validator: run_ase(input_structure_file=/tmp/methane.xyz) Validator-->>Agent: Blocked: current target is glucose, file belongs to methane Agent->>ToolNode: generate/reuse glucose coordinates, then run_ase on glucoseThe focused test is
test_route_tools_blocks_stale_target_coordinate_reuseintests/test_single_agent_routing.py.Validation
git diff --checkPYTHONPATH=src python -m py_compile src/chemgraph/utils/tool_validation.py src/chemgraph/graphs/single_agent.pyPYTHONPATH=src python -m pytest tests/test_single_agent_routing.py -q-> 6 passedPYTHONPATH=src python -m pytest tests/test_single_agent_routing.py tests/test_graphs.py tests/test_llm_agent.py tests/test_ui_main_interface.py -q-> 37 passedPYTHONPATH=src python -m pytest -q-> 168 passed, 6 skippedRefs #135