Add mi-24: Human-in-the-Loop Action Approval Gate mitigation - #339
Add mi-24: Human-in-the-Loop Action Approval Gate mitigation#339sunilp wants to merge 2 commits into
Conversation
Signed-off-by: sdev <sunil@sunilprakash.com>
ColinEberhardt
left a comment
ColinEberhardt
left a comment
There was a problem hiding this comment.
I think this is a really good addition, thanks - any feedback from other @finos/ai-governance-framework-maintainers ?
|
Agreed! Useful addition! The bit about "Which actions require approval must be defined declaratively" sounds like it rules out the "Automatically approve" feature of Claude Desktop and some coding assistants. I think there should be carve outs for those kinds of tools (which may require stern wording about risk analyses, user training, etc.). Otherwise adopting AIGF could put a large portion of the organization into non-compliance immediately. :-) |
|
@alvin-c-shih - Incorporated it in 917f5fe. The wording did read as a ban, and rather than carve out a tool category I made pre-authorization first class. New section on standing approval: an "always allow" toggle is a human with authority releasing a bounded class of future actions in advance, which is a form of the decision rather than a hole in the gate. The bounds are what matter. A grant cannot exceed the grantor's own delegated limit for that action class and environment, it carries scope, environment and expiry, it is recorded as approval evidence, and the agent must never be able to solicit, widen, or name-match its own grant. Also added a scope line in Purpose and a Challenges bullet: the gate applies to the action classes an institution designates, and the practical adoption task is bounding grants that already exist rather than introducing a gate where none exists. That should cover your day-one non-compliance point. Kept it vendor neutral, since the same class of tool that edits a file on a laptop can also hold a credential into production. |
Addresses review feedback that "defined declaratively" reads as ruling out the
pre-authorization ("always allow this tool") behaviour of everyday agent tooling.
- Purpose: scope the control to the action classes an institution designates
- Policy-Driven Classification: declarative constrains how a rule is expressed
and enforced, not who writes it or when it is decided
- Fail-Closed: no decision and no standing approval in force
- New section 5, Standing Approval and Pre-Authorization: grant as risk
acceptance, the forms are distinct controls, bounded effect surface only,
scope/environment/expiry, persistence and inheritance, evidence, and the
agent never obtaining or name-matching its own grant
- Challenges: bounding grants that already exist is the practical adoption task
- Operational Safeguards: broad standing grants as the end state of over-gating
Signed-off-by: sdev <sunil@sunilprakash.com>
917f5fe to
6f0a794
Compare
3 participants
This adds a preventative mitigation for agentic systems: an approval gate that intercepts a designated high-risk action after its parameters are resolved and holds execution until an authenticated human with the right authority decides.
Why a separate mitigation: mi-18 requires human approval in several of its scenarios (privilege escalation, operations over risk thresholds) but the framework does not yet define the approval mechanism those requirements depend on. This entry specifies that mechanism: the request lifecycle, durable pending state, binding of the decision to the exact action version, the oversight patterns (user confirmation, maker-checker, dual authorization), execution semantics across crash and recovery, and the minimum approval evidence set. mi-11 is adjacent but collects evaluative feedback; it does not authorize release of a specific action.
Content notes:
Relates to #200 and #120.
Happy to renumber if another mitigation lands first, and to trim or split sections per maintainer preference.