Skip to content

docs: add policy-as-code tutorial chapter 5 (approval workflows)#911

Merged
imran-siddique merged 2 commits intomicrosoft:mainfrom
harinarayansrivatsan:docs/706-policy-as-code-tutorial
Apr 11, 2026
Merged

docs: add policy-as-code tutorial chapter 5 (approval workflows)#911
imran-siddique merged 2 commits intomicrosoft:mainfrom
harinarayansrivatsan:docs/706-policy-as-code-tutorial

Conversation

@harinarayansrivatsan
Copy link
Copy Markdown
Contributor

Summary

  • Adds chapter 5 (approval workflows) to the policy-as-code tutorial series
  • Introduces human-in-the-loop escalation as a third decision tier between allow and deny
  • Covers creating escalation requests, human approve/deny flows, timeout with safe defaults, and mentions quorum-based approval for production use

New files

  • docs/tutorials/policy-as-code/05-approval-workflows.md — tutorial with bank teller analogy, five steps, ASCII flowchart, and exercises
  • docs/tutorials/policy-as-code/examples/05_approval_policy.yaml — three-tier policy (allow, deny, escalate)
  • docs/tutorials/policy-as-code/examples/05_approval_workflows.py — runnable demo with five parts

Updated files

  • docs/tutorials/policy-as-code/README.md — link chapter 5, update coming-soon notice
  • docs/tutorials/policy-as-code/04-conditional-policies.md — update Next navigation link

Test plan

  • Run python docs/tutorials/policy-as-code/examples/05_approval_workflows.py and verify clean output across all 5 parts
  • Verify YAML loads without errors
  • Verify all markdown links resolve correctly

Ref #706

Copilot AI review requested due to automatic review settings April 9, 2026 04:27
@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 9, 2026

Welcome to the Agent Governance Toolkit! Thanks for your first pull request.
Please ensure tests pass, code follows style (ruff check), and you have signed the CLA.
See our Contributing Guide.

@github-actions github-actions bot added documentation Improvements or additions to documentation size/XL Extra large PR (500+ lines) labels Apr 9, 2026
Copy link
Copy Markdown

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI Agent: code-reviewer

Feedback on Pull Request: docs: add policy-as-code tutorial chapter 5 (approval workflows)


🔴 CRITICAL: Security Concerns

  1. Escalation Timeout Default Action

    • The default timeout action is set to DENY, which is a safe default. However, the tutorial suggests experimenting with DefaultTimeoutAction.ALLOW. This could lead to unintended consequences if users adopt this setting for critical actions. The tutorial should explicitly warn against using ALLOW for sensitive actions and emphasize the importance of failing safely.

    Actionable Recommendation: Add a warning in the tutorial that DefaultTimeoutAction.ALLOW should only be used for non-critical actions where escalation is optional.

  2. Audit Trail Integrity

    • The tutorial mentions that the audit trail records who approved or denied requests, but it does not discuss mechanisms to ensure the integrity of this audit trail. If the InMemoryApprovalQueue is used in production, it could be vulnerable to tampering or loss of data.

    Actionable Recommendation: Clearly state that InMemoryApprovalQueue is for demonstration purposes only and recommend using a secure, persistent backend (e.g., a database with access controls or a blockchain-based ledger) for production environments.

  3. Escalation Request Spoofing

    • The EscalationHandler does not appear to validate the agent_id or action fields in the escalation request. This could allow malicious agents to spoof requests or escalate actions they are not authorized to perform.

    Actionable Recommendation: Add validation logic to ensure that the agent_id and action fields in escalation requests are legitimate and authorized. This could involve checking against a registry of known agents and their allowed actions.


🟡 WARNING: Potential Breaking Changes

  1. Policy Format Changes

    • The introduction of the "requires human approval" keyword in policy messages is a new convention. If users update their policies without including this keyword, actions intended for escalation may be denied outright.

    Actionable Recommendation: Provide backward compatibility by allowing users to configure the escalation keyword or fallback behavior for policies missing the keyword.


💡 Suggestions for Improvement

  1. Concurrency and Thread Safety

    • The InMemoryApprovalQueue implementation does not appear to handle concurrent access. In a multi-threaded or distributed environment, this could lead to race conditions or inconsistent states.

    Actionable Recommendation: Implement thread-safe mechanisms (e.g., locks) for the InMemoryApprovalQueue or recommend using a distributed queue system for production use.

  2. Quorum-Based Approval

    • The tutorial briefly mentions quorum-based approval but does not provide an example. This is a critical feature for high-stakes actions requiring multiple sign-offs.

    Actionable Recommendation: Add an example demonstrating quorum-based approval using QuorumConfig to help users understand how to implement it.

  3. Policy Testing

    • The tutorial mentions the importance of automated policy testing but defers it to a future chapter. Given the criticality of policy correctness, users should be encouraged to start testing immediately.

    Actionable Recommendation: Provide a basic example of policy testing in this chapter, even if the full details are covered later.

  4. Type Safety and Validation

    • While the tutorial uses PolicyDocument.from_yaml() to load policies, it does not explicitly mention whether the YAML schema is validated. This could lead to runtime errors if the YAML is malformed.

    Actionable Recommendation: Ensure that the PolicyDocument class uses Pydantic models for schema validation and document this feature in the tutorial.

  5. Sandbox Escape Vectors

    • The tutorial does not address potential sandbox escape vectors for the agent during escalation. For example, if the agent can modify its own agent_id or action, it could bypass restrictions.

    Actionable Recommendation: Include a section in the tutorial discussing how to secure the agent's runtime environment and prevent unauthorized modifications to escalation requests.

  6. Improved Error Handling

    • The tutorial does not cover error handling for cases where the escalation backend is unavailable or fails to process requests.

    Actionable Recommendation: Add examples of error handling and fallback mechanisms for escalation workflows.


Summary of Changes Needed

  • 🔴 Add warnings and safeguards against insecure configurations (e.g., DefaultTimeoutAction.ALLOW).
  • 🔴 Ensure audit trail integrity and validate escalation requests to prevent spoofing.
  • 🟡 Provide backward compatibility for policies missing the escalation keyword.
  • 💡 Improve thread safety, add quorum-based approval examples, and introduce basic policy testing.
  • 💡 Enhance documentation on type safety, sandbox security, and error handling.

This tutorial is well-written and provides valuable insights into human-in-the-loop escalation workflows. Addressing the above concerns will ensure its robustness and security for production use.

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 9, 2026

🤖 AI Agent: security-scanner — Security Analysis of the Pull Request

Security Analysis of the Pull Request

This pull request primarily introduces a new tutorial chapter (Chapter 5) on implementing approval workflows in the policy-as-code framework. It includes documentation, a YAML policy file, and a Python script demonstrating the implementation of a three-tier policy system (allow, deny, escalate). Below is a detailed security analysis based on the specified criteria:


1. Prompt Injection Defense Bypass

  • Assessment: The tutorial introduces a mechanism to escalate certain actions for human review based on a keyword match ("requires human approval") in the policy's message field. This approach could be vulnerable to prompt injection attacks if an attacker can manipulate the message field in the policy YAML or the reason field in the escalation request.
  • Attack Vector: If an attacker can inject malicious content into the message field or the reason field, they could potentially manipulate the behavior of the escalation system or mislead human reviewers.
  • Rating: 🟠 HIGH
  • Recommendation:
    • Validate and sanitize all user-provided input, including message and reason fields, to prevent injection attacks.
    • Avoid relying on free-text fields like message for critical decision-making. Instead, use structured data (e.g., enums or predefined codes) to indicate escalation conditions.

2. Policy Engine Circumvention

  • Assessment: The policy engine appears robust, with clear prioritization of rules and a default action (allow). However, the use of a default allow action in the YAML policy could lead to unintended behavior if a rule is accidentally removed or misconfigured.
  • Attack Vector: If a critical rule (e.g., escalate-transfer-funds) is removed or modified, the system could default to allowing sensitive actions without human review.
  • Rating: 🔴 CRITICAL
  • Recommendation:
    • Implement a mechanism to validate the integrity of the policy YAML file (e.g., checksums or signatures).
    • Add automated tests to verify that all critical actions (e.g., transfer_funds, send_email) are correctly categorized as escalate or deny.
    • Consider changing the default action to deny in the YAML policy to ensure a fail-safe behavior.

3. Trust Chain Weaknesses

  • Assessment: The tutorial uses an InMemoryApprovalQueue for demonstration purposes, which is not suitable for production. The documentation mentions the availability of a WebhookApprovalBackend for production use but does not provide details on securing the communication channel or verifying the authenticity of the approver.
  • Attack Vector: If the production approval backend does not implement secure communication (e.g., HTTPS) or lacks proper authentication and authorization mechanisms, an attacker could spoof approval requests or responses.
  • Rating: 🟠 HIGH
  • Recommendation:
    • Ensure that the WebhookApprovalBackend uses secure communication (e.g., HTTPS with certificate validation).
    • Require strong authentication and authorization for human reviewers (e.g., OAuth, SAML, or mutual TLS).
    • Log all approval/denial actions with timestamps and approver identities for auditing.

4. Credential Exposure

  • Assessment: No credentials or secrets are directly exposed in the code or documentation. However, the tutorial does not address how to securely manage credentials for the WebhookApprovalBackend or other production components.
  • Attack Vector: If credentials for the approval backend are hardcoded or improperly managed, they could be exposed in logs, version control, or error messages.
  • Rating: 🟡 MEDIUM
  • Recommendation:
    • Use environment variables or a secure secrets management system (e.g., Azure Key Vault) to store sensitive credentials.
    • Ensure that logging and error messages do not inadvertently expose sensitive information.

5. Sandbox Escape

  • Assessment: The tutorial does not involve any sandboxing or process isolation mechanisms. However, the EscalationHandler and InMemoryApprovalQueue could be exploited if they execute untrusted code or commands.
  • Attack Vector: If an attacker can inject malicious content into the escalation request (e.g., via the reason field), they might exploit vulnerabilities in the approval backend or the agent runtime.
  • Rating: 🟠 HIGH
  • Recommendation:
    • Validate and sanitize all input fields, especially reason and action, to prevent command injection or code execution.
    • Use a secure sandboxing mechanism for executing untrusted code or commands.

6. Deserialization Attacks

  • Assessment: The YAML policy file is loaded using PolicyDocument.from_yaml(). If the YAML parser is not configured securely, it could be vulnerable to deserialization attacks.
  • Attack Vector: An attacker could craft a malicious YAML file that exploits the deserialization process to execute arbitrary code.
  • Rating: 🔴 CRITICAL
  • Recommendation:
    • Use a safe YAML parser (e.g., ruamel.yaml or PyYAML with Loader=SafeLoader) to prevent deserialization of arbitrary objects.
    • Validate the structure and content of the YAML file against a schema before loading it.

7. Race Conditions

  • Assessment: The EscalationHandler and InMemoryApprovalQueue do not appear to handle concurrent access explicitly. In a production environment, concurrent modifications to the approval queue could lead to race conditions.
  • Attack Vector: If multiple agents or reviewers interact with the approval queue simultaneously, it could result in inconsistent or incorrect decisions.
  • Rating: 🟠 HIGH
  • Recommendation:
    • Use thread-safe or distributed data structures (e.g., a database with transaction support) for the approval queue in production.
    • Implement locking or other concurrency control mechanisms to prevent race conditions.

8. Supply Chain

  • Assessment: The tutorial relies on the agent-os-kernel package, which is assumed to be a trusted dependency. However, there is no mention of verifying the integrity of this package or its dependencies.
  • Attack Vector: If an attacker compromises the agent-os-kernel package or introduces a malicious dependency (e.g., via dependency confusion or typosquatting), they could compromise the entire system.
  • Rating: 🟠 HIGH
  • Recommendation:
    • Use a dependency management tool that supports integrity verification (e.g., pip with hash checking or pip-tools).
    • Regularly audit dependencies for known vulnerabilities using tools like pip-audit or Snyk.
    • Consider pinning dependency versions to known-good releases.

Summary of Findings

Finding Rating Recommendation
Prompt injection defense bypass 🟠 HIGH Validate/sanitize input; avoid free-text fields for critical decisions.
Policy engine circumvention 🔴 CRITICAL Validate policy integrity; add automated tests; consider default deny action.
Trust chain weaknesses 🟠 HIGH Use HTTPS, strong authentication, and logging for approval backend.
Credential exposure 🟡 MEDIUM Use secure secrets management; avoid logging sensitive data.
Sandbox escape 🟠 HIGH Validate/sanitize input; use secure sandboxing for untrusted code.
Deserialization attacks 🔴 CRITICAL Use safe YAML parsers; validate YAML structure against a schema.
Race conditions 🟠 HIGH Use thread-safe or distributed data structures; implement concurrency controls.
Supply chain 🟠 HIGH Verify dependency integrity; audit dependencies; pin versions.

General Recommendations

  1. Automated Testing: Ensure that the policy system is thoroughly tested with automated test cases to verify that all rules work as intended and that no critical rules are accidentally removed or misconfigured.
  2. Production Readiness: The tutorial should emphasize the differences between the in-memory implementation and the production-ready WebhookApprovalBackend. It should also provide guidance on securing the production setup.
  3. Documentation: Update the documentation to include security best practices, such as validating input, using safe defaults, and securing the approval backend.

This pull request introduces a useful feature for human-in-the-loop approval workflows, but it also introduces several potential security risks that need to be addressed before deployment in a production environment.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds Chapter 5 (“Approval Workflows”) to the policy-as-code tutorial series, introducing human-in-the-loop escalation (approve/deny/timeout) with accompanying runnable Python and YAML examples, plus tutorial navigation updates.

Changes:

  • Add new Chapter 5 markdown tutorial covering escalation requests, human decisions, and timeout-safe defaults.
  • Add Chapter 5 runnable Python demo and YAML example policy.
  • Update tutorial README and Chapter 4 “Next” link to include Chapter 5.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
docs/tutorials/policy-as-code/README.md Links Chapter 5 and updates “coming soon” notice.
docs/tutorials/policy-as-code/04-conditional-policies.md Updates Next navigation to point to Chapter 5.
docs/tutorials/policy-as-code/05-approval-workflows.md New Chapter 5 tutorial text, flow, and exercises for approval workflows.
docs/tutorials/policy-as-code/examples/05_approval_policy.yaml New example policy illustrating allow/deny + “escalation-worthy” actions.
docs/tutorials/policy-as-code/examples/05_approval_workflows.py New runnable demo showing escalation queue/handler, approve/deny, and timeout behavior.

@harinarayansrivatsan
Copy link
Copy Markdown
Contributor Author

Here are my decisions for chapter 5:

For escalation I went with a message-field convention. Rules set action: deny with "requires human approval" in the message, and the Python code picks that up to route to the EscalationHandler. The four existing actions are all accounted for and the require_human_approval flag on GovernancePolicy is all-or-nothing, so this felt like the cleanest path.

If you'd want to have a proper "action: escalate" in PolicyAction, I kept things flexible enough that only the YAML and classify() would need to change.

Regarding quorum, I mentioned it in this tutorial but held off on the demo. Since the approve() method for the queue never populates the votes, so resolve() would result in a timeout. So I didn't want to mention it.

The script simulates both sides in one place.

Introduce human-in-the-loop escalation as a third decision tier between
allow and deny. The tutorial covers creating escalation requests,
human approve/deny flows, and timeout with safe defaults.

New files:
- 05-approval-workflows.md: tutorial narrative with bank teller analogy,
  five steps, ASCII flowchart, and exercises
- examples/05_approval_policy.yaml: three-tier policy (allow, deny, escalate)
- examples/05_approval_workflows.py: runnable demo with five parts

Updated files:
- README.md: link chapter 5, update coming-soon notice
- 04-conditional-policies.md: update Next navigation link
@harinarayansrivatsan harinarayansrivatsan force-pushed the docs/706-policy-as-code-tutorial branch from 2b518b6 to c145de9 Compare April 9, 2026 04:53
Copy link
Copy Markdown

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI Agent: code-reviewer

Feedback on Pull Request: docs: add policy-as-code tutorial chapter 5 (approval workflows)


🔴 CRITICAL: Security Issues

  1. Escalation Timeout Default Behavior

    • The DefaultTimeoutAction is set to DENY as a safe default, which is appropriate for most sensitive actions. However, the tutorial suggests experimenting with DefaultTimeoutAction.ALLOW. This could lead to silent approvals in production if misconfigured, creating a security bypass for high-stakes actions. Ensure that the tutorial explicitly warns against using ALLOW for critical actions and provides clear guidelines on when it might be acceptable.

    Actionable Recommendation: Add a prominent warning in the tutorial about the risks of using DefaultTimeoutAction.ALLOW for sensitive actions. Highlight scenarios where this might be acceptable and emphasize the importance of careful consideration.

  2. Lack of Cryptographic Integrity for Escalation Requests

    • The EscalationRequest object does not appear to include cryptographic integrity checks (e.g., HMAC or digital signatures). This could allow malicious actors to tamper with escalation requests in transit or at rest, potentially altering the decision or bypassing human review.

    Actionable Recommendation: Implement cryptographic integrity checks for EscalationRequest objects. Use HMAC or digital signatures to ensure that requests cannot be tampered with. Update the tutorial to reflect this change.

  3. Insecure Default Backend

    • The tutorial uses InMemoryApprovalQueue as the default backend for escalation requests. While this is acceptable for demonstration purposes, it is critical to warn users that this backend is not suitable for production use. It lacks persistence and security features, making it vulnerable to data loss and unauthorized access.

    Actionable Recommendation: Add a clear warning in the tutorial that InMemoryApprovalQueue is for demonstration only. Recommend using a secure, persistent backend (e.g., WebhookApprovalBackend or a database-backed implementation) for production environments.


🟡 WARNING: Potential Breaking Changes

  1. Policy YAML Format

    • The introduction of the escalate action in the policy YAML file changes the structure of policy documents. If users have existing policies, they may need to update them to accommodate this new action. This could lead to compatibility issues with older versions of the library.

    Actionable Recommendation: Provide clear migration instructions for users who need to update their existing policies. Consider adding backward compatibility by allowing older policies to function without the escalate action.


💡 Suggestions for Improvement

  1. Thread Safety

    • The InMemoryApprovalQueue is used in the tutorial, but there is no mention of thread safety. If multiple agents are running concurrently and interacting with the same queue, race conditions could occur.

    Actionable Recommendation: Ensure that InMemoryApprovalQueue is thread-safe by using synchronization primitives like locks. Alternatively, explicitly state that this implementation is not thread-safe and recommend using a thread-safe backend for concurrent environments.

  2. Policy Testing

    • The tutorial mentions the importance of automated policy testing but does not provide concrete examples or tools for implementing such tests.

    Actionable Recommendation: Add a section with sample test cases for policy validation using pytest. For example, demonstrate how to test that transfer_funds always escalates and delete_database always denies.

  3. Quorum-Based Approval

    • The tutorial briefly mentions quorum-based approval via QuorumConfig but does not provide an example.

    Actionable Recommendation: Include a code snippet demonstrating how to configure and use QuorumConfig for actions requiring multiple sign-offs. This would make the tutorial more comprehensive.

  4. Type Safety and Validation

    • While the tutorial uses Pydantic models for policy documents, it does not explicitly demonstrate how validation errors are handled or how users can debug them.

    Actionable Recommendation: Add a section explaining how to handle validation errors when loading policies using PolicyDocument.from_yaml. Include examples of common validation errors and how to resolve them.

  5. Sandbox Escape Vectors

    • The tutorial does not address potential sandbox escape vectors for agents that may exploit the escalation mechanism to execute unauthorized actions.

    Actionable Recommendation: Add a section discussing best practices for securing the escalation mechanism, such as restricting the actions that can be escalated and validating the agent's identity using SPIFFE/SVID.

  6. Backward Compatibility

    • The tutorial introduces new functionality (escalate action) but does not address backward compatibility for existing users.

    Actionable Recommendation: Provide clear documentation on how users can upgrade their policies and code to use the new escalation feature without breaking existing functionality.


Summary

This pull request introduces a valuable feature for human-in-the-loop approval workflows, but it raises several critical security concerns and potential compatibility issues. Addressing these concerns will ensure the robustness and safety of the feature in production environments. Additionally, incorporating the suggested improvements will enhance the tutorial's utility and clarity for users.

Copy link
Copy Markdown
Member

@imran-siddique imran-siddique left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: CHANGES REQUESTED

Diff reviewed — 5 files: new tutorial chapter 5 (approval workflows) + example YAML + example Python + README/nav updates. 700 additions. Ref #706.

Security checklist:

  • No eval(), exec(), pickle, shell=True, innerHTML
  • No hardcoded secrets
  • Scope matches description — tutorial files only
  • Python example has MIT license header
  • Additions > 0

Issues to fix:

  1. Missing license header on new .md file: 05-approval-workflows.md needs the MIT copyright header:
    <!-- Copyright (c) Microsoft Corporation. -->
    <!-- Licensed under the MIT License. -->

  2. Missing license header on new .yaml file: examples/05_approval_policy.yaml needs:
    # Copyright (c) Microsoft Corporation.
    # Licensed under the MIT License.

Content quality is good — the bank teller analogy works well, step progression is clear, and the Python demo covers all 5 parts. Happy to approve once headers are added.

Copy link
Copy Markdown

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI Agent: code-reviewer

Review Summary

This pull request introduces Chapter 5 of the policy-as-code tutorial series, focusing on approval workflows and human-in-the-loop escalation mechanisms. The tutorial is well-written, provides clear examples, and introduces a critical concept for security-sensitive systems. However, there are areas that require attention to ensure correctness, security, and compliance with best practices.


🔴 CRITICAL: Security Issues

  1. Escalation Timeout Default Behavior

    • The tutorial defaults to DefaultTimeoutAction.DENY, which is a safe choice. However, the tutorial mentions the possibility of using DefaultTimeoutAction.ALLOW for less critical actions. This could lead to silent approvals in production environments if misconfigured, creating a security bypass risk.
    • Recommendation: Explicitly warn users against using DefaultTimeoutAction.ALLOW unless absolutely necessary. Highlight the risks of silent approvals in production environments.
  2. Audit Trail Integrity

    • The tutorial mentions recording the approver's identity (e.g., manager@corp.com) but does not specify how this data is secured or verified. If the identity is spoofed or tampered with, it could lead to unauthorized approvals.
    • Recommendation: Ensure that the approve() and deny() methods enforce cryptographic verification of the approver's identity (e.g., via SPIFFE/SVID or signed tokens).
  3. In-Memory Approval Queue

    • The InMemoryApprovalQueue is used for demonstration purposes but is not suitable for production. If accidentally used in production, it could lead to loss of escalation requests on process termination.
    • Recommendation: Add explicit warnings in the tutorial and code comments that InMemoryApprovalQueue is for testing only. Provide examples of production-ready backends (e.g., database, message broker).

🟡 WARNING: Potential Breaking Changes

  1. Policy YAML Format
    • The policy YAML introduces a new message field that is critical for escalation decisions. If users accidentally omit this field, the escalation mechanism will silently fail, and sensitive actions could be denied instead of escalated.
    • Recommendation: Update the PolicyDocument schema to enforce validation of the message field for escalation rules. Consider making this field mandatory for rules with action: deny.

💡 Suggestions for Improvement

  1. Thread Safety

    • The InMemoryApprovalQueue and EscalationHandler do not appear to be thread-safe. In concurrent agent execution scenarios, race conditions could occur when multiple agents escalate requests simultaneously.
    • Recommendation: Document the lack of thread safety in the tutorial and suggest using thread-safe or distributed backends for production.
  2. Quorum-Based Approval

    • The tutorial briefly mentions quorum-based approval but does not provide an example. This is a critical feature for high-stakes actions.
    • Recommendation: Add an example demonstrating quorum-based approval using QuorumConfig.
  3. Policy Testing

    • The tutorial ends by highlighting the importance of automated policy testing but does not provide actionable steps.
    • Recommendation: Add a section on how to write unit tests for policies using pytest and mock escalation handlers.
  4. Type Safety

    • The EscalationRequest and PolicyDocument classes should use Pydantic models for strict type validation. This ensures that invalid data (e.g., missing fields) is caught early.
    • Recommendation: Refactor these classes to use Pydantic models and enforce schema validation.
  5. Backward Compatibility

    • The addition of the message field in the policy YAML could break existing policies if users upgrade the library without updating their YAML files.
    • Recommendation: Provide migration guidance for users upgrading from earlier versions of the library.

Additional Observations

  1. Documentation Quality

    • The tutorial is well-written and provides clear examples. The use of a bank teller analogy is effective for explaining escalation workflows.
    • Suggestion: Consider adding diagrams or screenshots of a hypothetical approval dashboard to make the tutorial more engaging.
  2. Sandbox Escape Vectors

    • The tutorial does not address potential sandbox escape vectors for agents that are paused during escalation. If an agent can bypass the escalation mechanism, it could execute sensitive actions without approval.
    • Recommendation: Ensure that the agent's execution environment is sandboxed and that the escalation mechanism is enforced at the system level.

Action Items

  1. Address the 🔴 CRITICAL issues related to security bypass risks, audit trail integrity, and production readiness.
  2. Implement the 🟡 WARNING recommendation to enforce validation of the message field in the policy schema.
  3. Consider the 💡 Suggestions for thread safety, quorum-based approval examples, policy testing, type safety, and backward compatibility.

This tutorial is a valuable addition to the repository, but addressing the highlighted issues will ensure it meets the highest standards for security and reliability.

@harinarayansrivatsan
Copy link
Copy Markdown
Contributor Author

Review: CHANGES REQUESTED

Diff reviewed — 5 files: new tutorial chapter 5 (approval workflows) + example YAML + example Python + README/nav updates. 700 additions. Ref #706.

Security checklist:

  • No eval(), exec(), pickle, shell=True, innerHTML
  • No hardcoded secrets
  • Scope matches description — tutorial files only
  • Python example has MIT license header
  • Additions > 0

Issues to fix:

  1. Missing license header on new .md file: 05-approval-workflows.md needs the MIT copyright header:
    <!-- Copyright (c) Microsoft Corporation. -->
    <!-- Licensed under the MIT License. -->
  2. Missing license header on new .yaml file: examples/05_approval_policy.yaml needs:
    # Copyright (c) Microsoft Corporation.
    # Licensed under the MIT License.

Content quality is good — the bank teller analogy works well, step progression is clear, and the Python demo covers all 5 parts. Happy to approve once headers are added.

Thank you! I've now added the license headers on the files.

@imran-siddique imran-siddique merged commit 327411d into microsoft:main Apr 11, 2026
6 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/XL Extra large PR (500+ lines)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants