Revise Foundry agent creation, permissions, and integration instructions

Instructions/Consolidated/D-observe-evaluate-and-secure-agents.md

@@ -96,8 +96,8 @@ it runs". The **Optional** task then goes after safety.
 
 By completing the **Core** tasks of this exercise, you'll be able to:
 
-- **Trace an agent** with OpenTelemetry, export the traces to Azure Monitor, and read them
-  in the Foundry portal — including custom spans you add around your own code.
+- **Trace an agent** with OpenTelemetry, export the traces to Azure Monitor, and read them —
+  Foundry's own server-side trace and your custom spans are both there, in different views.
 - **Evaluate answer quality** against ground truth using built-in evaluators
   (groundedness, relevance, similarity) and a JSONL dataset.
 
@@ -141,11 +141,10 @@ how it stands up to attack.
 - **Core only (~1h):** do Tasks 1–2.
 - **Everything (~1h 35m):** add **Task 3**, the red team scan.
 
-> **One agent, three questions**: Task 1 traces an agent you create in code. Tasks 2 and 3
-> both point at the **grounded knowledge agent** from
-> [Lab B](B-integrate-agents-with-enterprise-knowledge-and-m365.md). If you haven't done Lab B,
-> one command creates an equivalent agent so this lab stands alone — see
-> [Getting started](D0-getting-started.md).
+> **One agent, three questions**: all three tasks point at the same **grounded knowledge
+> agent** — Task 1 traces it, Tasks 2 and 3 measure it. If you haven't done
+> [Lab B](B-integrate-agents-with-enterprise-knowledge-and-m365.md), one command creates an
+> equivalent agent so this lab stands alone — see [Getting started](D0-getting-started.md).
 
 ## Measure, don't guess
 
@@ -166,8 +165,8 @@ production.
 
 Across this lab you:
 
-- **Instrumented an agent** with OpenTelemetry, exported traces to Application Insights, and
-  read them — including your own custom spans — in the Foundry portal.
+- **Instrumented an agent** with OpenTelemetry and exported traces to Application Insights —
+  reading Foundry's automatic server-side trace and your own custom spans, side by side.
 - **Evaluated a grounded agent** against a ground-truth dataset with built-in groundedness,
   relevance and similarity evaluators, and got a score you can compare across changes.
 - (Optionally) **Red teamed the agent** with adversarial attack strategies and your own seed
@@ -182,7 +181,7 @@ If you're finished, delete the resources you created to avoid unnecessary Azure
 1. In the [Azure portal](https://portal.azure.com), navigate to the resource group that contains your Foundry resource.
 1. On the toolbar, select **Delete resource group**, enter the resource group name, and confirm.
 
-> The code you run in Task 1 deletes the agent version it creates. The agent Tasks 2 and 3
-> measure is removed when you delete the resource group. If you provisioned with `azd`, run
-> `azd down` instead — but note that Application Insights, if you created it from the Foundry
-> portal, is a separate resource and is deleted with the resource group rather than by `azd`.
+> All three tasks measure the same `caldova-knowledge-agent`, so deleting the resource group
+> removes it along with everything else. If you provisioned with `azd`, run `azd down`
+> instead — but note that Application Insights, if you created it from the Foundry portal, is
+> a separate resource and is deleted with the resource group rather than by `azd`.

Instructions/Consolidated/D0-getting-started.md

@@ -72,11 +72,16 @@ portal (the default), or provision them with one command using the Azure Develop
 ### Option B — Provision with azd (optional, one command)
 
 If you'd rather not click through the portal, the lab ships an optional `azd` template that
-creates the Foundry resource, a project, and a model deployment for you.
+creates the Foundry resource, a project, and a model deployment for you. This runs from inside
+the repo, so clone it first if you haven't already:
+
+```
+git clone https://github.com/MicrosoftLearning/mslearn-ai-agents.git
+```
 
 1. Install the [Azure Developer CLI](https://learn.microsoft.com/azure/developer/azure-developer-cli/install-azd).
 
-1. From the `Labfiles/D-observe-evaluate-and-secure-agents` folder, run:
+1. From the `Labfiles/D-observe-evaluate-and-secure-agents` folder in the repo you just cloned, run:
 
     ```
     azd auth login
@@ -90,29 +95,6 @@ creates the Foundry resource, a project, and a model deployment for you.
     > connect that in the portal using the steps below. When you're done with the lab, run
     > `azd down` to delete everything it created.
 
-## Connect Application Insights (needed for Task 1)
-
-Foundry stores traces in an **Application Insights** resource connected to your project. Connect
-one now — it takes a minute, and once it's connected Foundry starts recording server-side traces
-for your agents without any code at all.
-
-1. In the [Foundry portal](https://ai.azure.com), open your project.
-
-1. In the left navigation, select **Agents**, then select **Traces** at the top.
-
-1. Select **Connect**, then either pick an existing Application Insights resource or select
-    **Create new** and complete the wizard.
-
-    > If you don't see the **Connect** button, select **Manage** in the upper right, then
-    > **Project details** > **Connected resources** > **Add connection** > **Application Insights**.
-
-1. To *read* the traces you'll need the **Log Analytics Reader** role on that Application
-    Insights resource. If you created it yourself, you already have it.
-
-> **Why this matters**: your Foundry project can only hand your code a connection string if
-> something is connected. Task 1 asks the project for that string, so this step has to happen
-> first.
-
 ## Get the starter code
 
 1. In VS Code, open the Command Palette (**Ctrl+Shift+P**), run **Git: Clone**, and enter:
@@ -121,6 +103,8 @@ for your agents without any code at all.
     https://github.com/MicrosoftLearning/mslearn-ai-agents.git
     ```
 
+    > If you already cloned the repo for the `azd` option above, skip this and just open it.
+
 1. Open the cloned repo, then **File > Open Folder** and select `mslearn-ai-agents/Labfiles/D-observe-evaluate-and-secure-agents/Python`. This single folder holds the starter code for **every** task in this lab — you use one virtual environment and one `.env` throughout.
 
 1. Right-click **requirements.txt** and choose **Open in Integrated Terminal**. Then create a virtual environment and install packages:
@@ -138,9 +122,9 @@ for your agents without any code at all.
 
     > **Tip**: In the Foundry Toolkit VS Code extension, right-click your project deployment and select **Copy Project Endpoint** to get the endpoint URL.
 
-## Get an agent to measure (needed for Tasks 2 and 3)
+## Get an agent to measure (needed for every task)
 
-Tasks 2 and 3 measure a **grounded** agent — one that answers from the Caldova
+Every task uses a **grounded** agent — one that answers from the Caldova
 knowledge base rather than from the model's own memory. You have two ways to get one:
 
 - **You did [Lab B](B-integrate-agents-with-enterprise-knowledge-and-m365.md)**: set `AGENT_NAME`
@@ -160,7 +144,28 @@ knowledge base rather than from the model's own memory. You have two ways to get
     This uploads the documents in `Python/knowledge/`, grounds an agent named
     `caldova-knowledge-agent` on them with File Search, and writes `AGENT_NAME` into your `.env`.
 
-> Task 1 doesn't need this agent — it creates and deletes its own.
+## Connect Application Insights (needed for Task 1)
+
+Foundry stores traces in an **Application Insights** resource connected to your project. Connect
+one now — it takes a minute, and once it's connected Foundry starts recording server-side traces
+for your agents without any code at all.
+
+1. In the [Foundry portal](https://ai.azure.com), open your project.
+
+1. In the left navigation, select **Agents**, then select **Traces** at the top.
+
+1. Select **Connect**, then either pick an existing Application Insights resource or select
+    **Create new** and complete the wizard.
+
+    > If you don't see the **Connect** button, select **Manage** in the upper right, then
+    > **Project details** > **Connected resources** > **Add connection** > **Application Insights**.
+
+1. To *read* the traces you'll need the **Log Analytics Reader** role on that Application
+    Insights resource. If you created it yourself, you already have it.
+
+> **Why this matters**: your Foundry project can only hand your code a connection string if
+> something is connected. Do this before starting Task 1, which asks the project for that
+> string.
 
 ## Check you're ready for a task
 

Instructions/Consolidated/D1-trace-your-agent.md

@@ -21,10 +21,12 @@ lab:
 *Part of the **Observe, evaluate, and secure your agents** lab. New here? Start with [Getting started](D0-getting-started.md).*
 
 > **Set up (start here):** This task needs a Foundry project, an **Application Insights
-> resource connected to it**, and the starter code. If you haven't already, complete
-> [Getting started](D0-getting-started.md) to create your project, connect Application
-> Insights, clone the code, and set `PROJECT_ENDPOINT` and `MODEL_DEPLOYMENT_NAME` in
-> `Python/.env`. Then, from the `Python` folder you opened in VS Code, verify you're ready:
+> resource connected to it**, a grounded agent to trace, and the starter code. If you
+> haven't already, complete [Getting started](D0-getting-started.md) to create your
+> project, clone the code, set `PROJECT_ENDPOINT` and `AGENT_NAME` in `Python/.env` (point
+> it at your [Lab B](B-integrate-agents-with-enterprise-knowledge-and-m365.md) agent, or
+> create one with `python ../setup/bootstrap_agent.py`), and connect Application Insights.
+> Then, from the `Python` folder you opened in VS Code, verify you're ready:
 
 ```
 python ../setup/check_env.py --task 1
@@ -76,8 +78,10 @@ them at a different backend tomorrow without rewriting your instrumentation.
 
 > **Server-side traces come free.** Now that Application Insights is connected to your
 > project, Foundry already records traces for agents it hosts — no code required. What you
-> add here is **client-side** instrumentation: spans around *your* code, so you can see your
-> logic and the agent's work in one timeline.
+> add here is **client-side** instrumentation: spans around *your* code. Both land in the same
+> Application Insights resource, but Foundry's **Agents > Traces** page only renders its own
+> server-side view (`Invoke Agent` / `Execute tool` / `Chat`) — to see your own spans and
+> attributes alongside it, you'll look directly at Application Insights.
 
 Open the `Python` folder and activate the virtual environment from [Getting started](D0-getting-started.md) (`.\labenv\Scripts\Activate.ps1`), then continue below.
 
@@ -93,7 +97,6 @@ Open **traced_agent.py** and add code at each commented placeholder.
     # Add references
     from azure.identity import DefaultAzureCredential
     from azure.ai.projects import AIProjectClient
-    from azure.ai.projects.models import PromptAgentDefinition
     from azure.monitor.opentelemetry import configure_azure_monitor
     from opentelemetry import trace
     ```
@@ -146,23 +149,18 @@ Open **traced_agent.py** and add code at each commented placeholder.
     tracer = trace.get_tracer(__name__)
     ```
 
-1. **Create the agent staff are talking to**:
+1. **Look up the agent** — its `id` (not just its name) is needed to correlate traces with
+    this specific agent in the Foundry portal:
 
     ```python
-    # Create the agent staff are talking to
-    agent = project_client.agents.create_version(
-        agent_name=AGENT_NAME,
-        definition=PromptAgentDefinition(
-            model=model_deployment,
-            instructions=INSTRUCTIONS,
-        ),
-    )
-    print(f"Agent created (name: {agent.name}, version: {agent.version})")
+    # Look up the agent so its id can be included in agent_reference
+    agent = project_client.agents.get(agent_name=agent_name)
     ```
 
 1. **Ask each question inside its own span** — this is the part that pays off. An outer span
     represents the review; each question gets a child span, tagged with attributes you choose
-    so you can tell them apart in the portal:
+    so you can tell them apart in the portal. This reuses `caldova-knowledge-agent` rather
+    than standing up a separate agent just for this task:
 
     ```python
     # Ask each question inside its own span
@@ -176,21 +174,13 @@ Open **traced_agent.py** and add code at each commented placeholder.
                 response = openai_client.responses.create(
                     conversation=conversation.id,
                     input=question,
-                    extra_body={"agent_reference": {"name": agent.name, "type": "agent_reference"}},
+                    extra_body={"agent_reference": {"name": agent.name, "id": agent.id, "type": "agent_reference"}},
                 )
                 question_span.set_attribute("caldova.answer_length", len(response.output_text))
                 print(f"\nQ{number}: {question}")
                 print(f"A{number}: {response.output_text}")
     ```
 
-1. **Clean up the agent version** so you don't leave test agents behind:
-
-    ```python
-    # Clean up resources by deleting the agent version
-    project_client.agents.delete_version(agent_name=agent.name, agent_version=agent.version)
-    print("\nAgent deleted")
-    ```
-
 1. Save the file (**Ctrl+S**).
 
 ### Run and test
@@ -205,32 +195,41 @@ Open **traced_agent.py** and add code at each commented placeholder.
     python traced_agent.py
     ```
 
-1. You should see the three answers print, then the agent delete itself:
+1. You should see the three answers print:
 
     ```
-    Agent created (name: caldova-planning-assistant, version: 1)
-
     Q1: How long does review take for a capacity request with a complete brief?
     A1: ...
-
-    Agent deleted
     ```
 
     > If you get an error about the connection string, Application Insights isn't connected to
     > your project yet — go back to [Getting started](D0-getting-started.md) and connect it.
 
 ### Read the traces
 
+**Two views, two purposes.** Foundry's **Agents > Traces** page shows the automatic
+**server-side** trace of the agent's own turn (`Invoke Agent` > `Execute tool` / `Chat`) — it
+confirms tracing is working, but it doesn't surface the **client-side** spans your script just
+added. To see `morning-planning-review`, `planner-question`, and their custom attributes, look
+at the Application Insights resource itself:
+
 1. In the [Foundry portal](https://ai.azure.com), open your project, select **Agents**, then
-    **Traces**.
+    **caldova-knowledge-agent**, then **Traces**, to confirm the run arrived (telemetry takes a
+    minute or two — wait and refresh if it isn't there yet). Selecting a trace here shows
+    Foundry's own `Invoke Agent` / `Execute tool` / `Chat` view, not your custom spans.
+
+1. Open the Application Insights resource in the [Azure portal](https://portal.azure.com):
+    open the resource group you created for this project, and select the Application Insights
+    resource in it.
 
-1. Find the most recent trace and select it. Telemetry takes a minute or two to arrive — if
-    it isn't there, wait and refresh.
+1. In Application Insights, expand **Investigate** in the left navigation, select **Search**,
+    and search for `morning-planning-review`.
 
-1. Step through the spans. You should see your `morning-planning-review` span at the top with
-    three `planner-question` children, and inside each one the model call the SDK emitted.
+1. Select a matching result to open its **end-to-end transaction details**. This is the raw
+    span tree: your `morning-planning-review` span at the top, three `planner-question`
+    children, and inside each one the model call the SDK emitted.
 
-1. Select a `planner-question` span and look at its attributes. Your `caldova.question_number`
+1. Select a `planner-question` span and look at its properties. Your `caldova.question_number`
     and `caldova.answer_length` are there alongside the standard GenAI attributes.
 
 1. Compare the durations of the three questions. That's the planning lead's complaint,

Instructions/Consolidated/D2-evaluate-answer-quality.md

@@ -33,8 +33,8 @@ python ../setup/check_env.py --task 2
 ```
 
 > **Continuing from a previous task?** If you just finished another task in the same
-> `Python` folder, your project, virtual environment, and `.env` are already set — but Task 1
-> didn't need `AGENT_NAME`, so check that it's set before you start.
+> `Python` folder, your project, virtual environment, and `.env` are already set — go
+> straight to **Look at the dataset first** below.
 
 ---
 
@@ -149,7 +149,7 @@ Open **evaluate_agent.py** and add code at each commented placeholder.
 
 1. **Run the evaluation** — `evaluate()` reads the dataset, calls the target once per row, and
     passes each evaluator exactly the columns it needs. `column_mapping` is how you say which
-    column is which: `${data.x}` comes from the file, `${outputs.x}` comes back from the target:
+    column is which: `${data.x}` comes from the file, `${target.x}` comes back from the target:
 
     ```python
     # Run the evaluation
@@ -166,20 +166,20 @@ Open **evaluate_agent.py** and add code at each commented placeholder.
                 "column_mapping": {
                     "query": "${data.query}",
                     "context": "${data.context}",
-                    "response": "${outputs.response}",
+                    "response": "${target.response}",
                 }
             },
             "relevance": {
                 "column_mapping": {
                     "query": "${data.query}",
-                    "response": "${outputs.response}",
+                    "response": "${target.response}",
                 }
             },
             "similarity": {
                 "column_mapping": {
                     "query": "${data.query}",
                     "ground_truth": "${data.ground_truth}",
-                    "response": "${outputs.response}",
+                    "response": "${target.response}",
                 }
             },
         },

Instructions/Consolidated/D3-red-team-your-agent.md

@@ -112,6 +112,8 @@ Open **red_team_agent.py** and add code at each commented placeholder.
     credential = DefaultAzureCredential()
     project_client = AIProjectClient(endpoint=project_endpoint, credential=credential)
     openai_client = project_client.get_openai_client()
+    # Look up the agent so its id can be included in agent_reference
+    agent = project_client.agents.get(agent_name=agent_name)
     ```
 
 1. **Build the callback that sends one attack prompt to your agent** — the red team calls this
@@ -125,7 +127,7 @@ Open **red_team_agent.py** and add code at each commented placeholder.
         try:
             response = openai_client.responses.create(
                 input=query,
-                extra_body={"agent_reference": {"name": agent_name, "type": "agent_reference"}},
+                extra_body={"agent_reference": {"name": agent.name, "id": agent.id, "type": "agent_reference"}},
             )
             return response.output_text
         except Exception as error:  # a blocked prompt is a result, not a crash
@@ -173,7 +175,7 @@ Open **red_team_agent.py** and add code at each commented placeholder.
                 AttackStrategy.Flip,
                 AttackStrategy.Compose([AttackStrategy.Base64, AttackStrategy.ROT13]),
             ],
-            output_path=str(OUTPUT),
+            output_path=str(OUTPUT_DIR),
         )
     ```
 
@@ -219,9 +221,10 @@ Open **red_team_agent.py** and add code at each commented placeholder.
     ]
     ```
 
-1. Open **redteam_scan.json** and look at `redteaming_data`. Every attack-response pair is
-    there, with the prompt that was sent, what your agent said, and whether the attack was
-    judged successful. Read a few — seeing the disguised prompts is most of the value.
+1. Open **redteam_output/evaluation_result.json** and look at `redteaming_data`. Every
+    attack-response pair is there, with the prompt that was sent, what your agent said, and
+    whether the attack was judged successful. Read a few — seeing the disguised prompts is
+    most of the value.
 
 1. Compare `baseline_asr` with the per-complexity numbers in `attack_technique_summary`. If a
     strategy scores higher than the baseline, that transformation is getting past something the