Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
110 changes: 110 additions & 0 deletions 10.0/AI/LocalChatClientWithAgents/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
---
name: .NET MAUI - Local Chat Client with Agents
description: AI Travel Planner using on-device Apple Intelligence with the Microsoft Agent Framework, Microsoft.Extensions.AI, and .NET MAUI.
page_type: sample
languages:
- csharp
- xaml
products:
- dotnet-maui
urlFragment: local-chat-client-with-agents
---

# LocalChatClientWithAgents (MAUI + Apple Intelligence + Agent Framework)

A .NET MAUI sample that demonstrates a multi-agent AI Travel Planner powered entirely by on-device Apple Intelligence. It uses the Microsoft Agent Framework to orchestrate a pipeline of specialized agents — from parsing user intent, to RAG-based destination matching, to streaming itinerary generation with tool calling, to conditional translation — all running locally on iOS and macCatalyst.

| Landmarks | Trip Planning | Itinerary | Chat |
|:-:|:-:|:-:|:-:|
| ![landmarks](images/landmarks.png) | ![trip planning](images/trip_planning.png) | ![itinerary](images/itinerary.png) | ![chat](images/chat.png) |

## What you'll learn

- How to register and consume `IChatClient` (Microsoft.Extensions.AI) backed by Apple Intelligence in a MAUI app
- How to build a multi-agent workflow using the Microsoft Agent Framework (`Microsoft.Agents.AI`)
- How to use Retrieval-Augmented Generation (RAG) with on-device `NLEmbeddingGenerator` for semantic search
- How to implement streaming JSON deserialization for progressive UI updates
- How to use tool calling with agents to discover points of interest
- How to conditionally route agents (e.g., skip translation when the language is English)

## Prerequisites

- .NET 10 SDK (preview)
- macOS with Xcode 26 beta (for Apple Intelligence / FoundationModels framework)
- An Apple device or simulator running iOS 26+ or macOS 26+ with Apple Intelligence enabled

> **Note:** This sample only runs on iOS and macCatalyst. Android and Windows will throw `PlatformNotSupportedException` at startup.

## Architecture

The app uses a 4-agent pipeline to generate travel itineraries:

```
User Input → [Travel Planner] → [Researcher] → [Itinerary Planner] → [Translator?] → Output
```

| Agent | Purpose | Key Feature |
|-------|---------|-------------|
| **Travel Planner** | Extracts destination, duration, and language from natural language | NLP intent parsing |
| **Researcher** | Matches user's destination against a local landmark database | RAG with NL embeddings |
| **Itinerary Planner** | Generates a multi-day itinerary with real places | Tool calling + streaming JSON |
| **Translator** | Translates the itinerary if a non-English language was requested | Conditional routing |

## Key files

- `MauiProgram.cs` — Registers Apple Intelligence as `IChatClient` (keyed as `local-model` and `cloud-model`), plus `NLEmbeddingGenerator` for embeddings. Throws `PlatformNotSupportedException` on non-Apple platforms.
- `AI/ItineraryWorkflowExtensions.cs` — Configures the 4-agent workflow graph with conditional branching for translation.
- `AI/ItineraryWorkflowTools.cs` — Provides RAG search and `findPointsOfInterest` tool for the agents.
- `Services/ItineraryService.cs` — Orchestrates streaming itinerary generation with progressive JSON deserialization.
- `Services/DataService.cs` — Manages a local landmark database with semantic embedding search.
- `Services/ChatService.cs` — Chat assistant with tool calling for landmark search, trip planning, and weather.
- `Pages/LandmarksPage.xaml` — Browse world landmarks with semantic search.
- `Pages/TripPlanningPage.xaml` — Configure trip duration and language for a selected landmark.
- `Pages/ItineraryPage.xaml` — Stream and display an AI-generated multi-day itinerary.

## Sample prompts

Try these in the 💬 chat:

- "Show me landmarks in Oceania"
- "Plan a 3-day trip to the Grand Canyon"
- "What can I do in Maui?"
- "Search for desert landmarks"
- "Plan a trip to Mount Fuji in Japanese"

## Run

Build and run on an iOS 26+ simulator/device or macCatalyst target:

```bash
dotnet build -t:Run -f net10.0-maccatalyst
```

Or open `LocalChatClientWithAgents.slnx` in Visual Studio / VS Code and select an iOS or Mac Catalyst target.

Once running, browse landmarks on the home page, tap a destination to see details, configure duration and language, then tap **Generate Itinerary** to stream an AI-generated travel plan. You can also use the 💬 chat button to ask the assistant to plan trips, search destinations, check weather, and more.

## NuGet feed configuration

This sample requires the .NET 10 preview NuGet feed for `Microsoft.Maui.Essentials.AI`. The included `NuGet.config` adds:

```
https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet10/nuget/v3/index.json
```

## Useful docs and resources

- Microsoft.Extensions.AI overview — [learn.microsoft.com/dotnet/ai/microsoft-extensions-ai](https://learn.microsoft.com/dotnet/ai/microsoft-extensions-ai)
- Apple Intelligence in .NET MAUI — [learn.microsoft.com/dotnet/maui/platform-integration/communication/apple-intelligence](https://learn.microsoft.com/dotnet/maui/platform-integration/communication/apple-intelligence)
- Microsoft Agent Framework — [github.com/microsoft/agents](https://github.com/microsoft/agents)
- NuGet packages:
- [Microsoft.Extensions.AI](https://www.nuget.org/packages/Microsoft.Extensions.AI)
- [Microsoft.Agents.AI](https://www.nuget.org/packages/Microsoft.Agents.AI)
- [CommunityToolkit.Mvvm](https://learn.microsoft.com/dotnet/communitytoolkit/mvvm/)

## Notes

- This sample uses **only on-device Apple Intelligence** — no cloud API keys or endpoints are required.
- The landmark database is embedded as JSON in `Resources/Raw/` and searched using Apple's Natural Language embeddings.
- The itinerary is streamed progressively using a zero-allocation JSON deserializer for smooth UI updates.
- Translation is conditional — if the user requests English, the translator agent is skipped entirely.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
using Microsoft.Agents.AI;
using Microsoft.Agents.AI.Workflows;
using Microsoft.Extensions.AI;
using Microsoft.Extensions.Logging;

namespace LocalChatClientWithAgents.AI;

/// <summary>
/// Agent 1: Travel Planner - Parses natural language to extract intent.
/// No tools - just NLP to extract destinationName, dayCount, language.
/// </summary>
internal sealed class TravelPlannerExecutor(AIAgent agent, ILogger logger)
: ChatProtocolExecutor("TravelPlannerExecutor", new ChatProtocolExecutorOptions { AutoSendTurnToken = false })
{
/// <summary>
/// Declares TravelPlanResult as a sent message type so the edge router can map it to downstream executors.
/// Without this, ChatProtocolExecutor only declares List&lt;ChatMessage&gt; and TurnToken, causing
/// TravelPlanResult to be silently dropped with DroppedTypeMismatch.
/// </summary>
protected override ProtocolBuilder ConfigureProtocol(ProtocolBuilder protocolBuilder)
=> base.ConfigureProtocol(protocolBuilder).SendsMessage<TravelPlanResult>();

protected override async ValueTask TakeTurnAsync(
List<ChatMessage> messages,
IWorkflowContext context,
bool? emitEvents = null,
CancellationToken cancellationToken = default)
{
logger.LogDebug("[TravelPlannerExecutor] Starting - parsing user intent");

await context.AddEventAsync(new ExecutorStatusEvent("Analyzing your request..."), cancellationToken);

var response = await agent.RunAsync<TravelPlanResult>(messages, cancellationToken: cancellationToken);

logger.LogTrace("[TravelPlannerExecutor] Raw response: {Response}", response.Text);

var result = response.Result;

logger.LogDebug("[TravelPlannerExecutor] Completed - extracted: destination={Destination}, days={Days}, language={Language}",
result.DestinationName, result.DayCount, result.Language);

var summary = result.Language != "English"
? $"Planning {result.DayCount}-day trip to {result.DestinationName} in {result.Language}"
: $"Planning {result.DayCount}-day trip to {result.DestinationName}";
await context.AddEventAsync(new ExecutorStatusEvent(summary), cancellationToken);

await context.SendMessageAsync(result, cancellationToken);
}
}
56 changes: 56 additions & 0 deletions 10.0/AI/LocalChatClientWithAgents/src/AI/2_ResearcherExecutor.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
using Microsoft.Agents.AI;
using Microsoft.Agents.AI.Workflows;
using Microsoft.Extensions.Logging;

namespace LocalChatClientWithAgents.AI;

/// <summary>
/// Agent 2: Researcher - Uses TextSearchProvider (RAG) to automatically inject matching destinations
/// into the AI context before each invocation, then the AI selects the best match.
/// The TextSearchProvider is configured in with BeforeAIInvoke mode, so candidate destinations are
/// automatically searched and injected.
/// </summary>
internal sealed partial class ResearcherExecutor(AIAgent agent, ILogger logger)
: Executor("ResearcherExecutor")
{
[MessageHandler]
private async ValueTask<ResearchResult> HandleAsync(
TravelPlanResult input,
IWorkflowContext context,
CancellationToken cancellationToken = default)
{
logger.LogDebug("[ResearcherExecutor] Starting - finding best matching destination for '{DestinationName}'", input.DestinationName);
logger.LogTrace("[ResearcherExecutor] Input: {@Input}", input);

await context.AddEventAsync(new ExecutorStatusEvent("Searching destinations..."), cancellationToken);

// TextSearchProvider (configured via CreateAgent) automatically searches
// DataService.SearchLandmarksAsync and injects results as context before
// the AI call. We just need to ask the AI to pick the best match.
var prompt = input.DestinationName;

logger.LogTrace("[ResearcherExecutor] Prompt: {Prompt}", prompt);

var response = await agent.RunAsync<DestinationMatchResult>(prompt, cancellationToken: cancellationToken);

logger.LogTrace("[ResearcherExecutor] Raw response: {Response}", response.Text);

// Parse the AI's response — both name and description come from RAG context
var matchResult = response.Result;

logger.LogDebug("[ResearcherExecutor] AI selected '{MatchedName}'", matchResult.MatchedDestinationName);

var result = new ResearchResult(
matchResult.MatchedDestinationName,
matchResult.MatchedDestinationDescription,
input.DayCount,
input.Language);

logger.LogDebug("[ResearcherExecutor] Completed - selected destination: {Name}", matchResult.MatchedDestinationName);
logger.LogTrace("[ResearcherExecutor] Output: {@Result}", result);

await context.AddEventAsync(new ExecutorStatusEvent($"Found destination: {matchResult.MatchedDestinationName}"), cancellationToken);

return result;
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
using System.Text;
using Microsoft.Agents.AI;
using Microsoft.Agents.AI.Workflows;
using Microsoft.Extensions.AI;
using Microsoft.Extensions.Logging;

namespace LocalChatClientWithAgents.AI;

/// <summary>
/// Agent 3: Itinerary Planner - Builds the travel itinerary with streaming output.
/// Tools are used to assist in generating the itinerary.
/// Uses RunStreamingAsync to emit partial JSON as it's generated.
/// </summary>
internal sealed partial class ItineraryPlannerExecutor(AIAgent agent, ILogger logger)
: Executor("ItineraryPlannerExecutor")
{
[MessageHandler]
private async ValueTask<ItineraryResult> HandleAsync(
ResearchResult input,
IWorkflowContext context,
CancellationToken cancellationToken = default)
{
logger.LogDebug("[ItineraryPlannerExecutor] Starting - building {Days}-day itinerary for '{Destination}'",
input.DayCount, input.DestinationName ?? "unknown");
logger.LogTrace("[ItineraryPlannerExecutor] Input: {@Input}", input);

await context.AddEventAsync(new ExecutorStatusEvent("Building your itinerary..."), cancellationToken);

if (input.DestinationName is null)
{
logger.LogDebug("[ItineraryPlannerExecutor] No destination found - returning error");
await context.AddEventAsync(new ExecutorStatusEvent("Error: No destination found"), cancellationToken);
return new ItineraryResult(System.Text.Json.JsonSerializer.Serialize(new { error = "Destination not found" }), input.Language);
}

var prompt = $"""
Generate a {input.DayCount}-day itinerary to {input.DestinationName}.
Destination description: {input.DestinationDescription}
""";

logger.LogTrace("[ItineraryPlannerExecutor] Prompt: {Prompt}", prompt);

// Use streaming to emit partial JSON as it's generated
// Tools and ResponseFormat are configured at agent level in ItineraryWorkflowExtensions
var fullResponse = new StringBuilder();
await foreach (var update in agent.RunStreamingAsync(prompt, cancellationToken: cancellationToken))
{
foreach (var content in update.Contents)
{
if (content is not TextContent textContent)
continue;

fullResponse.Append(textContent.Text);

await context.AddEventAsync(new ItineraryTextChunkEvent(Id, textContent.Text), cancellationToken);
}
}
var responseText = fullResponse.ToString();

logger.LogTrace("[ItineraryPlannerExecutor] Raw response: {Response}", responseText);
logger.LogDebug("[ItineraryPlannerExecutor] Completed - itinerary generated, language: {Language}", input.Language);

await context.AddEventAsync(new ExecutorStatusEvent($"Created {input.DayCount}-day itinerary for {input.DestinationName}"), cancellationToken);

return new ItineraryResult(responseText, input.Language);
}
}
60 changes: 60 additions & 0 deletions 10.0/AI/LocalChatClientWithAgents/src/AI/4_TranslatorExecutor.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
using System.Text;
using Microsoft.Agents.AI;
using Microsoft.Agents.AI.Workflows;
using Microsoft.Extensions.AI;
using Microsoft.Extensions.Logging;

namespace LocalChatClientWithAgents.AI;

/// <summary>
/// Agent 4: Translator - Translates the itinerary to target language (conditional) with streaming.
/// No tools - just translation.
/// Uses RunStreamingAsync to emit partial translated JSON.
/// </summary>
internal sealed partial class TranslatorExecutor(AIAgent agent, ILogger logger)
: Executor("TranslatorExecutor")
{
[MessageHandler]
private async ValueTask<ItineraryResult> HandleAsync(
ItineraryResult input,
IWorkflowContext context,
CancellationToken cancellationToken = default)
{
logger.LogDebug("[TranslatorExecutor] Starting - translating to '{Language}'", input.TargetLanguage);
logger.LogTrace("[TranslatorExecutor] Input JSON: {Json}", input.ItineraryJson);

await context.AddEventAsync(new ExecutorStatusEvent($"Translating to {input.TargetLanguage}..."), cancellationToken);

var prompt = $"""
Translate to {input.TargetLanguage}:

{input.ItineraryJson}
""";

logger.LogTrace("[TranslatorExecutor] Prompt: {Prompt}", prompt);

// Use streaming to emit partial JSON as it's generated
// ResponseFormat is set at agent creation time in ItineraryWorkflowExtensions
var fullResponse = new StringBuilder();
await foreach (var update in agent.RunStreamingAsync(prompt, cancellationToken: cancellationToken))
{
foreach (var content in update.Contents)
{
if (content is not TextContent textContent)
continue;

fullResponse.Append(textContent.Text);

await context.AddEventAsync(new ItineraryTextChunkEvent(Id, textContent.Text), cancellationToken);
}
}
var responseText = fullResponse.ToString();

logger.LogTrace("[TranslatorExecutor] Raw response: {Response}", responseText);
logger.LogDebug("[TranslatorExecutor] Completed - translation to '{Language}' finished", input.TargetLanguage);

await context.AddEventAsync(new ExecutorStatusEvent($"Translated to {input.TargetLanguage}"), cancellationToken);

return new ItineraryResult(responseText, input.TargetLanguage);
}
}
27 changes: 27 additions & 0 deletions 10.0/AI/LocalChatClientWithAgents/src/AI/5_OutputExecutor.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
using Microsoft.Agents.AI.Workflows;
using Microsoft.Extensions.Logging;

namespace LocalChatClientWithAgents.AI;

/// <summary>
/// Final executor that marks the workflow as complete.
/// The itinerary JSON has already been streamed by ItineraryPlannerExecutor or TranslatorExecutor.
/// </summary>
internal sealed partial class OutputExecutor(ILogger logger)
: Executor("OutputExecutor")
{
[MessageHandler]
private async ValueTask HandleAsync(
ItineraryResult input,
IWorkflowContext context,
CancellationToken cancellationToken = default)
{
logger.LogDebug("[OutputExecutor] Starting - finalizing itinerary (language: {Language})", input.TargetLanguage);
logger.LogTrace("[OutputExecutor] Final JSON: {Json}", input.ItineraryJson);

// Don't re-emit the JSON - it was already streamed by ItineraryPlannerExecutor or TranslatorExecutor
await context.AddEventAsync(new ExecutorStatusEvent("Your itinerary is ready!"), cancellationToken);

logger.LogDebug("[OutputExecutor] Completed - workflow finished");
}
}
Loading
Loading