How to add human interaction into workflow?

[Tennyleaz]

I was using a single AgentThread and without workflow, I could add a new ChatMessage from user anytime.

Now I want to create a Workflow and add human/external interaction to it. How do I do exactly? Using an InputPort and add another ChatMessage to the workflow?
For example, in a sample scenario a sequential workflow could be:

1. An agent looks for products
2. A human approves the found product (add a new user message to workflow?)
3. Another agent orders the product

ChatClientAgent productSearchAgent = ...
ChatClientAgent productOrderAgent = ...
InputPort humanInputPort = InputPort.Create<string, string>("Human-approve");

WorkflowBuilder workflowBuilder = new WorkflowBuilder(productSearchAgent);
workflowBuilder.AddEdge(productSearchAgent, humanInputPort);
workflowBuilder.AddEdge(humanInputPort, productOrderAgent);
Workflow workflow = workflowBuilder.Build();

List<ChatMessage> messages = new List<ChatMessage> { new(ChatRole.System, "Search for given product in a list.") };
StreamingRun run = await InProcessExecution.StreamAsync(workflow, messages);
await run.TrySendMessageAsync(new TurnToken(emitEvents: true));

List<ChatMessage> result = new();
await foreach (WorkflowEvent e in run.WatchStreamAsync().ConfigureAwait(false))
{
    if (e is RequestInfoEvent requestInputEvt)
    {
        // Request for user input
    }
}

This code throws exception like:

InvalidOperationException: Message type System.Collections.Generic.List`1[[Microsoft.Extensions.AI.ChatMessage, Microsoft.Extensions.AI.Abstractions, ...]]... could not be interpreted as a value of Request Type System.String

It seems there isn't much documentation about how could we get to this?
Any help is appreciated.

[ekzhu]

I think you need to create a custom executor that handles the productSearchAgent's response, in that handler, sends a string message to the human input port for approval; then a separate handler that handles the string message from human input port, and depending on the approval status, sends a new message to the productOrderAgent.

So it looks something like this:

productSearchAgent --> customAgent
customAgent --> humanInputPort
humanInputPort --> customAgent
customAgent --> productOrderAgent

[KINGGUOKUN]

What can I do for a workflow with 3 nodes A, B, C , B and C need approval of different human ?

[jefhughe]

That is what I am seeing as a major flaw for workflows and checkPoints. I would expect B and C to have tools that can interact with some ticketing system where the tool would create the ticket and the checkPoint state could be cached on say redis or cosmos durable cache. Once the human authorizes an event is fired and resumes the workflow and re-inflates the checkPoint to continue to the next agent defined in the workflow.

[lokitoth]

What can I do for a workflow with 3 nodes A, B, C , B and C need approval of different human ?

Not quite sure what you mean, but my suggestion would be to have the request object contain the destination (or alternatively the category of request); the host of the workflow, on receiving the ExternalRequest object containing the request will then route it to the appropriate human.

[lokitoth]

That is what I am seeing as a major flaw for workflows and checkPoints. I would expect B and C to have tools that can interact with some ticketing system where the tool would create the ticket and the checkPoint state could be cached on say redis or cosmos durable cache. Once the human authorizes an event is fired and resumes the workflow and re-inflates the checkPoint to continue to the next agent defined in the workflow.

I believe this is possible to do, but the management of the checkpointing (e.g. supplying a CheckpointManager), then resuming the workflow from the right checkpoint would be the responsibility of the layer running the workflow. You would need to send some input into the Workflow, and if you want that input to be enter somewhere other than the starting Executor you will need to use the ExternalRequest mechanism. For now, this requires an InputPort, but there is work to relax that requirement and allow Executor instances to send the appropriate request directly; Python recently implemented a similar change, though for .NET we are going to keep InputPort around even after the refactor.

The only tricky bit is that you will need to ensure that you get the right CheckpointId, since the Executors will run (and invoke the Tool calls) before the checkpoint is made. One possible improvement here is that we can generate the future CheckpointId earlier in the step, and make it available on the SuperStepStartedEvent, so you can associate it with the ticketing system to restart. Unfortunately, that also means that you will need to deal with the case where the step fails after the tickets have been created, since that CheckpointId will never correspond to a real checkpoint.

[lokitoth]

Hi @Tennyleaz,

Sorry for missing your question earlier. In this case, the output from the Agent Executor in the workflow is going to be the set of ChatMessages that were generated by it as part of running it. The reason the InputPort executor is not playing nice with it is that it expects a string as a request.

In the case of Agent, it follows a "Turn Taking Chat Protocol" (this is a gap in the documentation as you noticed) where-in the expectation is that the Executor will accumulate incoming ChatMessages (via at least List), followed by taking its turn when a TurnToken (note that this is a difference against Python, though there is work to bridge that gap that should be landing soon, among with better support for your use case too). The easiest way to support that is to use the ChatProtocolExecutor base class, which takes care of most of this for you. All you need to implement is TakeTurnAsync. From there, you would have access to the AIContent which represents, for example, a FunctionApprovalRequestContent (see here) if you have a tool that is set up to require approval, or you can process ChatMessages into the request type you want.

The other thing you can do is switch the request port to take List<ChatMessage>, and rely on the ForwardMessage<T>() (link) or ForwardExcept<T> (link) extension methods to ensure that only the List<ChatMessage> gets passed and the TurnToken does not.

A related, but separate thing I would recommend is to make the approval message be more structured, even if you intend to let the user offer text feedback as well. This makes it easier to process the result (and either feed it back to the first agent in case of a rejection or text feedback or forward it to the buying agent in case of an approval).