LLM-Human-LLM Interrogation

For complex tasks where the initial instructions or data might be ambiguous, you can implement a multi-turn "interrogation" loop. In this pattern, the agent can choose to either provide a final result or request clarification from the human.

Pattern: The Interrogation Loop

This pattern feeds user responses back into the agent's message history until a final verdict is reached.

1. Define the Unified Output Model

Create a Pydantic model that allows the agent to return either the final result or a request for more information.

from pydantic import BaseModel, Field
from typing import Literal

class ClarificationRequest(BaseModel):
    """The agent needs more info before it can proceed."""
    type: Literal["clarification"] = "clarification"
    question: str = Field(..., description="The question to ask the user.")
    options: list[str] | None = Field(
        None, 
        description="Suggested labels for the user to choose from."
    )
    context: str | None = Field(
        None, 
        description="Extra context to show the user above the question."
    )

class FinalAnalysis(BaseModel):
    """The agent has enough info to provide the final result."""
    type: Literal["final"] = "final"
    verdict: str
    confidence: float

# The agent will return one of these two types
AgentOutput = ClarificationRequest | FinalAnalysis

2. Configure the Agent

Use build_agent_from_settings with your union type as the output_type. This returns an AgentRunner, which supports fallback chains automatically.

from aifred_tk.core.llm import build_agent_from_settings, AgentRunner

self._agent: AgentRunner[AgentOutput] = build_agent_from_settings(
    self.tool_id,
    self._settings,
    output_type=AgentOutput,
    system_prompt="""
    You are a requirements analyst. 
    Analyze the user's request. 
    If anything is ambiguous, return a ClarificationRequest.
    If you have everything you need, return a FinalAnalysis.
    """
)

3. Implement the Loop

The execute method runs a loop that feeds user responses back into the agent's message history.

from aifred_tk.elicitation import Choice, elicit_choice
from aifred_tk.core.models import ElicitationCancelledError
from aifred_tk.core.interfaces import ToolResult
from pydantic_ai.messages import ModelMessage

def execute(self, args: ToolArgs) -> ToolResult:
    agent = self._get_agent()
    user_prompt = f"Analyze this: {args.input_text}"
    message_history: list[ModelMessage] = []

    while True:
        # 1. Run the agent
        result = agent.run_sync(user_prompt, message_history=message_history)
        output = result.output

        # Update history for the next turn
        message_history = list(result.all_messages())

        # 2. Check if it's a final result
        if isinstance(output, FinalAnalysis):
            return ToolResult(
                output={"verdict": output.verdict, "confidence": output.confidence},
                llm_name=agent.model_name
            )

        # 3. Otherwise, it's a clarification request. Elicit info from the human.
        choices = [Choice(opt) for opt in (output.options or [])]

        try:
            elicit_result = elicit_choice(
                question=output.question,
                choices=choices,
                context=output.context,
                allow_free_input=True
            )
        except ElicitationCancelledError:
            return ToolResult(output={"status": "error", "message": "User cancelled the interrogation."})

        # 4. Feed the human's answer back as the next user prompt
        if elicit_result.is_free_input:
            user_prompt = elicit_result.free_text
        else:
            user_prompt = f"User selected: {elicit_result.selected_label}"

Handling Cancellations

In MCP environments, a user might close the dialog or explicitly hit "Cancel". This raises an ElicitationCancelledError. Your tool should handle this gracefully (as shown in the loop example above).