LLM Probe Plugin¶

The llm_probe plugin provides the probe_llm tool. It forwards a system prompt and a user prompt directly to the configured LLM and returns the raw text response. Intended for iterative prompt engineering and parameter tuning before embedding prompts in higher-level tools.

Quick start¶

Ensure your settings declare an LLM and point the tool at it:

# ~/.config/aifred-tk/settings.yml
llms:
  my-llm:
    provider: openai
    model: gpt-4o-mini

tools:
  probe_llm:
    llm:
      type: ref
      ref: my-llm

Then invoke the tool:

aifred-tk probe_llm --system-prompt "You are a helpful assistant." --user-prompt "What is 2+2?"

Configuration¶

LLM setting¶

The tools.probe_llm.llm key is required. It accepts a reference to a named LLM or an inline definition. See LLMs for the full format.

# Reference a named LLM
tools:
  probe_llm:
    llm:
      type: ref
      ref: my-llm

# Or define inline
tools:
  probe_llm:
    llm:
      type: custom
      provider: ollama
      model: gemma4:e4b
      host: 127.0.0.1
      port: 11434

Enabling and disabling¶

The tool respects the standard enabled flag:

tools:
  probe_llm:
    enabled: false

Usage¶

CLI¶

aifred-tk probe_llm --system-prompt TEXT --user-prompt TEXT [--temperature FLOAT] [--max-tokens INT]

Option	Description
`--system-prompt TEXT`	System prompt sent to the LLM. Required.
`--user-prompt TEXT`	User prompt sent to the LLM. Required.
`--temperature FLOAT`	Sampling temperature in the range [0.0, 2.0]. Uses the provider default when omitted.
`--max-tokens INT`	Maximum number of tokens in the response. Uses the provider default when omitted.

MCP¶

The tool is registered automatically when the MCP server starts. Invoke it as aifred_probe_llm with the required system_prompt and user_prompt arguments, and optional temperature and max_tokens arguments.

Output¶

Success¶

{"response": "4"}

Error¶

{"status": "error", "message": "Agent error: <provider error details>"}

Notes¶

A fresh agent is created on every call; the system prompt is supplied at runtime by the caller and is not baked into the plugin.
temperature and max_tokens are forwarded to the model only when explicitly provided; omitting them lets the provider apply its own defaults.