---

name: agents-and-llm-orchestration description: "Build txtai LLM, RAG, and Agent workflows with backend selection, tools, templates, teams, and troubleshooting guidance." disable-model-invocation: true metadata: disco-role: operating license: Apache 2.0

Agents and LLM Orchestration

Use this sub-skill when the task is to connect txtai generation, retrieval, and agentic tool use. It covers LLM, RAG, and Agent construction, YAML configuration, tool definitions, prompt templates, agent teams, backend selection, and optional dependency troubleshooting.

For low-level embeddings store creation and query tuning, route to embeddings-search. For deterministic pipeline/workflow task chains, route to pipelines-and-workflows. For serving agents/RAG over API, OpenAI-compatible endpoints, or hosted app deployment, route to api-and-deployment.

Quick Routing

User goal	Use
Generate text directly	LLM(path=None, method=None, **kwargs)
Answer with retrieved context	RAG(similarity, path, template=..., context=..., output=...)
Let a model choose tools over multiple steps	Agent(model=..., tools=[...], max_steps=...)
Build multi-agent research/delegation	Agents as tool targets inside another Agent
Define via YAML/Application	top-level llm:, rag:, and agent: sections
Validate RAG config safely	python scripts/rag_config_template.py --write-template rag_template.py

Minimal Import Check

from txtai import Agent, Embeddings, LLM, RAG

print(LLM)
print(RAG)
print(Agent)

If constructing Agent(...) raises ImportError: smolagents is not available - install "agent" extra to enable, install an agent-capable environment such as pip install "txtai[agent]". If LLM backends fail, see backend-and-tooling.

Direct LLM Pattern

from txtai import LLM

llm = LLM("ibm-granite/granite-4.0-350m")
answer = llm("Summarize retrieval augmented generation", maxlength=256)

LLM accepts a model path, optional method, and backend keyword arguments. It supports strings, lists of strings, chat-message dictionaries with role and content, streaming via stream=True, stop strings, defaultrole, and stripthink cleanup. Use method="transformers", method="llama.cpp", method="litellm", or a custom generation class path when inference cannot be inferred from the model path.

RAG Pattern

from txtai import Embeddings, RAG

embeddings = Embeddings(content=True)
embeddings.index([
    ("install", "txtai supports semantic search, RAG, agents, workflows and APIs"),
    ("agent", "txtai agents use tools such as Python functions, embeddings stores and web search"),
])

template = """
Answer using only the context.
Question: {question}
Context: {context}
"""

rag = RAG(
    embeddings,
    "ibm-granite/granite-4.0-350m",
    template=template,
    system="You answer from retrieved context.",
    context=3,
    output="flatten",
)

result = rag("What can txtai agents use?", maxlength=256)

RAG joins a similarity source with a generator or question-answering pipeline. The similarity source can be an Embeddings instance, a Similarity pipeline, a callable batch searcher, or an Application-provided embeddings index. Build and persist retrieval stores with embeddings-search before focusing on prompt generation here.

Agent Pattern

from datetime import datetime
from txtai import Agent

def today() -> str:
    """
    Gets the current date.

    Returns:
        current date in ISO format
    """
    return datetime.today().isoformat()

agent = Agent(
    model="Qwen/Qwen3-4B-Instruct-2507",
    tools=[today, "websearch"],
    max_steps=5,
)

answer = agent("What is today's date and why might it matter for a news search?", maxlength=2048)

Use agents for multi-step questions where tool choice matters. Prefer plain RAG or deterministic workflows for simple, repeatable flows.

Tool Shapes

• Direct Python callable: add the function or callable instance to tools; type hints and Google-style docstrings improve generated tool schemas.
• Explicit function dictionary: {"name": "lookup", "description": "...", "inputs": {...}, "target": callable}.
• Embeddings dictionary: {"name": "kb", "description": "Searches the knowledge base", "path": "index-path"} or {"target": embeddings}; results are lists of dicts with id, text, and score when content is available.
• Built-in shortcuts: bash, defaults, edit, glob, grep, python, read, todowrite, websearch, webview, and write.
• MCP tools: HTTP strings such as http://localhost:8000/mcp load a Model Context Protocol tool collection.
• Skill files: strings ending in .md load a skill.md-style tool with frontmatter name and description.

Agent Teams

Agents can be tool targets for a supervisor agent.

from txtai import Agent, LLM

llm = LLM("Qwen/Qwen3-4B-Instruct-2507")
websearcher = Agent(model=llm, tools=["websearch"], max_steps=3)

supervisor = Agent(
    model=llm,
    tools=[{
        "name": "websearcher",
        "description": "Searches the web and returns concise findings",
        "target": websearcher,
    }],
    max_steps=6,
)

Share a single LLM instance when team members should use the same backend configuration. Keep each agent tool description narrow so the supervisor can choose correctly.

YAML/Application Pattern

llm:
  path: Qwen/Qwen3-4B-Instruct-2507

agent:
  researcher:
    max_steps: 5
    tools:
      - websearch
      - name: today
        description: Gets the current date
        target: package.module.today

rag:
  path: ibm-granite/granite-4.0-350m
  template: |
    Answer using only the context.
    Question: {question}
    Context: {context}
  output: flatten
  context: 3

Application(config) resolves configured pipelines first, then agents, then embeddings. A configured rag: pipeline can receive the Application embeddings index when its similarity is omitted. In agent: sections, target values are resolved as configured pipelines, workflows, or importable callables.

RAG Output Modes

output	Shape
default	(name, answer) tuples, or dicts with answer for dict/string inputs
flatten	answer strings only
reference	(name, answer, reference) tuples or dicts with references

RAG input can be a string, a list of strings, tuples shaped like (name, query, question, snippet), or dict rows with name, query, question, and snippet. Use output="flatten" for simple chat answers and output="reference" when the caller needs source ids from retrieved context.

References

• LLM, RAG, and Agent recipes for concrete Python and YAML patterns.
• Backend and tooling for extras, credentials, backends, and tool schemas.
• Troubleshooting for placeholder agents, template bugs, retrieval quality, output shapes, web/network failures, and CPU/GPU issues.

Validation Checklist

• Confirm imports and constructor signatures for LLM, RAG, and Agent in the target environment.
• For RAG, ensure the retrieval store is content-enabled and the prompt template includes both {question} and {context}.
• For agents, verify optional extras by instantiating a tiny Agent(model=..., tools=[], max_steps=1) or by importing smolagents directly.
• For hosted LLMs, verify credentials outside source files and run a one-prompt smoke test before wiring tools.
• Keep source repository docs, examples, and absolute checkout paths out of generated runtime content.