---

name: dd-instrument-llmo description: Instrument the current project with Datadog LLM Observability for Python or Node.js/Next.js backends that call LLMs or run AI agents. Detects the runtime and LLM framework, provisions credentials, adds SDK init (ddtrace/dd-trace) with the correct kwargs, persists the dependency into the deploy manifest, and audits session-ID plumbing for gaps — fixing them when found. Use when the user says "instrument this project with LLM Observability", "add LLM Observability", "monitor my AI app in Datadog", "add LLM spans", "add agent session tracking", or "verify/repair my LLM Observability setup". metadata: version: "0.1.0" author: datadog-labs repository: https://github.com/datadog-labs/agent-skills tags: datadog,llm-observability,llmobs,instrumentation,python,nodejs,nextjs,ddtrace,dd-trace,agents alwaysApply: "false"

Datadog LLM Observability Instrumentation

This skill assesses the current project (backend runtime, LLM/agent framework, existing instrumentation) and routes you to the right reference file under references/ for the actual setup steps. It is fully self-contained — it does not call any Datadog MCP server. Credential provisioning uses the local CreateApiKey tool, and all code changes are made by you, directly, using your own file-editing tools.

Stay within LLM Observability scope. Do not add RUM, APM application instrumentation, or unrelated Datadog products. RUM and a Datadog Agent are relevant only because they change what session/trace linking is achievable (see the "Beyond SDK init" section in the reference files).

Do NOT invent tool names. Use only CreateApiKey as described in references/common-credentials.md; every other step is done with your normal file-editing/search tools.

Do NOT write anything to memory during or after this skill. Project paths, frameworks, credentials, and ML app names are project-specific and must not be stored in persistent memory.

Ground rules while instrumenting

• Verify before you assert. If you're not sure about file content or codebase structure, read the files — do not guess.
• Check for existing instrumentation first. Before touching the backend, check whether ddtrace/dd-trace is already initialized with LLM Observability enabled. If it is, do not add a second, competing init/enable call. This only means skip re-init — it does not mean skip the work. SDK presence is not the same as a well-formed trace or a session ID that actually flows; run the session-ID plumbing audit in Phase 1d and close any gap it finds, even when the SDK is already present.
• Only use packages/features you've explicitly been told to add. Don't decide on your own to enable an additional Datadog product or SDK feature beyond what this skill specifies.
• Persist added dependencies into the deploy's manifest. Any Datadog package you add (ddtrace, dd-trace) must be written into the dependency manifest the build/deploy installs from — the one identified in Phase 1e — not just installed into the local environment. A clean deploy install reads only the manifest and will crash with a missing-module error (e.g. ModuleNotFoundError: No module named 'ddtrace') if the package isn't declared there.
• Don't make stylistic changes to code you're not otherwise touching.
• No package aliases when importing Datadog packages.
• Copy Datadog SDK package names, import paths, and init keyword arguments verbatim from the applicable reference section — do not paraphrase them from memory. The Python LLM Observability import is exactly from ddtrace.llmobs import LLMObs — the module is ddtrace.llmobs, not ddtrace.llm_observability (that module does not exist and produces a deploy-time ModuleNotFoundError). The Node package is dd-trace, initialized with the form that matches the project: CommonJS require('dd-trace').init(...), an ESM import-based form for "type": "module"/.mjs projects, or Next.js's dd-trace/initialize.mjs in instrumentation.ts — all shown in references/llmobs-nodejs.md; never force require(...) into an ESM project. Use the exact LLMObs.enable(...) / .init({...}) keyword arguments shown; do not add, drop, or rename kwargs based on general Datadog knowledge.
• Use a real edit tool for existing files (Edit/Write, not sed -i, awk, or scripted find/replace via Bash). If an edit-by-text-match fails, re-read the file first rather than retrying the identical edit.
• Checklist discipline. Before starting the instrumentation steps, post a short checklist of the steps you're about to take. Check items off as you go, and review the checklist at the end of the run.
• After any code change, check package.json for a lint:fix, fix, or format script (or eslint --fix / prettier --write config) and run it automatically — no need to ask permission.
• Verify the app still builds/runs before declaring success (see references/common-verify-report.md).
• LLM Observability session IDs propagate from the root span only. Never pass a session ID to a child span/decorator — see the "Adding spans" section in references/llmobs-python.md / references/llmobs-nodejs.md.
• Don't promise what the environment doesn't support. If no RUM SDK is detected, don't claim RUM session linking; if no dd-agent is detected, don't claim navigable APM trace linking. State the gap and the upgrade path instead (see the "Beyond SDK init" section in the reference files).

---

Phase 1: Analysis

Inspect the relevant application directory before asking questions or editing files.

1a. Detecting the backend LLM/agent runtime

• Python signals: requirements.txt, pyproject.toml, Pipfile, or *.py files → runtime python
• Node.js signals: a backend entry point (server.js, index.js, an Express/Next.js/Fastify app) → runtime nodejs
• If both exist (e.g. a Next.js app with a Python worker), instrument each backend runtime independently.
• If neither is present, there is nothing to instrument — stop and tell the user.

1b. Detecting the LLM framework/SDK (optional, informational)

Check dependency files for signals of: openai, @anthropic-ai/sdk / anthropic, langchain, langgraph, ai (Vercel AI SDK), boto3 + Bedrock usage, google-generativeai / google-genai, crewai, litellm, pydantic-ai, an MCP SDK, google-adk. This doesn't change the init code (ddtrace/dd-trace auto-instruments these SDKs once the tracer is initialized) — it's only used for confirming the setup with the user and for special-cased frameworks noted in the reference files (Next.js, Vercel AI SDK).

1c. Detecting the backend application framework

• Python: fastapi, flask, or django dependency
• Node.js: express dependency, or next (Next.js API routes / server actions)

1d. Detecting existing instrumentation, and auditing session-ID plumbing

• Grep for ddtrace init (LLMObs.enable(, ddtrace-run) in Python, or dd-trace init (require('dd-trace').init(, dd-trace/initialize) in Node.js.
• Also grep for a frontend RUM SDK (datadogRum.init(, @datadog/browser-rum) — not to set it up, but because its presence determines whether the opt-in RUM↔LLMObs pivot is available (that pivot reuses the RUM session ID as the LLMObs session ID).
• If LLMObs init is already present, do not add a second enable/init call — but do not stop there. SDK presence only means "don't re-init"; it says nothing about whether a session ID actually flows. Run this audit whenever its prerequisite surfaces are present:
  • Session-ID plumbing — applicable whenever Phase 1a found an LLM/agent backend. Check whether a stable, conversation/operation-scoped session ID flows from the appropriate source: for web apps, the frontend sends a per-conversation ID and the backend request model/route reads it; for CLI/background jobs, the backend reuses an existing job/request/task ID or mints one UUID per invocation. A root agent/workflow span must then set it as session_id/sessionId (see "Beyond SDK init" and "Session ID intake by environment" in references/llmobs-python.md / references/llmobs-nodejs.md). LLMObs.enable()/dd-trace().init() alone does not establish this. If a RUM SDK is also present, the RUM↔LLMObs pivot is available as an opt-in — reusing the RUM session ID as the session_id — but that is a deliberate trade-off (the LLMObs session then spans the whole browser session), not the default; don't flag its absence as a gap.
  • If the check finds a gap, tell the user what's missing and fix it (same reference files, same ground rules) even though the SDK itself doesn't need re-initializing. If it passes, say so explicitly. Note the RUM pivot as not applicable rather than a gap when there's no RUM SDK.

1e. Detecting the dependency manifest to persist into

For each backend runtime found in 1a, identify the dependency manifest the build/deploy actually installs from. This is where any Datadog package you add (ddtrace/dd-trace) must be persisted so a clean deploy install includes it. It is not necessarily "whichever manifest file happens to exist": a repo can have several (e.g. an empty requirements.txt alongside a pyproject.toml, or multiple package.json files where only one is the deployed workspace), and editing a non-authoritative one is a silent no-op at deploy time.

Resolve it in this order:

1. Deploy/build install command first (authoritative). Read the project's build/deploy configuration and use the install source it names: render.yaml (buildCommand), Procfile, Dockerfile (RUN … install …), Makefile, CI workflows, package.json scripts. Examples: pip install -r <file> → that requirements file; poetry install / uv sync / pdm install → pyproject.toml (+ its lockfile); pipenv install → Pipfile; npm ci / yarn install / pnpm i → package.json.
2. Else, the highest-priority manifest present. Python: a lockfile-backed manager (uv.lock or poetry.lock → pyproject.toml) > pyproject.toml [project.dependencies]/[tool.poetry.dependencies] > requirements*.txt > Pipfile > setup.py/setup.cfg. Node.js: package.json (always).

Record the manifest path and the manager that owns it. Persisting commands (poetry add, uv add, pdm add, pipenv install, npm/yarn/pnpm/bun add) write the manifest. Non-persisting commands (pip install, uv pip install, conda install) touch only the current environment — with those you must also hand-edit the manifest. If requirements.txt is generated from a requirements.in (pip-tools), edit the .in and recompile. After hand-editing a manifest that has a lockfile, regenerate the lock so a frozen deploy install picks up the new package.

---

Phase 2: Instrumentation — routing table

Provision DD_API_KEY via references/common-credentials.md, then follow the reference for each backend runtime found in Phase 1a:

Need	Read
LLM Observability — Python (SDK init, spans, session ID, RUM/APM linking)	references/llmobs-python.md
LLM Observability — Node.js / Next.js (SDK init, spans, session ID, RUM/APM linking)	references/llmobs-nodejs.md

If Phase 1d found existing LLMObs instrumentation on a runtime, skip only that runtime's init/enable call and credential provisioning — still act on any gap the Phase 1d session-ID plumbing audit found, using the same reference files.

---

Phase 3: Verification and Reporting

See references/common-verify-report.md for the verify step and the JSON report shape.