---

name: embeddings description: "Add embeddings to Unstructured elements with provider-specific encoders, credential-safe configuration, and metadata-preserving enrichment checks. Use when an agent needs BaseEmbeddingEncoder, EmbeddingConfig, OpenAI, OctoAI, Mixedbread, VoyageAI, VertexAI, Bedrock, or HuggingFace embedding guidance after partitioning or chunking." disable-model-invocation: true metadata: disco-role: operating license: Apache 2.0

Embeddings

Use this sub-skill after documents have already been partitioned or chunked into Unstructured Element objects and the task is to attach vector embeddings to those elements or embed a search query with the same provider.

Start Here

1. Route upstream work first:
   • Use partitioning to create elements from files, URLs, HTML, text, or streams.
   • Use chunking before embedding when the vector store should index RAG-sized chunks rather than raw document elements.
   • Use elements-and-metadata when inspecting serialized element JSON or preserving coordinate/table/source metadata.
2. Select the provider module from references/provider-reference.md and install only the provider SDKs needed for that route.
3. Read credentials from environment variables or a secret manager at runtime; never hard-code, log, serialize, or commit API keys.
4. Instantiate the provider-specific EmbeddingConfig and EmbeddingEncoder, then call:
   • embed_documents(elements) to mutate and return the same element objects with element.embeddings populated.
   • embed_query(query) to produce a single query vector for retrieval.
5. Validate vector count and dimensions before persistence, especially when mixing providers or overriding model dimensions.

Core Model

• unstructured.embed.interfaces.EmbeddingConfig is a Pydantic base class; concrete providers add credential, model, region, batching, or client options.
• unstructured.embed.interfaces.BaseEmbeddingEncoder defines embed_documents(elements), embed_query(query), num_of_dimensions, is_unit_vector, and initialize().
• Provider implementations convert each element to text with str(element) before embedding, so empty text-like elements can create low-value vectors or provider errors.
• embed_documents() writes vectors to element.embeddings; it is not a separate enrichment record and generally mutates the input elements in place.
• Serialized text elements may include an embeddings field, but ordinary metadata fields should remain intact if the element objects are preserved.

Safe Pattern

import os
from unstructured.embed.openai import OpenAIEmbeddingConfig, OpenAIEmbeddingEncoder

config = OpenAIEmbeddingConfig(
    api_key=os.environ["OPENAI_API_KEY"],
    model_name="text-embedding-3-small",
)
encoder = OpenAIEmbeddingEncoder(config=config)
embedded_elements = encoder.embed_documents(chunks)
query_vector = encoder.embed_query("invoice due date")

For provider-specific names, credentials, dependency extras, batching behavior, and model caveats, use references/provider-reference.md.

Metadata and Provenance

• Preserve source metadata by embedding existing Element objects rather than converting them to plain strings and rebuilding them.
• If downstream systems require enrichment provenance, add non-secret metadata outside the embedding vector itself, for example provider name, model name, and embedding timestamp managed by your application.
• Keep credentials and raw provider responses out of ElementMetadata, logs, vector-store payloads, and serialized JSON fixtures.
• If embedding chunks, preserve metadata.orig_elements only when retrieval or citation workflows need source traceability; otherwise omit it upstream with the chunking sub-skill to keep payloads smaller.

Bundled Helpers

Use the bundled checker to verify importability and environment-variable presence without contacting providers or printing secret values:

python sub-skills/embeddings/scripts/embedding_config_check.py --provider openai
python sub-skills/embeddings/scripts/embedding_config_check.py --all --json

The checker intentionally does not call get_client(), embed_query(), or embed_documents(), because those may create SDK clients, read credential material, write provider-specific credential files, or make network calls.

Routing Boundaries

• Raw document partitioning, OCR, table extraction, and partition() kwargs belong to partitioning.
• Chunk sizing, overlap, table chunking, and orig_elements decisions belong to chunking.
• Element JSON conversion, schema inspection, coordinates, and staging conversions belong to elements-and-metadata.
• Broad ingest connectors and destination vector-store writes are excluded; the in-repo embed package README notes this area moved toward Unstructured Ingest.

Review Checklist

• Confirm the provider SDK and optional dependencies are installed before suggesting runtime embedding.
• Confirm credentials are sourced securely and never shown in code examples, logs, or serialized elements.
• Confirm len(elements) equals the number of returned vectors; providers use assertions for this in several implementations.
• Confirm element text is non-empty and appropriate for the provider token/model limits.
• Confirm metadata, element_id, table metadata, and source provenance survive after embedding.
• Confirm query vectors use the same provider/model/dimension as indexed document vectors.