---

name: elements-and-metadata description: "Understand and manipulate Unstructured Element classes, metadata, coordinates, data-source metadata, staging conversions, and bundled inspection helpers. Use when inspecting partition output JSON, preserving table metadata or coordinates, or converting element JSON to JSON, NDJSON, Markdown, text, or HTML; route raw document partitioning to partitioning and chunk composition to chunking." disable-model-invocation: true metadata: disco-role: operating license: Apache 2.0

Elements and Metadata

Use this sub-skill when a task starts from already-partitioned Unstructured elements or element JSON and needs schema inspection, metadata debugging, coordinate reasoning, or format conversion. Do not use it to partition source documents or compose chunks.

Core Model

• Elements are instances of unstructured.documents.elements.Element subclasses such as Title, NarrativeText, ListItem, Table, TableChunk, Image, Formula, CheckBox, CompositeElement, Header, Footer, and Text.
• Serialized element dictionaries generally use type, element_id, text, and metadata; CheckBox also uses checked, while text elements may include embeddings.
• ElementMetadata is sparse: unset fields are omitted from to_dict() and JSON output. Unknown ad-hoc metadata can exist, but known fields should be preferred for portable workflows.
• CoordinatesMetadata requires both points and system; one without the other is invalid. Coordinate dictionaries include points, system, layout_width, and layout_height.
• DataSourceMetadata lives under metadata.data_source and supports source-level fields such as url, version, record_locator, dates, and permissions.

See references/element-schema.md for concrete JSON examples and metadata field guidance.

Staging and Conversion

For in-process conversion, prefer unstructured.staging.base:

from unstructured.staging.base import elements_from_json, elements_to_json, elements_to_md

elements = elements_from_json(filename="elements.json")
markdown = elements_to_md(elements, exclude_binary_image_data=True)
json_text = elements_to_json(elements, indent=2)

For command-line use, use the bundled helpers:

python sub-skills/elements-and-metadata/scripts/convert_elements_json.py elements.json --format markdown --output out.md
python sub-skills/elements-and-metadata/scripts/convert_elements_json.py elements.json --format ndjson --output out.ndjson
python sub-skills/elements-and-metadata/scripts/render_elements_html.py elements.json --output rendered.html

See references/staging-formats.md for JSON, NDJSON, Markdown, text, and HTML behavior.

Table Fidelity

• Preserve metadata.text_as_html whenever table structure matters; Markdown and HTML conversion can use it for richer table output than plain element.text.
• Preserve metadata.table_as_cells, table_id, chunk_index, and num_carried_over_header_rows when reconstructing or validating split tables.
• Compact table HTML normalizes whitespace and strips cosmetic attributes, while preserving structural rowspan and colspan.
• When converting or filtering element JSON, check that Table and TableChunk elements retain both text and structural metadata.

Coordinates

• RelativeCoordinateSystem uses width and height 1; PixelSpace is screen-oriented with origin at the upper-left; PointSpace is Cartesian with origin at the lower-left.
• Conversion between coordinate systems may invert the y-axis depending on orientation. Use element.convert_coordinates_to_new_system(new_system, in_place=False) to inspect without mutation.
• JSON round-trips should preserve all four coordinate fields. Missing layout_width or layout_height can prevent rehydrating concrete coordinate systems.

Debugging Flow

1. Load JSON with elements_from_json() or scripts/convert_elements_json.py --inspect to catch malformed JSON early.
2. Confirm each element has a supported type; unknown text-like types may be dropped by elements_from_dicts() if they are not mapped.
3. Inspect metadata sparsely: missing keys often mean unset fields, not failed serialization.
4. For coordinates, verify points and system are both present and that layout_width/layout_height match the originating page or image.
5. For tables, compare text, metadata.text_as_html, and metadata.table_as_cells before and after conversion.
6. For oversized orig_elements, image payloads, or compressed metadata, watch base64/gzip limits and prefer exclude_binary_image_data=True when rendering.

See references/troubleshooting.md for failure modes and fixes.

Boundaries

• Raw document partitioning belongs to the partitioning sub-skill.
• Chunking, CompositeElement, TableChunk composition, and chunk metadata consolidation belong to the chunking sub-skill unless the task is only inspecting existing chunk JSON.
• Review/test artifacts should not be added to this runtime subtree.