---

name: speaker-diarization description: "Use for NeMo Speech speaker recognition, diarization, VAD, Sortformer, speaker embeddings, ASR+diarization scoring, forced alignment, RTTM/CTM manifests, and speaker workflow troubleshooting." disable-model-invocation: true metadata: disco-role: operating license: Apache 2.0

Speaker Diarization

Use this sub-skill for NeMo Speech speaker workflows: speaker recognition and verification, speaker embedding extraction, clustering diarization, Sortformer end-to-end diarization, VAD-backed diarization, ASR plus diarization transcript scoring, forced alignment manifests and CTM/ASS outputs, RTTM handling, and speaker workflow troubleshooting.

Route elsewhere when the task is mostly general ASR transcription, ASR fine-tuning, or decoding; generic manifest sharding/tokenizer utilities; audio enhancement; or repository build/test work. For ASR model selection, decoding, timestamps, and fine-tuning, use ../asr/SKILL.md.

Start Here

1. Read references/workflows.md to choose between Sortformer, clustering diarization, speaker recognition, VAD, ASR+diarization, and forced-alignment workflows.
2. Read references/data-formats.md before creating or debugging manifests, RTTM, CTM, UEM, speaker labels, or output file paths.
3. Read references/forced-alignment.md for NeMo Forced Aligner concepts, manifest requirements, CTM/ASS output contracts, and EOU alignment notes.
4. Read references/troubleshooting.md when imports, optional dependencies, CUDA, manifests, Hydra overrides, VAD/clustering, Sortformer, ASR+diarization, or forced alignment fail.
5. Run scripts/check_speaker_manifest.py --help before long diarization, recognition, VAD, or forced-alignment runs.

Safe Bundled Tool

• scripts/check_speaker_manifest.py validates JSONL manifests and common output path fields without importing NeMo, reading audio, downloading checkpoints, training, or writing outputs.
• Example: python scripts/check_speaker_manifest.py manifest.json --task diarization-eval --require-rttm --check-rttm-speakers --require-matching-basename.

Core Facts

• Installed package evidence verified nemo-toolkit version 3.1.0+8f85359 from editable source-backed code. NeMo Speech docs currently target Python 3.12+, PyTorch 2.7+, and GPU/CUDA for training; GPU is strongly recommended for inference.
• Speaker recognition uses nemo.collections.asr.models.EncDecSpeakerLabelModel for TitaNet, ECAPA-TDNN, SpeakerNet, embeddings, verification, and batch inference.
• End-to-end diarization uses nemo.collections.asr.models.SortformerEncLabelModel; offline and streaming Sortformer variants emit speaker segments such as [begin_seconds, end_seconds, speaker_index].
• Cascaded diarization uses nemo.collections.asr.models.ClusteringDiarizer with VAD, speaker embeddings, and clustering. It can also combine ASR word timestamps with diarization labels for speaker-attributed transcripts and cpWER-style evaluation.
• Forced alignment uses CTC ASR models to align audio to reference text or ASR-predicted text and can emit CTM and ASS outputs. The long source aligner scripts are reference-only in this skill; use the bundled references for config and data contracts.

Evidence Base

This sub-skill distills repository evidence from docs/source/asr/speaker_diarization/**, docs/source/asr/speaker_recognition/**, docs/source/asr/speech_classification/**, examples/speaker_tasks/**, scripts/speaker_tasks/**, scripts/voice_activity_detection/**, tools/nemo_forced_aligner/**, nemo/collections/asr/models/label_models.py, nemo/collections/asr/models/sortformer_diar_models.py, tests/collections/speaker_tasks/**, and tools/nemo_forced_aligner/tests/**. Runtime guidance is self-contained; future agents should not need to reopen those source files.