---

name: ml-research-engineer-safeguards description: | Guides ML/research engineering for safeguards—safety classifier development, harm benchmarks and eval suites, labeled dataset design, fine-tuning and ablations, calibration and slice analysis, attack-surface research memos, and promotion criteria for new moderation models. Use when building or evaluating guardrail models, designing safety benchmarks, measuring precision/recall on policy categories, comparing mitigation techniques, or writing research reports on classifier improvements—not for production inference gateways (ml-infrastructure-engineer-safeguards), PII/leakage privacy research (privacy-research-engineer-safeguards), red-team attack campaigns (ai-redteam), AI governance policy (ai-risk-governance), general non-safety research (ai-researcher), or token-efficiency studies (research-engineer-scientist-tokens).

ML / Research Engineer, Safeguards

When to Use

• Define research questions on harm detection, jailbreak resistance, or policy categories
• Curate or audit safety datasets — labeling guidelines, bias checks, version control
• Train or fine-tune classifiers, rankers, or small LLM judges for moderation
• Design benchmarks and eval suites — golden sets, adversarial slices, regression harnesses
• Run ablations — architecture, threshold, data mix, ensemble vs single model
• Analyze metrics — precision/recall, calibration, false positive/negative slices
• Write research memos — methods, results, limitations, production recommendation
• Specify promotion bar for a new safeguard model version

When NOT to Use

• Deploy gateways, GPU serving, canary routing → ml-infrastructure-engineer-safeguards
• Execute structured red-team engagements on prod → ai-redteam
• Draft acceptable-use policy or risk tiers → ai-risk-governance
• Build customer-facing RAG/agents → ai-engineer
• General literature survey unrelated to safety → ai-researcher
• Token/context compression research → research-engineer-scientist-tokens
• Product A/B and business metrics → data-scientist
• PII detection benchmarks, memorization, logging minimization → privacy-research-engineer-safeguards

Related skills

Need	Skill
Privacy research for safeguards	privacy-research-engineer-safeguards
Production safeguard path and rollout	ml-infrastructure-engineer-safeguards
Adversarial attack campaigns	ai-redteam
Governance sign-off and model cards	ai-risk-governance
Production eval harness in app	ai-engineer
General research methodology	ai-researcher
Classical ML and statistics	data-scientist
Token efficiency ablations	research-engineer-scientist-tokens
Release gates and ops cadence	ai-lead-ops

Core Workflows

1. Research framing (safety)

Hypotheses, harm taxonomy, success metrics.

See references/research_framing_safety.md.

2. Benchmarks and datasets

Golden sets, labeling, versioning.

See references/safety_benchmarks_datasets.md.

3. Model development

Training, fine-tuning, ensembles.

See references/classifier_model_development.md.

4. Evaluation and metrics

Slices, calibration, error analysis.

See references/evaluation_metrics_analysis.md.

5. Ablations and experiments

Controls, reproducibility.

See references/ablation_experiment_design.md.

6. Handoff to production

Promotion criteria, monitoring hooks.

See references/research_to_production_handoff.md.

Outputs

• Research brief — question, baseline, hypothesis, metrics
• Dataset card — sources, label schema, known limitations
• Benchmark spec — cases, categories, pass/fail rubric
• Results table — metrics by slice with confidence intervals where possible
• Error analysis — representative FP/FN clusters
• Promotion recommendation — go/no-go vs current production classifier

Principles

• Measure what policy cares about — category-level recall on high-severity harms
• Report failures honestly — FPs hurt UX; FNs hurt safety
• Hold out adversarial refresh — do not train on the only test set
• Reproducible — seeds, data version, model hash, eval script
• Separate research from ops — research proves lift; infra ships it