---

name: drift-monitoring description: "Detect and interpret NannyML feature, output, target, and multivariate drift, then rank suspicious columns by alerts or performance correlation." disable-model-invocation: true metadata: disco-role: operating license: Apache 2.0

Drift Monitoring

Use this sub-skill when the task is about NannyML drift detection: univariate feature drift, model output drift, target distribution drift, multivariate PCA reconstruction-error drift, multivariate domain-classifier drift, or ranking drifted columns for follow-up investigation.

Route model performance estimation/calculation to the performance sub-skill, data-quality-only checks to the data-setup sub-skill, and YAML/CLI orchestration to the CLI sub-skill.

Quick routing

• Read references/workflows.md for univariate drift, output/target drift, PCA reconstruction-error drift, domain-classifier drift, and ranking recipes.
• Read references/methods.md for supported drift methods, feature-type handling, default thresholds, result columns, and ranker constraints.
• Read references/troubleshooting.md when errors mention missing columns, wrong categorical/continuous type, ignored chi2 thresholds, one-method ranking requirements, not-fitted rankers, or domain-classifier runtime cost.
• Read ../../references/results-and-plots.md for filter, to_df, plot, distribution plots, and compare patterns.
• Read ../data-setup/SKILL.md for chunking, thresholds, and data requirements shared by drift workflows.

Choose the right drift workflow

Task	Use	Output
Detect per-feature distribution changes	nannyml.UnivariateDriftCalculator	One drift metric per selected method/column/chunk
Detect drift in prediction labels or scores	UnivariateDriftCalculator on y_pred and/or y_pred_proba	Output drift over chunks
Detect target distribution drift after labels arrive	UnivariateDriftCalculator on y_true	Target drift over chunks
Detect multivariate drift with a single less-explainable score	DataReconstructionDriftCalculator	reconstruction_error per chunk
Detect distribution separability between reference and chunks	DomainClassifierCalculator	domain_classifier_auroc per chunk
Prioritize drifted features by alert count	AlertCountRanker	Ranking DataFrame with number_of_alerts, column_name, rank
Prioritize drifted features by correlation with performance change	CorrelationRanker	Ranking DataFrame with Pearson correlation and drift flags

Minimal univariate drift example

import nannyml as nml

reference, analysis, _ = nml.load_synthetic_car_loan_dataset()
features = ['car_value', 'salary_range', 'debt_to_income_ratio', 'loan_length', 'repaid_loan_on_prev_car', 'size_of_downpayment', 'driver_tenure']

calculator = nml.UnivariateDriftCalculator(
    column_names=features,
    timestamp_column_name='timestamp',
    continuous_methods=['jensen_shannon', 'wasserstein'],
    categorical_methods=['jensen_shannon', 'l_infinity'],
    chunk_size=5000,
)
result = calculator.fit(reference).calculate(analysis)
print(result.filter(period='analysis', methods=['jensen_shannon']).to_df(multilevel=False).head())

Minimal multivariate drift example

import nannyml as nml

reference, analysis, _ = nml.load_synthetic_car_loan_dataset()
features = ['car_value', 'salary_range', 'debt_to_income_ratio', 'loan_length', 'repaid_loan_on_prev_car', 'size_of_downpayment', 'driver_tenure']

pca_drift = nml.DataReconstructionDriftCalculator(
    column_names=features,
    timestamp_column_name='timestamp',
    chunk_size=5000,
).fit(reference).calculate(analysis)

print(pca_drift.filter(period='analysis').to_df(multilevel=False).head())

Minimal ranking example

Filter univariate results to one method per feature before ranking.

ranker = nml.AlertCountRanker()
ranking = ranker.rank(result.filter(period='analysis', methods=['jensen_shannon']), only_drifting=True)
print(ranking.head())

For correlation ranking, use one performance metric and one drift method:

correlation_ranker = nml.CorrelationRanker()
correlation_ranker.fit(estimated_performance.filter(period='reference', metrics=['roc_auc']))
ranking = correlation_ranker.rank(
    result.filter(period='analysis', methods=['jensen_shannon']),
    estimated_performance.filter(period='analysis', metrics=['roc_auc']),
)

Decision points

• Use univariate drift when explainability by feature or output column matters.
• Use PCA reconstruction-error drift when a single multivariate score is useful and less explainability is acceptable.
• Use domain-classifier drift when the user wants a separability-based multivariate score and accepts internal LightGBM/FLAML runtime cost.
• Use treat_as_categorical / treat_as_numerical to override pandas dtype inference when feature semantics are known.
• Use output drift on prediction labels/scores when targets are unavailable but model outputs may shift.
• Use target drift only when actual target labels are available.
• Ranking is a follow-up to univariate/data-quality/summary results; it is not a replacement for calculating drift.

Route elsewhere

• CBPE, DLE, realized performance, or estimated-vs-realized comparison -> ../performance-monitoring/SKILL.md
• Missing/unseen/range data-quality checks, summary statistics, chunking, thresholds, or built-in datasets -> ../data-setup/SKILL.md
• CLI config, scheduling, output writers, stores -> ../cli-and-automation/SKILL.md