---

name: text-and-datasets description: "Use Yellowbrick dataset loaders, cache controls, and text visualizers for tabular data, corpora, and text embeddings." disable-model-invocation: true metadata: disco-role: operating license: Apache 2.0

Yellowbrick Text and Datasets

Use this sub-skill when the task is about Yellowbrick dataset loading, dataset cache/download behavior, the load_* helpers, the hobbies corpus, or text-specific visualizers. It covers FreqDistVisualizer, TSNEVisualizer, UMAPVisualizer, DispersionPlot, WordCorrelationPlot, and PosTagVisualizer.

For package-wide Yellowbrick lifecycle, axes reuse, style, and headless saving, read shared visualizer patterns. For broad install, Matplotlib backend/font, and scikit-learn compatibility failures, read the root troubleshooting reference before applying text/dataset-specific fixes.

Use this sub-skill for

• Choosing among Yellowbrick example datasets and matching them to regression, classification, clustering, time-series-style, or text-analysis tasks.
• Loading tabular datasets with load_bikeshare(), load_concrete(), load_credit(), load_energy(), load_game(), load_mushroom(), load_occupancy(), load_spam(), load_walking(), or load_nfl().
• Loading the hobbies corpus with load_hobbies() and using Corpus.data, Corpus.target, Corpus.labels, or Corpus.files.
• Deciding between default X, y returns, return_dataset=True, to_numpy(), to_pandas(), and to_dataframe().
• Controlling or inspecting the dataset cache with data_home, YELLOWBRICK_DATA, and the no-download cache inspector.
• Using text visualizers after the corpus is already vectorized, tokenized, or part-of-speech tagged.
• Explaining optional extras: pandas for DataFrame returns, umap-learn for UMAPVisualizer, and nltk/spacy plus data/models for parser-based POS tagging.

Route elsewhere when

• The user has already loaded data and now wants classifier score plots such as reports, confusion matrices, ROC/PR curves, or threshold tuning: route to classifier visualizers.
• The user wants residuals, prediction error, Cook's distance, or alpha selection: route to regressor visualizers.
• The user wants feature ranking, PCA/manifold feature projections, class balance, binning, or feature-target correlation: route to feature-target visualizers.
• The user wants clustering diagnostics, validation/learning curves, CV scores, RFECV, feature importances, or dropping curves: route to cluster-model-selection.
• The user asks about contrib wrappers, decision boundaries, missing-value contrib plots, or statsmodels adapters: route to contrib-and-extensions.

Required read and run map

Need	Read or run	When to use it
Dataset names, task fit, load_*, return types, Dataset/Corpus, cache rules, downloader flags, and offline behavior	datasets reference	Read before giving loader code, cache advice, or no-network guidance.
Text visualizer selection, exact input formats, optional extras, and safe file output	text visualizers reference	Read before using any yellowbrick.text visualizer or diagnosing text input-shape errors.
Dataset cache/download/signature/offline failures, missing optional dependencies, wrong class names, parser/tagset errors, and display/font warnings	text/dataset troubleshooting	Read after any failure symptom or before advising a risky cleanup/download action.
Common Yellowbrick fit/score/transform/show(outpath=...), Matplotlib Agg, axes, and style rules	shared visualizer patterns	Read before writing report-quality code or headless automation.
Broad installation, backend, font, or scikit-learn compatibility issues	root troubleshooting	Read when the problem is not specific to text or datasets.
No-download cache inspection	check_dataset_cache.py	Run when the user is offline, wants to know what is already cached, or needs signature/cache state without downloads or deletes.
Safe text plotting smoke	text_smoke.py	Run to confirm yellowbrick.text can render inline synthetic text plots with Matplotlib Agg; it uses no external data and no optional downloads.

Example safe runs from an environment where Yellowbrick is importable:

python skills/disco/yellowbrick/sub-skills/text-and-datasets/scripts/check_dataset_cache.py --dataset hobbies
python skills/disco/yellowbrick/sub-skills/text-and-datasets/scripts/text_smoke.py --outdir /tmp/yellowbrick-text-smoke

Dataset quick map

Loader	Object	Shape or size	Best fit	Notes
load_bikeshare()	Dataset	X=(17379, 12)	regression	Bike sharing demand examples.
load_concrete()	Dataset	X=(1030, 8)	regression	Small, numeric, good for regressor demos.
load_credit()	Dataset	X=(30000, 23)	binary classification / clustering	Credit default data; can be large for quick CV examples.
load_energy()	Dataset	X=(768, 8)	regression / multi-output source data	Default y is the documented primary target.
load_game()	Dataset	X=(67557, 42)	multiclass classification	Categorical features usually need encoding before sklearn estimators.
load_hobbies()	Corpus	448 docs, 5 labels	text analysis / classification / clustering	Returns a Corpus, not X, y; use corpus.data and corpus.target.
load_mushroom()	Dataset	X=(8123, 3)	binary classification / clustering	Categorical mushroom attributes need preprocessing for many estimators.
load_occupancy()	Dataset	X=(20560, 5)	binary classification	Multivariate time-series-style occupancy data.
load_spam()	Dataset	X=(4600, 57)	binary classification / threshold analysis	Useful for threshold and imbalance demonstrations.
load_walking()	Dataset	X=(149332, 4)	clustering / time-series-style diagnostics	Large row count; sample for quick visual checks.
load_nfl()	Dataset	X=(494, 23)	clustering	Football receiver clustering examples.

Text visualizer quick map

Visualizer	Input required	Best fit	Constraints
FreqDistVisualizer	Count-vectorized document matrix plus feature names	Top-token counts in a corpus or by class	Use FreqDistVisualizer, not FrequencyVisualizer, in public guidance.
TSNEVisualizer	Numeric vectorized corpus matrix	2D document embedding and class/cluster separation	t-SNE is expensive; vectorize first and lower perplexity for tiny samples.
UMAPVisualizer	Numeric vectorized corpus matrix	2D document embedding with UMAP	Requires optional umap-learn; use metric="cosine" for many text vectors.
DispersionPlot	Tokenized documents: list/generator of token lists	Lexical dispersion of known terms across documents	Search terms must survive tokenization and casing.
WordCorrelationPlot	Raw text documents: list/generator of strings	Word or phrase co-occurrence heatmap	Every requested word/phrase must appear in the corpus.
PosTagVisualizer	Pre-tagged nested docs or raw text with parser	POS tag distributions	Prefer pre-tagged tuples for no-optional-dependency smoke checks.

Canonical dataset pattern

from yellowbrick.datasets import load_concrete, load_hobbies

# Tabular data: pandas DataFrame/Series if pandas is installed, otherwise numpy arrays.
X, y = load_concrete()

# Request the object when you need metadata or explicit return control.
dataset = load_concrete(return_dataset=True)
X_np, y_np = dataset.to_numpy()

# Text corpus loaders return Corpus objects, not X/y directly.
corpus = load_hobbies()
docs = corpus.data
target = corpus.target
labels = corpus.labels

Canonical text pattern

import matplotlib
matplotlib.use("Agg")  # set before pyplot in headless sessions

from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer
from yellowbrick.text import FreqDistVisualizer, TSNEVisualizer

count_vectorizer = CountVectorizer()
X_counts = count_vectorizer.fit_transform(docs)
features = count_vectorizer.get_feature_names_out()

freq = FreqDistVisualizer(features=features)
freq.fit(X_counts, target)
freq.show(outpath="freqdist.png", clear_figure=True)

X_tfidf = TfidfVectorizer().fit_transform(docs)
tsne = TSNEVisualizer(random_state=13, perplexity=10)
tsne.fit(X_tfidf, target)
tsne.show(outpath="tsne.png", clear_figure=True)

Safety checklist

• Do not call a loader in offline mode unless its dataset directory already exists in the selected cache; missing data triggers Yellowbrick's downloader.
• Use the cache inspector before telling an offline user that a loader will work.
• Use return_dataset=True only on tabular loaders; load_hobbies() already returns a Corpus object.
• Confirm whether pandas is installed before promising DataFrame/Series returns; numpy arrays are the normal fallback.
• Confirm text input state: raw strings for vectorizers and WordCorrelationPlot, token lists for DispersionPlot, numeric vectors for TSNE/UMAP, and nested (token, tag) tuples for parser-free POS visualization.
• Confirm optional dependencies before recommending UMAPVisualizer or raw-text parser mode for PosTagVisualizer.
• Save figures with a non-interactive backend and class visualizer API when the output must be a file.