---

name: data-loading description: "Load, inspect, validate, and split Surprise rating datasets." disable-model-invocation: true metadata: disco-role: operating license: BSD 3-Clause

Surprise data loading

Use this sub-skill when the task is to load ratings into Surprise, inspect the resulting Trainset, diagnose parser/layout errors, or prepare train/test material from local data. It is intentionally about data ingestion and trainset construction only.

Fast route

1. Identify the data source:
   • Built-in cache/download: Dataset.load_builtin(name, prompt=...).
   • One ratings file: Reader(...) + Dataset.load_from_file(file_path, reader).
   • Pandas dataframe: Reader(rating_scale=...) + Dataset.load_from_df(df[[user, item, rating]], reader).
   • Predefined train/test files: Dataset.load_from_folds([(train_file, test_file), ...], reader) + PredefinedKFold().split(data).
2. Validate the layout before using an algorithm: check Reader.line_format, sep, skip_lines, dataframe column order, and fold file existence.
3. Build or materialize trainsets:
   • Full local dataset: trainset = data.build_full_trainset().
   • Predefined folds: iterate for trainset, testset in PredefinedKFold().split(data): ....
4. Use raw ids at public prediction/testset boundaries and inner ids only for Trainset internals. Convert with to_inner_uid, to_inner_iid, to_raw_uid, and to_raw_iid.
5. For details, read references/data-loading.md. For errors, read references/troubleshooting.md.

API selection

Need	Use	Contract to remember
Built-in MovieLens/Jester data	`Dataset.load_builtin("ml-100k"	"ml-1m"
Custom delimited file	Reader(line_format=..., sep=..., rating_scale=..., skip_lines=...) then Dataset.load_from_file(...)	line_format tokens are space-separated field names; sep is the actual delimiter in the file.
Dataframe	Dataset.load_from_df(df[[user_col, item_col, rating_col]], reader)	Column names are ignored; positional order must be user, item, rating. Only rating_scale is needed on the reader.
Existing train/test folds	Dataset.load_from_folds(folds_files, reader)	folds_files is a list of (train_file, test_file) tuples, and every path must exist up front.
Trainset inspection	n_users, n_items, n_ratings, ur, ir, all_ratings(), build_testset(), build_anti_testset()	ur, ir, all_*, and knows_* use inner ids; build_*testset() returns raw ids.

Bundled smoke scripts

Run these from this sub-skill directory after installing Surprise and any script-specific dependency:

• python scripts/load_custom_file_smoke.py — creates a temporary ratings file, loads it with Reader/load_from_file, checks zero-rating preservation, and verifies a wrong separator fails.
• python scripts/load_from_dataframe_smoke.py — creates a tiny dataframe, loads it with explicit column ordering, and checks the bad-ordering trap.
• python scripts/load_predefined_folds_smoke.py — creates temporary train/test fold files, loads them with load_from_folds, materializes PredefinedKFold, and checks nonexistent fold paths fail early.

Boundaries

This sub-skill excludes algorithm choice, similarity math, cross-validation metrics/search, top-N recommendation generation, and model serialization. Use sibling sub-skills for those tasks: ../prediction-algorithms/, ../evaluation-and-search/, and ../recommendation-and-analysis/.