onboard-cityStands up a new Project Sidewalk city end to end — neighborhood data hunting, the street build and its QGIS QA loop, an imagery preflight across providers, `make onboard-city`, translations, config review, and the server handoff. The scripts do the deterministic work; this skill supplies the judgment calls around them.
Independent GitHub snapshot · Automated classification · Original documentation governs
make onboard-city, translations, config review, and the server handoff. The scripts do the deterministic work; this skill supplies the judgment calls around them.Two commands and this skill. Read docs/onboarding-a-city.md once; it is the runbook this skill drives.
make build-city-data id=<city-id> args="..." # tools/city/onboard_city.py → db/onboarding/<city-id>/ (QA gpkg, SQL, report, endpoints csv)
make check-imagery id=<city-id> args="--sample --<provider>" # preflight, one row per provider in preflight_report.md
make onboard-city id=<city-id> # tools/city/setup_new_city.py → configs, GA, schema, fill, scan, gradient, dump + handoff
Everything under db/onboarding/ is git-ignored. The db container sees it at /opt/onboarding/ only from the main
checkout (db/ is the bind mount), so run these from the main checkout, not a worktree.
laurens-ia), the country elsewhere
(bayonne-fr). It becomes SIDEWALK_CITY_ID, the schema (sidewalk_laurens_ia), and the output dir.sidewalk-laurens), unless a clearly larger city shares the name
(sidewalk-newport-ky).gsv unless the partner says otherwise; mapillary / panoramax / infra3d need the
matching credentials in the web container (Panoramax needs none). If unsure, the preflight in step 3 decides.--regions-source, a
URL or the collaborator's email — it lands in region.data_source):
--region-name-col).--place), not the source's — otherwise a small town is the
neighbourhood "Census Tract 7801" in every mission message and API response. Use --single-region-name with a
--boundary-file, or to override. Names you chose (--regions-file, --rename-regions, a --merge-regions
target) are left alone.--place "<City, State, Country>" geocodes the OSM admin boundary; check the report's
"Boundary:" line names the right place. A partner file goes in with --boundary-file.onboard-city asks; the imagery scan
covers everything either way).make build-city-data id=laurens-ia args="--place 'Laurens, Iowa, USA'"
make build-city-data id=bayonne-fr args="--boundary-file /path/city.geojson --regions-file /path/quartiers.geojson \
--region-name-col nom --regions-source 'https://…'"
Read db/onboarding/<city-id>/report.md before anything else and put the numbers in front of the maintainer:
--merge-tiny-m and whether the source data is
split oddly.OVERSIZED (> 60 km of streets) means split in QGIS; SPARSE/EMPTY means fold into a neighbour
with --merge-regions "A:B" (names, not ids) and rerun. Seattle's regions carry 20–36 km each.The human QA gate: open <city-id>_qa.gpkg in QGIS (qgis_road, qgis_region, city_boundary,
dropped_segments, rider_merges) over a basemap. Fixes come back two ways: parameter changes rerun the build;
hand edits (delete a street, move a boundary) are re-exported with
make build-city-data id=<city-id> args="--from-gpkg", which validates the layers and rewrites the SQL. Never
load a stale SQL over hand edits.
Nothing else checks them: the build keeps the source's names and the fill stores them exactly as staged, and they show up in the region picker, mission messages, and the API. Review them for every city once the region set is settled (merges and boundary edits done), so the list you review is the one that ships.
List them with repr, which shows stray, doubled and non-breaking spaces a plain listing hides:
docker exec projectsidewalk-web sh -c "cd /home && python3.13 -c \"import geopandas as gpd; \
gpkg = 'db/onboarding/<city-id>/<city-id>_qa.gpkg'; \
[print(r.region_id, repr(r.name)) for r in gpd.read_file(gpkg, layer='qgis_region').itertuples()]\""
Look for shape problems (ALL CAPS, all lowercase, leading/trailing/double spaces, non-breaking spaces or
control characters, empty names); repeats, which the build numbers "X (2)": find what actually tells them
apart (#5252 found 1437 shared names across 12 prod cities); and damage from the source — CDMX's file had
accents stripped or letters deleted (SECCIN for Sección, CAADA for Cañada) and plain typos.
Propose fixes by the city's own conventions, not a rule. What #4619 learned fixing 1806 prod names:
de/del lowercase inside a name but capitalized opening one (Del Valle); en, para, y
lowercase; articles lowercase only after de/en (Santa Cruz de las Salinas, but Barrio Los Reyes);
ordinals lowercase (1a Sección, 2do Reacomodo), but a block letter keeps its capital (Picos Iztacalco 1B).UV 2 (Unidad Vecinal), PSE&G, P.I.C.O., LA-32.
Pronounceable ones follow local usage (Infonavit, Pemex).Show the maintainer a table of only the names that change (region_id | current | proposed | why), with the
uncertain ones marked, and apply only what they approve.
Apply by writing the approved rows to db/onboarding/<city-id>/region_renames.csv (current_name,new_name;
write it with Python's csv module so commas, quotes and padding survive — current_name must match exactly,
spaces included). Then rerun the last build command with --rename-regions added, so the GeoPackage, the SQL
and the full report all carry the new names; list them again to confirm. If the GeoPackage holds hand edits a
fetch rerun would lose, use make build-city-data id=<city-id> args="--from-gpkg --rename-regions" instead:
only the SQL and report change (the GeoPackage keeps the old names, and the report loses its fetch-only lines),
so confirm from the Renamed N region(s) log line.
From then on pass --rename-regions on every build of this city: rows already applied are skipped, and
renames run before --merge-regions, so a merge added later uses the new names.
make check-imagery id=<city-id> args="--sample --gsv"
make check-imagery id=<city-id> args="--sample --mapillary" # run every plausible provider
Each run adds a row to preflight_report.md: coverage of a 150-street sample, failures, and how fresh the newest
captures are. Recommend the provider from that table; a failed count that is not zero means a key or quota
problem, not a coverage figure. Under ~70% coverage, say so plainly before continuing — the full scan will hide
that share of the city.
make onboard-city id=<city-id>
It shows the report and preflight, asks for the display name, country/state, provider, status (default private),
launch date (the Friday of next week), and URLs; registers the city in conf/cityparams.conf, conf/messages, and
the docs City IDs table; creates GA properties when ga-service-account.json is present; asks to add both hostnames to the live production
Maps key (skipped with a pointer when gcloud can't edit it); clones a donor schema
(the dev container's city by default — refused, with the schemas that disagree named, if its top evolution is another
branch's under the same number, i.e. its hash is neither the file's nor the other schemas'; pass --donor then);
boots the app once to apply any missing evolutions — and, right after a clone, to let Play verify every applied
hash; loads and fills; runs the full imagery scan for the chosen provider; samples the street gradient (step 8,
from the build's street_structures.csv, so it needs no nightly osm_way cache; a country with no registered
elevation model gets the --dem-dir recipe printed and the run goes on — docs/street-gradient.md); checks that
nothing but onboarding has written to the schema; dumps it to db/<schema>-dump; prints the handoff. Rerunning
skips finished steps. The script's flags go through args= (make onboard-city id=<city-id> args="--skip-scan"):
--dry-run previews the file edits and drives no container (the one mode allowed from a checkout the
containers do not mount — the script hashes the file each step uses against the container's copy, so a
worktree or a second clone is refused).--yes takes every default without asking — the only way to run unattended; without it, a run with nothing
on stdin stops at the first question that is a choice. Pair it with --donor, --country, --pano-type,
--tutorial-region and --regions for the answers that have no default worth taking, and --recreate to
drop an existing schema without being asked.:9100), so npm start and make qa-worktree
stay up; what it needs is the main checkout's target/. If a build holds that, the step stops and names its
pid. A build in a worktree is not in the way (the caches are shared by design). --allow-running-apps boots
past an idle build in the main checkout; a boot an earlier run left behind is never overridable. The nightly
actors are off for the boot, so it writes no job rows into the new schema.region_completion too, which the app recomputes), so a local QA pass or a job run as the city (from
/clustering, or an app left pointed at the schema) stays local and out of the dump; the step prints what it
left out. The one value it changes is street_edge_priority, which a QA walk moves: it resets them to 1 on
y (the default; --yes takes it). --dump-only reruns only this step, which is how a city QA'd after its
first dump gets a clean one without passing "drop and recreate?"; it refuses a schema that is still unfilled.Watch the fill's closing summary (streets, km, sub-20 m share, per-region km, center/zoom) against the report.
messages.<lang>
gets a line: zh-TW transliterated; es/nl/de/pt-BR/fr with the exonym where one exists (Nueva York,
États-Unis) and the English spelling otherwise — the line goes in even then, so a missing line always means
"not looked at yet". English city/state/country names go in the base messages (proper nouns are
language-neutral); the US state abbreviation goes in messages.en. make lint-locales must stay green.config row review. The clone carries the donor's excluded_tags (a European city may want a different tag
set), update_offset_hours (Mikey's load-spreading spreadsheet assigns these), and make_crops; the fill prints
all three and clears the donor's mapathon_event_link and official contact (#5462; a city that wants one sets it on
/admin/partners). Ask; don't guess.private-profiles-by-default, global-leaderboard-excluded,
ai-label-submission-enabled (all false by default).python3 tools/city/create_ga_properties.py <city-id> fills both id maps later; new
properties go inside the existing "Project Sidewalk - Prod/Test" accounts, never new accounts.docs/dev-environment.md is a convenience copy of cityparams.conf.Follow the checklist the orchestrator prints: dump to the server (scp to <netid>@makelab1.cs.washington.edu,
renamed to <schema>-empty-dump at the destination), the IT tooling's setup-new.pl, Maps-key referrers (step 2
asks before adding them to the live production key; if gcloud can't edit it, python3 tools/city/maps_key_referrers.py <city-id>), DNS, the pano scraper's manifest row (<city-id>,<prod fqdn> in /etc/sidewalk/cities.csv on the
scraper host), then the PR (configs + messages + docs). The checklist's step 7 says whether the street grades are in
the dump; when they are not (no registered elevation model, or --skip-gradient), the maintainer either samples
before the dump (for a hand-downloaded model, a rerun with args="--dem-dir … --dem-name … --dem-resolution-m …")
or backfills the live city later. Point the maintainer at the QA items only a person can do:
open the landing page as the new city (map centered, neighborhood names right), walk one street in Explore on the
chosen imagery, check the Explore tag lists against excluded_tags. That walk leaves an audit_task, thousands of
interaction rows and a moved audited_distance in the schema, so after local QA, dump again: make onboard-city id=<city-id> args="--dump-only" lists what the walk left, clears it on y, and writes the dump the server should get.