City Walk/WissenZum StadtspaziergangViewer
ImpressumDatenschutz
Leitfaden
  • So funktioniert der Stadtspaziergang
  • Woher die Daten kommen
  • Quellen nach Stadt
  • Wie der Viewer zu einer Angabe kommt
  • Der Weg der Daten
  • Bedienung
  • Glossar
Guide (English)
  • How the city walker works
  • Where the data comes from
  • Sources by city
  • How the viewer comes by what it shows
  • From download to browser
  • Using the viewer
  • Glossary
Entwicklung (English)
  • docs/ — the project knowledge base
  • Rendering — how the data becomes pixels
  • Data pipeline — the bakes, the build step and the artifact contracts
  • Architecture decision records
  • Data → feature provenance
  • Transformation catalog
  • Portability — rendering any location, with whatever data it has
  • Writing a walk
  • Routines: the twin's scheduled agents
  • Implementation plans, backlog and audit history
    • Completed plans — condensed records
    • Plan 086 [rewrite]: The rewrite series — faster on the device, fewer bytes on the wire, less to hold in the head
    • Plan 017: Any German city — the open half: a second Land, NAS input, OSM land cover, a site check
    • Plan 019: The GPU checklist — look at everything built headless, on real GPUs and phones
    • Plan 021: /wissen on Astro Starlight instead of a hand-built Next route
    • Plan 047 (spike): "Problem melden" — the crash report with somewhere to go
    • Plan 048 (spike): Share a view as a link, and commit the reference views the GPU checks need
    • Plan 050: Landmarks and what LoD2 leaves out — measured in DOM1, named by OSM and Wikidata
    • Plan 051: Stand-ins where a Land publishes less, and looks derived from each city's data
    • Plan 052: A queryable twin — identity, semantics and provenance, asked on demand
    • Plan 053: Time and live sources — the twin's fourth dimension
    • Plan 054: Scenarios — what-if studies drawn in the scene's own hand
    • Plan 069: Find what holds the load at 52 %, then take it off the tile path
    • Plan 070: Each tile holds a fraction of the raster memory it holds today
    • Plan 071: A phone renders a lean scene, a desktop a lighter one
    • Plan 074: Bake the static dressing, load the optional features on demand, one store for the HUD
    • 076 — Streaming without holes: the ground first, a plate under everything, coarse buildings and trees from the start
    • Plan 077: One rasters module — a tile's rasters loaded, shared and weighed behind one interface, their files named by the artifact table
    • 078 — One scene environment the dressings bind at birth
    • Plan 079: A page that lost its GPU comes back where it stood, through one recovery session
    • 080 — One row per inquiry kind: the twin's per-kind dispatch in one place
    • 082 — One answer turn, and a code for every way it ends
    • 084 — One session for an answer's lifetime: reading aloud, dictation and the scene's waits in one module
    • 085 — Window niches and shop windows: performance and follow-ups
    • Plan 087 [rewrite]: Plates compared on a real GPU, and one ledger every plan of the series writes its figures into
    • Plan 088 [rewrite]: The tileset is a published product, named by a lock file; the web build bakes nothing and data/ leaves git
    • Plan 089 [rewrite]: The bakes are type-checked in CI against a baseline that only shrinks
    • Plan 090 [rewrite]: A tile's dressing arrives in the layout the GPU draws from
    • Plan 091 [rewrite]: A tile's rasters arrive as the GPU reads them — plain arrays for ids, blocks for soft fields, a few files and bindings
    • Plan 092 [rewrite]: The bakes work on site-wide typed tables, cut tiles last, and no step rewrites another's file
    • Plan 093 [rewrite]: The stream's tile is smaller than the bake's — a quadtree whose levels replace the raster twins, built only if a phone still asks for it
    • Plan 094 [rewrite]: The clay's limits are counted by a test, and every pipeline the scene draws with is known at boot
    • Plan 095 [rewrite]: The viewer is a package with one entrance, a coordinate's frame is its type, and bootApp is owners
    • Plan 096 [rewrite]: Every bake heuristic that guesses has a reference set and a score a bake PR must hold
  • What the data streams carry that the walker does not use yet
Contents
Leitfaden
  • So funktioniert der Stadtspaziergang
  • Woher die Daten kommen
  • Quellen nach Stadt
  • Wie der Viewer zu einer Angabe kommt
  • Der Weg der Daten
  • Bedienung
  • Glossar
Guide (English)
  • How the city walker works
  • Where the data comes from
  • Sources by city
  • How the viewer comes by what it shows
  • From download to browser
  • Using the viewer
  • Glossary
Entwicklung (English)
  • docs/ — the project knowledge base
  • Rendering — how the data becomes pixels
  • Data pipeline — the bakes, the build step and the artifact contracts
  • Architecture decision records
  • Data → feature provenance
  • Transformation catalog
  • Portability — rendering any location, with whatever data it has
  • Writing a walk
  • Routines: the twin's scheduled agents
  • Implementation plans, backlog and audit history
    • Completed plans — condensed records
    • Plan 086 [rewrite]: The rewrite series — faster on the device, fewer bytes on the wire, less to hold in the head
    • Plan 017: Any German city — the open half: a second Land, NAS input, OSM land cover, a site check
    • Plan 019: The GPU checklist — look at everything built headless, on real GPUs and phones
    • Plan 021: /wissen on Astro Starlight instead of a hand-built Next route
    • Plan 047 (spike): "Problem melden" — the crash report with somewhere to go
    • Plan 048 (spike): Share a view as a link, and commit the reference views the GPU checks need
    • Plan 050: Landmarks and what LoD2 leaves out — measured in DOM1, named by OSM and Wikidata
    • Plan 051: Stand-ins where a Land publishes less, and looks derived from each city's data
    • Plan 052: A queryable twin — identity, semantics and provenance, asked on demand
    • Plan 053: Time and live sources — the twin's fourth dimension
    • Plan 054: Scenarios — what-if studies drawn in the scene's own hand
    • Plan 069: Find what holds the load at 52 %, then take it off the tile path
    • Plan 070: Each tile holds a fraction of the raster memory it holds today
    • Plan 071: A phone renders a lean scene, a desktop a lighter one
    • Plan 074: Bake the static dressing, load the optional features on demand, one store for the HUD
    • 076 — Streaming without holes: the ground first, a plate under everything, coarse buildings and trees from the start
    • Plan 077: One rasters module — a tile's rasters loaded, shared and weighed behind one interface, their files named by the artifact table
    • 078 — One scene environment the dressings bind at birth
    • Plan 079: A page that lost its GPU comes back where it stood, through one recovery session
    • 080 — One row per inquiry kind: the twin's per-kind dispatch in one place
    • 082 — One answer turn, and a code for every way it ends
    • 084 — One session for an answer's lifetime: reading aloud, dictation and the scene's waits in one module
    • 085 — Window niches and shop windows: performance and follow-ups
    • Plan 087 [rewrite]: Plates compared on a real GPU, and one ledger every plan of the series writes its figures into
    • Plan 088 [rewrite]: The tileset is a published product, named by a lock file; the web build bakes nothing and data/ leaves git
    • Plan 089 [rewrite]: The bakes are type-checked in CI against a baseline that only shrinks
    • Plan 090 [rewrite]: A tile's dressing arrives in the layout the GPU draws from
    • Plan 091 [rewrite]: A tile's rasters arrive as the GPU reads them — plain arrays for ids, blocks for soft fields, a few files and bindings
    • Plan 092 [rewrite]: The bakes work on site-wide typed tables, cut tiles last, and no step rewrites another's file
    • Plan 093 [rewrite]: The stream's tile is smaller than the bake's — a quadtree whose levels replace the raster twins, built only if a phone still asks for it
    • Plan 094 [rewrite]: The clay's limits are counted by a test, and every pipeline the scene draws with is known at boot
    • Plan 095 [rewrite]: The viewer is a package with one entrance, a coordinate's frame is its type, and bootApp is owners
    • Plan 096 [rewrite]: Every bake heuristic that guesses has a reference set and a score a bake PR must hold
  • What the data streams carry that the walker does not use yet
Entwicklung (English)

Plan 089 [rewrite]: The bakes are type-checked in CI against a baseline that only shrinks

Executor instructions: Read 086-rewrite-programme.md and AGENTS.md first. Five phases, each one PR. Phases 0 and 1 touch no bake module and can go at once; phases 2 to 4 edit bake modules and wait for the maintainer's word on when (see "The cost nobody sees"). Run every Verify before the next phase; on a STOP condition, stop and report. Update this plan's row in docs/plans/README.md after each phase.

Drift check (run first): git diff --stat 5dfa300d..HEAD -- pipeline/pyproject.toml pipeline/uv.lock pipeline/bake/common.py pipeline/bake/step_cache.py .github/workflows/ci.yml package.json — and run the checker again (below): the counts in Current state are of 5dfa300d.

Status

  • Priority: P2 — cheap to start, and what plan 092 stands on
  • Effort: M (phases 0–1: S)
  • Risk: LOW for the gate; MED for the fixes, each of which re-runs bake steps on the maintainer's machine
  • Depends on: none. Needed by 092.
  • Category: rewrite — less complexity (contracts that are checked, not remembered); beside the aims: a re-bake that fails before it runs, not hours in
  • Series: 086-rewrite-programme.md
  • Planned at: commit 5dfa300d, 2026-11-16
  • Status: TODO

Why this matters

pipeline/bake/ is 41 757 lines of Python in 83 files, and nothing checks its types. 1 696 of its 2 031 functions state a return type, so the intent is written down — and unread: CI's pipeline job runs pytest and ruff. A bake step runs for minutes to hours on raw data only the maintainer's machine holds, so a None that arrives where a geometry was expected is found late and far from its cause. Plan 092 will rewrite the input and output of every step (site-wide typed tables in place of a file per tile); doing that to unchecked code would trade one drift for another.

Current state

Measured on 2026-11-16 at 5dfa300d with pyright 1.1.414 in its default mode, Python 3.13 (cd pipeline && uv sync --locked && pyright --pythonpath .venv/bin/python --outputjson bake): 593 errors in 56 of 83 files, 27 files clean. By what they are:

ClassFindingsWhat it is
The bakes' own annotation184229 uses of shapely.Geometry in 35 modules (def carriageways(tile: Tile) -> list[shapely.Geometry], ways.py). In the stubs the checker has for shapely that is the bare base class without area, buffer, is_empty, geom_type; the class meant is shapely.geometry.base.BaseGeometry. Most asked: geom_type 29, area 25, is_empty 14, length 14, buffer 12, interpolate 12, exterior 9
None not handled67reportOptionalMemberAccess 29, reportOptionalSubscript 13, reportOptionalOperand 7, "None is not iterable" 18 — p.text.split() on an XML element (citygml.py), tile.tree_cadastre.credit (cadastre.py), bridge.py, rail.py, cultivated.py
Possibly unbound15a name bound in one branch and read after it: roofs.py (lab, faces, surface, drawn), facades.py (ws), lsc.py, facade_measure.py
A library without types93labels, n = ndi.label(mask) 60 (scipy's partial stubs); rasterio.transform, pyogrio.raw "not a known attribute of module" 16; laspy's array views 12; from scipy.spatial import cKDTree 5
Calls52"no overloads match" 32 (numpy), a wrong count of positional arguments 20 (_moved(mask, *moves[k], 0, res) in dormers.py: a star over a tuple of unknown length; Window(c0, r0, …) of rasterio)
What follows from the rows above182reportArgumentType 119, reportOperatorIssue 29, the rest of reportAttributeAccessIssue 16, reportReturnType 9, reportIndexIssue 5, reportAssignmentType 4 — mostly numpy unions grown from an untyped call upstream

Most findings: roofs.py 56, furniture.py 43, water.py 36, roof_planes.py 34, dormers.py 33, lowveg.py 32. The modules plan 092 rewrites are nearly clean already: common.py 4, osm.py 1, osm_index.py 6, spec.py 1, citygml.py 6, and none in step_cache.py, rasters.py, fetch.py, net.py, __main__.py.

Which libraries ship types (looked up in pipeline/.venv, 2026-11-16): numpy, pyproj, Pillow and scikit-image do (py.typed); shapely, rasterio, pyogrio, networkx and onnxruntime do not; scipy and laspy ship a few .pyi files. For shapely the checker brings stubs of its own, so a new version of the checker can move the counts.

The files that matter:

  • pipeline/pyproject.toml — the dependencies, pinned exactly; the dev group is ["pytest==9.2.0", "ruff==0.17.3"]; ruff's rules are E, F, I, B, UP, SIM. No checker's section yet.
  • .github/workflows/ci.yml, job pipeline: uv sync --locked, uv run pytest -q tests, uv run ruff check ., uv run ruff format --check .. package.json's test:pipeline runs the same three.
  • pipeline/bake/common.py — what every step reads and writes with: Tile, read_layer (returns tuple[np.ndarray, dict[str, np.ndarray]]), feature(geometry: dict, properties: dict | None) -> dict, write_geojson(path, features: list[dict], epsg, …), geometry_json, table_json, save_grey_png. A handed-over file's shape is a dict literal here and a zod schema in lib/city/<name>-file.ts (ADR 0059).
  • pipeline/bake/step_cache.py — code_hash(module) hashes the bytes of the step's module and of every bake module it reaches, and locked_runtime(uv.lock): the lockfile's lines for the project's dependencies and what they depend on, "not the dev group — a new ruff or pytest bakes nothing differently" (tests/test_step_cache.py, test_the_locked_runtime_is_what_a_bake_runs_with).
  • Today's escapes in bake/: 5 type: ignore, 2 cast(.

The cost nobody sees

Because code_hash hashes a module's bytes, an annotation is a code change to the step cache. Replacing shapely.Geometry in common.py re-keys every step of every tile of every site: the next bun run bake runs them all again, on the maintainer's machine, from raw downloads some of which were not reachable at the last re-bake (plan 022's row: GeoSN's laser scan). A checker in the dev group, its settings in pyproject.toml, a script outside bake/ and stub files re-key nothing.

Commands you will need

PurposeCommandExpected
Environmentcd pipeline && uv sync --locked.venv/ (git-ignored)
Python testsbun run test:pipelinepytest and ruff pass
The checker, as measuredcd pipeline && pyright --pythonpath .venv/bin/python --outputjson bakeJSON, summary.errorCount
A re-bake of one tilebun run bake dresden 33412_5656_2_snneeds the raw data: the maintainer's machine
TypeScript gatebun run verifypasses (only package.json and docs change on that side)

Scope

In scope: pipeline/pyproject.toml, pipeline/uv.lock, pipeline/typecheck.py and pipeline/typecheck-baseline.json (new), pipeline/typings/ (new, stubs), pipeline/tests/test_typecheck.py (new), pipeline/bake/**/*.py (phases 2–4 only), .github/workflows/ci.yml (the pipeline job), package.json (test:pipeline), AGENTS.md (the Commands block), docs/data-pipeline.md (the paragraph on pipeline/tests/).

Out of scope

  • pipeline/tests/ under the checker — the bakes first; say so in the settings (include = ["bake"]).
  • What a step computes. A fix that changes a written file is a STOP.
  • pipeline/bake/step_cache.py's key (see decision B below).
  • A typed record for each handed-over file: 40 hand-written TypedDicts would be a fourth copy of each shape. Plan 092 derives them from the one schema.
  • Stricter modes (strict, unknown-type reports): after the baseline is empty, not before.

Phases

Phase 0 — choose the checker, pin it, prove it re-keys nothing

  1. Decision A (executor, with a note to the maintainer). The baseline above was measured with pyright. Before pinning it, try what is current: a checker must run under uv run from an exact pin in the dev group, on CI's runner as it is (pyright's Python package fetches a Node program when first run — check that the pipeline job needs no new set-up step and no download at run time, or name the alternative that does not), print machine-readable findings, and know shapely. Run each candidate once over bake/ and put its count, its time and what it needs into Findings. Without a reason against it, take the one measured here.
  2. Add it to [dependency-groups] dev with an exact pin, its settings to pyproject.toml (include = ["bake"], the Python version the lower bound of requires-python, the environment .venv, the stub path typings).
  3. Prove the claim of "The cost nobody sees": with the new uv.lock, step_cache.locked_runtime returns what it returned before.

Verify: cd pipeline && uv run pytest -q tests/test_step_cache.py passes; with the lockfile of before saved to a scratch file (git show HEAD:pipeline/uv.lock > "$TMPDIR/uv.lock.before"), uv run python -c "import os; from bake import step_cache as s; print(s.locked_runtime(open(os.environ['TMPDIR'] + '/uv.lock.before').read()) == s.locked_runtime(s.LOCK.read_text()))" prints True; git diff --stat -- pipeline/bake is empty.

Phase 1 — the baseline and the gate

  1. pipeline/typecheck.py (outside bake/, so no step's closure reaches it): runs the checker with JSON output, folds the findings to {file: {rule: count}} — no line numbers, so an edit above a finding moves nothing — and compares with pipeline/typecheck-baseline.json:
    • a count above the baseline, or a file or rule the baseline lacks: exit 1, listing the new findings with their lines;
    • a count below it: exit 1 too, "the baseline is stale — run uv run python typecheck.py --update"; --update writes the file and refuses to raise any count;
    • it also counts type: ignore and cast( in bake/ and holds them at today's 5 and 2. The checker has no baseline of its own that can be trusted to stay; this file is ours, and the comparison is a pure function with a test.
  2. CI: one more line in the pipeline job, uv run python typecheck.py; the same in package.json's test:pipeline. AGENTS.md's Commands block says pytest + ruff + types.
  3. docs/data-pipeline.md says in its pipeline/tests/ paragraph that the bakes are type-checked against a baseline and how to lower it.

Verify: bun run test:pipeline passes; a scratch edit that adds x: int = None to bake/classes.py fails it with that line; the baseline's counts sum to what the checker reports; git diff --stat -- pipeline/bake is empty.

Phase 2 — stubs for what the libraries lack (still no bake edit)

Stub files change no module's bytes, so do them first.

  1. pipeline/typings/scipy/ndimage.pyi (or the narrowest file the checker accepts): label returns tuple[NDArray[np.int32], int] for the call the bakes make — 60 findings.
  2. rasterio: the surface the bakes use and no more (open, transform.xy, windows.Window, features.rasterize, …: list it from grep -rhoE "rasterio\.[a-z_.A-Z]+" pipeline/bake | sort -u); pyogrio.raw.read; laspy's views as the arrays they behave as.
  3. Run phase 1's --update and commit the lower baseline.

Verify: uv run python typecheck.py passes; the baseline no longer has an entry for "is not iterable" on Literal[…] nor for an unknown attribute of a module; git diff --stat -- pipeline/bake is empty.

Decision B (maintainer) — when the bake modules may be edited

Phases 3 and 4 re-key the step cache. Three ways; pick one and write it into Findings:

  • With plan 092's re-bake (recommended in the series). 092 re-bakes every site anyway. Phases 3 and 4 are prepared as a stack of PRs and merged in the days before that re-bake; nothing is baked twice.
  • Now, and pay. Merge, re-bake the affected steps, and take the proof that comes with it: a re-bake after an edit that changed no behaviour writes no file anew (git status data/ is clean; under plan 088, the source set's hash is the old one).
  • Teach the cache. Make code_hash hash what a module does (its syntax tree without annotations, comments and docstrings) and bump VERSION. It changes what the cache trusts, costs one full re-key itself, and belongs in a PR and a test of its own. Not without the maintainer's word.

Phase 3 — the bakes' own annotations

  1. One name for "a geometry" in bake/common.py (an alias of shapely.geometry.base.BaseGeometry), and the 229 uses of shapely.Geometry in annotations take it (every use is an annotation: none constructs or tests one). Where a function needs what only one kind has (exterior 9 findings, geoms 7), say the real class in the signature, or narrow once at the top where another kind really can arrive (if not isinstance(g, Polygon): return …; the bakes already write if g is not None and not g.is_empty).
  2. --update the baseline.

Verify: uv run python typecheck.py passes; grep -rc "shapely\.Geometry" pipeline/bake | awk -F: '{s+=$2} END {print s}' prints 0; uv run pytest -q tests and ruff pass; per decision B, one tile re-baked shows no changed file.

Phase 4 — the findings that are real

In this order, a module a PR where it helps review (roofs.py, furniture.py, water.py are the largest):

  1. None (67). For each: can it be None here? If the code above rules it out, say so where it is ruled out (return early, or raise with the file and the tile in the message — the bakes fail loudly, plan 063). If it can, that is a defect: write the pytest that reaches it first, then the fix.
  2. Possibly unbound (15): bind before the branch or return in it.
  3. Calls (20 wrong counts, 32 overloads) and the 182 that follow: re-run after steps 1–2 and the stubs; fix what is ours (a star over a tuple: give the tuple its length in its type); a numpy overload that is right at run time and wrong in numpy's types stays in the baseline with one line in Findings per kind.
  4. Never silence: no new type: ignore, no cast (phase 1 counts them).

Verify: uv run python typecheck.py passes; the baseline has no entry for reportOptionalMemberAccess, reportOptionalSubscript, reportOptionalOperand or reportPossiblyUnboundVariable; common.py, osm.py, osm_index.py, spec.py, citygml.py, rasters.py, fetch.py, net.py, step_cache.py and __main__.py have no entry at all; every defect found has its test.

Test plan

  • pipeline/tests/test_typecheck.py (new): the comparison alone — a raised count fails and names the file and rule; a lowered one fails as stale; --update lowers and never raises; an unknown rule in the findings is a failure, not a pass. Pattern: pipeline/tests/test_step_cache.py (pure functions on literal inputs, no bake run).
  • Every defect of phase 4 step 1: a test in the module's existing test file that reaches the None.
  • No test runs the checker itself; CI does.

Done criteria

  • bun run test:pipeline runs the checker and passes
  • pipeline/typecheck-baseline.json has no reportOptional* and no reportPossiblyUnboundVariable entry, and none for the ten modules named in phase 4's Verify
  • grep -rc "type: ignore" pipeline/bake | awk -F: '{s+=$2} END {print s}' prints at most 5
  • locked_runtime is unchanged by the checker's pin (test_step_cache.py passes)
  • decision B is recorded, and what was re-baked after phases 3–4 wrote no changed file
  • docs/plans/README.md: this plan's row updated, with the baseline's total before and after

STOP conditions

  • The checker cannot run in CI's pipeline job without a new set-up step or a download at run time, and no candidate can.
  • Adding it changes locked_runtime's text (every step would re-run).
  • A fix changes a file a step writes. The plan is types, not behaviour: report the difference and the tile.
  • More than a handful of the 67 None findings turn out to be real defects in files already published: stop and list them — that is a finding about the data, and the maintainer decides what to re-bake.
  • A phase can only pass by raising a count in the baseline.

Maintenance notes

  • A bump of the checker can move counts in either direction (its shapely stubs travel with it): bump it in a PR of its own and --update there, saying which rule moved.
  • A bump of shapely, rasterio or pyogrio: check typings/ against the calls the bakes make.
  • New bake modules start clean: a file the baseline does not name may have no finding.
  • When the baseline is empty, delete it and the comparison, and let the checker's exit code be the gate.

What the series changes about this plan

Built alone, this plan would finish its fixes whenever they are ready and pay a re-bake for each batch. In the series:

  • Plan 092 rewrites every step's input and output. It needs phase 4's ten modules clean first (they are the readers and writers), and it — not this plan — gives each handed-over file its Python type, from the one schema both languages read. So this plan stops at the writers' signatures and writes no TypedDict by hand.
  • 092's full re-bake is the window for phases 3 and 4 (decision B): 086 puts this plan beside 088, early; only phases 0–2 go then.
  • Plan 088 moves the bakes' outputs out of git. After it, "wrote no changed file" is read from the source set's hash, not from git status.

Findings

(empty — phase 0 writes the checker's choice here, decision B after it)

Previous
Plan 088 [rewrite]: The tileset is a published product, named by a lock file; the web build bakes nothing and data/ leaves git
Next
Plan 090 [rewrite]: A tile's dressing arrives in the layout the GPU draws from
Source on GitHub: docs/plans/089-rewrite-python-types.md
On this page
  • Status
  • Why this matters
  • Current state
  • The cost nobody sees
  • Commands you will need
  • Scope
  • Phases
  • Phase 0 — choose the checker, pin it, prove it re-keys nothing
  • Phase 1 — the baseline and the gate
  • Phase 2 — stubs for what the libraries lack (still no bake edit)
  • Decision B (maintainer) — when the bake modules may be edited
  • Phase 3 — the bakes' own annotations
  • Phase 4 — the findings that are real
  • Test plan
  • Done criteria
  • STOP conditions
  • Maintenance notes
  • What the series changes about this plan
  • Findings