Most software projects accumulate features. Research archives accumulate debt. The distinction matters: features add capability; debt is the difference between what the code says and what the project actually needs. This month we settled a significant account.

Retiring the L7014 pipeline

The US Army L7014 georeferencing pipeline — five API routes, two utility modules, a datum correction library, a propagation algorithm — has been removed from the codebase. Not archived, not commented out. Deleted.

This is the right outcome. The pipeline completed its mission: 500+ sheets georeferenced, datum-corrected from Indian 1960 to WGS84, uploaded to Internet Archive, served over IIIF. The work is done and permanent. The code that produced it does not need to persist in a codebase built around different problems. Keeping it would have meant maintaining it, explaining it to contributors, and treating a completed batch job as if it were ongoing infrastructure. It wasn't. It was a one-time construction project and the building is standing.

The two functions that were genuinely generic — building a W3C Georeference Annotation from IIIF source and GCP corners, and fetching IIIF info.json with retry logic — have been moved to src/lib/iiif/iiifImageInfo.ts, where they belong. Everything else is gone. The codebase is materially smaller and the remaining code is materially clearer about what the project does.

What the cleanup revealed

Removing the pipeline exposed a design assumption that had been quietly propagating errors. The allmaps_id column — the Allmaps annotation identifier that links a map image to its georeferencing — had been copied onto two other tables, label_tasks and footprint_submissions, as a convenience. The idea was that queries involving footprints wouldn't need a join to the maps table to find the annotation URL.

In practice, the copies drifted. When an annotation was updated, the copy wasn't. We found a footprint submission where the map_id foreign key pointed to "Saigon & Surroundings (1882)" but the denormalized allmaps_id pointed to the "Plan Cadastral (1882)" — a different map entirely. The join was cheap. The denormalization was expensive and wrong.

Both copies have been removed. The join is now required. The principle is the same one that governs how we store building footprints: a fact belongs in one place, and dependent data is computed on demand. Pixel coordinates + georeferencing annotation = geographic coordinates. Map UUID + maps table = annotation URL. Neither result needs to be stored separately.

The maps data layer

The more substantial work was rebuilding how maps are described and sourced. The original maps table had nine columns: an ID, name, type, summary, description, thumbnail, featured flag, year, and the Allmaps annotation identifier. This was enough to display a catalog and load an overlay. It was not enough to treat a map as an archival object with provenance.

Fourteen columns have been added, grouped into three concerns.

Source provenance. source_type (the originating institution: BnF, EFEO, Internet Archive, etc.), collection (the human-readable collection name), source_url (the canonical page at the institution — a citable URL, not an API endpoint), and ia_identifier for Internet Archive items specifically. A BnF Gallica map now carries a link to its Gallica record. A researcher citing a building footprint derived from that map can trace the provenance chain: footprint → annotation → map → Gallica record. That chain didn't exist before.

IIIF metadata. iiif_manifest and iiif_image store the manifest and image service URLs. These were previously implicit — reconstructed from the Allmaps annotation on every request. They're now first-class fields, with a corresponding map_iiif_sources table that lets a map have multiple IIIF image sources: the original BnF scan, a re-uploaded copy on Internet Archive for better tile performance, a self-hosted version for maps with restrictive access terms. One is designated primary; the others are ordered fallbacks. The primary source's URLs are kept in sync on the parent row via a database trigger.

Classification and coverage. map_type separates what the map is (cadastral, topographic, city plan, panorama) from the city it depicts. bbox stores the geographic bounding box for spatial queries. status tracks the lifecycle: pending georeferencing → georeferenced → processing → published. The old type column, which contained city names like "Saigon-HCMC", remains for backward compatibility while we migrate.

Multiple IIIF sources per map

The multi-source design deserves a note. This is not about redundancy for its own sake. It's about the gap between where a map lives archivally and where it should be served from operationally.

BnF Gallica serves excellent scans, but the IIIF Image API response times from Paris are slow for tile-by-tile loading from Southeast Asia. For most of our maps, the scan lives on BnF; a IIIF-compatible copy lives on Internet Archive, which has edge caching. The system records both sources, marks one primary, and can switch without losing the provenance link to the original institution.

The same pattern applies to maps with complicated access terms. EFEO maps, for instance, may be freely downloadable but not freely redistributable. Keeping the BnF or EFEO source as the canonical record, with a self-hosted copy for serving, lets the display layer do the right thing without the data layer pretending the institutional relationship doesn't exist.

Metadata backfill: 37 maps, 20 minutes

The new fields are only useful if they're populated. For all 37 maps in the current collection, we ran an automated backfill: fetch the Allmaps annotation, extract the IIIF image service URL, detect the source institution from URL patterns, fetch the IIIF manifest, parse title/creator/date/rights, compute a thumbnail URL, and write everything back. For the 20 BnF Gallica maps, the canonical Gallica page URL is derivable from the manifest URL arithmetically — the ark identifier is the same, the path prefix is different. All 20 were filled in without manual lookup.

The 17 Internet Archive maps require manual source attribution. These were often scanned from physical copies at various institutions and uploaded without structured provenance metadata — the IA manifest might say "1882" as a title. We're going through these individually to establish where the original scan came from. That's the remaining gap.

What changed in the codebase structure

Beyond the maps domain, the reorganization consolidated several modules that had accumulated around the pipeline and early prototyping: a pipeline/ module became two functions in iiif/; a studio/ module became one file in annotate/; a viewer/ module merged into map/; a core/ module merged into utils/. Styles moved from a lib/styles/ directory that required relative imports to src/styles/ with a $styles alias. Seven orphaned components were deleted. The type checker now reports zero errors on the full build.

None of these changes are visible to users. They're the kind of changes that make the next visible change easier to ship.

What's next

The maps data layer is the foundation for the catalog redesign — filter by city, by type, by collection, by coverage area. That's the next front-end milestone. In parallel: the supabase/types.ts generated types need regenerating to reflect the new schema, and the admin map-editing UI needs updating to surface the new fields. We're also mid-way through attributing the 17 IA maps to their original institutions — if you know where the 1863 "Plan du port de Saigon" or the 1882 "Saigon & Surroundings" were first scanned, tell us.