Until this week, adding maps to VMA was a one-by-one job: paste a IIIF manifest URL into the admin form, click "Fetch from Allmaps," fill in the gaps, save. That works fine for the maps we already know about. It doesn't work for the ones we don't — and the colonial Vietnam corpus is scattered across at least a dozen institutions worldwide, most of which have public catalogs but no obvious entry point.
So we built a scout. 3,373 candidate maps are now sitting in a reviewable queue at /admin/scout, pulled from four institutions in one pass, scored for relevance, and ready to bulk-ingest with thumbnails and full Dublin Core metadata.
What "scout" means here
Each source has its own API quirks, but the shape of the work is the same: query for Vietnam-related material, normalize the metadata, dedup against what's already in the VMA catalog, derive a thumbnail, and score the result for relevance. We hit four sources:
- BnF Gallica via the SRU API — 468 records. This also picks up federated partners (Bordeaux 3, Bibliothèques de Paris, Sorbonne, Institut catholique de Paris) for free.
- Humazur (Université Côte d'Azur, Omeka S) — 2,827 records from the Cartothèque ASEMI set (417 pure maps) and the Indochine française set (1,500+ mixed maps and photos, filtered for cartographic items).
- David Rumsey Map Collection at Stanford — 53 Vietnam-tagged records out of ~994 raw hits.
- Library of Congress — 48 records across the Vietnam keywords.
What we deliberately didn't scout: Internet Archive (3,500+ hits but no clean map filter), Cartomundi (their catalog is a JavaScript app with no API — would need a headless browser), Princeton GeoBlacklight (geographically indexed by bounding box, not by keyword — returns zero for "vietnam" as a subject query), Harvard LibraryCloud (endpoint quirks we didn't have time to chase), and HathiTrust (Cloudflare-blocked). Each of those is technically reachable; none was worth the effort relative to what the first four already give us.
Scoring is heuristic, but useful
Not all 3,373 are actually maps worth ingesting. The Indochine française set on Humazur, especially, is full of colonial-era photographs — pagoda interiors, market scenes, railway stations — that mention Vietnamese places in their metadata but aren't cartographic. So we apply a relevance score and a category guess to every record:
- +20 for a Vietnamese place name in the title, subject, or coverage fields
- +20 for a year between 1850 and 1955 (the colonial core), +10 for pre-1850
- +15 for a map-type keyword in the title (plan, carte, topographique, cadastral, atlas, levé)
- −60 for photo-subject keywords without any map term (vue, intérieur, rue, pagode, village, monument, …)
- −50 for world maps with Vietnam as a projection center, −30 for railway plates
The result, across 3,373 records: 758 score ≥40 (likely maps), 100 score 20–39 (borderline), 1,540 score 0–19 (unclear, often non-cartographic atlas pages), and 975 score below zero (correctly flagged photos and railway scenes). A human still has to look, but the high-score bucket is now small enough to review in an afternoon instead of a week.
Thumbnails took most of the engineering
A grid of unlabeled rows is unusable. A grid of thumbnails is fast to scan. So the loader had to produce a thumbnail for each candidate before they ever hit the UI, and that turned out to be the harder problem.
Gallica is easy — every ARK has a stable thumbnail URL at https://gallica.bnf.fr/<ark>/f1.thumbnail. David Rumsey returns thumbnail URLs (urlSize2) directly in its search response. The Library of Congress includes image_url in each result.
Humazur is where it got interesting. Omeka S, the platform it runs on, doesn't expose thumbnails on the item object — they live on the media object that the item points to. So the backfill script has to fetch /api/items/{id}, read the first o:media[@id], fetch /api/media/{media_id}, and pluck out o:thumbnail_urls.medium. Three round-trips per candidate, throttled at 150ms each. On the top-200 high-score Humazur rows, the success rate was 97%. The rest of the 2,400+ Humazur records can be backfilled in the background.
One bug we caught in the process: our initial Humazur scout was building manifest URLs out of media IDs (iiif/<media_id>/manifest) when the correct pattern is item ID (iiif/<item_id>/manifest). Every Humazur manifest URL was returning 404. The loader silently fixes this at insert time so the stored URLs are correct.
The review UI
The page at /admin/scout is a thumbnail grid with filters along the top (status, source, category, minimum score, title search), facet counts that update as you filter, and per-card Approve / Reject / Revert buttons. Selecting multiple cards lets you bulk-approve, bulk-reject, or — for already-approved candidates — bulk-ingest as draft maps rows. Each ingested map carries an extra_metadata.scout_candidate_id reference so we can always trace a catalog row back to the source record that produced it.
The ingest itself reuses the same metadata model we just standardized in last week's holding_institution work: each ingested row gets source_type mapped from the holding institution (Bibliothèque nationale de France → bnf, David Rumsey → rumsey, others → other), a populated holding_institution field separate from collection (which is the sub-collection at the holder), and the IIIF manifest URL ready for further enrichment via the "Fetch metadata from IIIF manifest" button in MapEditModal.
Why this matters for the project
VMA's core constraint is volunteer attention. Every minute spent hunting for a candidate map in BnF's catalog is a minute not spent georeferencing, OCR-ing, or annotating. By front-loading discovery into a tool that produces a curatable queue, we shift the work from "find one map at a time" to "review a batch and approve in bulk." The first pass already surfaces roughly twice as many viable candidates as VMA's current catalog of 100 maps — and that's before we extend the scout to Vietnamese-language sources (the National Archives have a digital catalog, and so does Hanoi's Institute of Sino-Nôm Studies).
It's also a small piece of infrastructure that scales: each new source is one normalize function and one entry in the scout runner. Adding a fifth source — say, EFEO's Bibliothèque Numérique when we get around to it — is an afternoon of work, not a rebuild.
What's next
Two things follow from this:
- Curate and ingest the high-score bucket. 758 candidates is a real afternoon's work, but a quiet one — most are obvious approves. Once they're in the catalog as drafts, the existing pipeline (georeference → OCR → annotate) picks up from there.
- Extend to Vietnamese-language and regional sources. The biggest gap right now is anything held inside Vietnam. EFEO's collection, the Institut d'Asie Orientale at Lyon, and any digitized holdings from Hanoi or Ho Chi Minh City archives would push the corpus toward 5,000+ unique candidates.
If you've been waiting for a way to help VMA without doing pixel-level work, scout review is exactly that — fast, judgment-based, and visible in its impact. Drop us a note and we'll get you admin access to /admin/scout.