Contribute

The correction files

From corrections/README.md, rendered as committed.

Files

volumes.json — per-field value corrections

[
  {
    "volume": "v_5c7be0913a2d",
    "field": "release_date",
    "value": "2013-10-13",
    "precision": "day",
    "source_url": "https://kodansha.us/volume/vinland-saga-1/",
    "reason": "publisher page; Wikipedia had the reprint date",
    "checked": "2026-09-04"
  }
]

Required: volume, field, value, source_url, checked. Optional: precision (for a date), reason.

volume is an OpenTome volume id (v_…, volumes.tome_id in the published artifact) or an ISBN-13, which is resolved to the volume that carries it — key on the id when the same ISBN is on more than one volume. Fields: release_date (with optional precision), isbn13, page_count, title, cover_url (a picked cover: stored as a correction claim, which the exporter prefers over every ISBN lookup; the value "none" removes a wrong cover when no source has the right one — an interior page, another market's cover — because no cover beats a wrong one: the reader falls back to its own).

Each correction is written to the override table — where tier2/resolve.py already ranks it above every source, so the confidence layer reports it as manual_override — and onto the volume row, which is what the export reads. Writing only the first would produce a correction that is visible in the provenance and absent from the artifact.

aliases.json — extra names for a release line

[
  {
    "line": "rl_13affd6972b3",
    "alias": "Vinland Saga Deluxe",
    "source_url": "https://kodansha.us/series/vinland-saga/",
    "reason": "the name the publisher prints on the spine",
    "checked": "2026-09-04"
  }
]

Required: line, alias, source_url, checked.

line is an OpenTome release-line id (series.tome_id in the published artifact). Aliases are added to that line only, never fanned out.

Folder-name bridges belong here — a library whose folder is misspelled ("Jujustu Kaisen") is a fact about that library, and recording it as a correction keeps it out of the catalogue's own naming.

aliases.json — curated removal ("remove": true, same file)

For a specific alias the export's own fan-out generated that is actually a volume or story-arc title, not a name anyone uses for the whole line: My Hero Academia's alias list includes the bare fragment "My Hero" (a head-split of the alternate title "My Hero: Ultra Impact"), which is ambiguous and belongs to no reader's folder name.

[
  {
    "line": "rl_82d4b3ac7f3f",
    "alias": "My Hero",
    "remove": true,
    "source_url": "https://en.wikipedia.org/wiki/List_of_My_Hero_Academia_chapters",
    "reason": "a head-split fragment, too generic to stand alone as a series name",
    "checked": "2026-09-23"
  }
]

Required: same as an addition (line, alias, source_url, checked), plus remove: true. This is a curated, exact-string removal — not a rule. An earlier attempt automated this (drop any alias whose normalized form equals a volume title anywhere reachable from the line) and was reverted: of 360 aliases it dropped, only about 30 were actually bad; the rest included every native-script series name whose ASCII-only ambiguity check made it collapse to nothing, and a handful of real series and licensed titles (see HANDOFF.md and .superpowers/sdd/2026-09-23-followups/review.md). A removal entry targets one exact string on one exact line; it never infers a second one.

export/to_mangarr.py applies every removal last, after every other alias source (the auto-generated fan-out and every aliases.json addition), and deletes both the string as written and its normalize()-d form — the same two rows an addition's variants() would have inserted for it, since Mangarr's own lookup queries the normalized form. It raises if a removal matches nothing: the export regenerates every alias from scratch each run, so a removal that deletes 0 rows means the string or the line is stale, the same "fails loudly" contract every other correction has.

lines.json — a whole edition the sources do not carry

For an edition that is on no source at all: the English Mushoku Tensei: Roxy Gets Serious (Seven Seas, 12 volumes) is absent from the English Wikipedia list, Open Library has never seen its ISBNs, and the publisher's site refuses robots — so the pipeline cannot know it exists, and a person reading the publisher's page can.

[
  {
    "work": "w_179929c7bc15",
    "market": "EN",
    "medium": "manga",
    "name": "Mushoku Tensei: Roxy Gets Serious",
    "publisher": "Seven Seas Entertainment",
    "volumes": [
      {"number": "1", "isbn13": "978-1-64505-XXX-X", "release_date": "2020-10-06", "contains": [1]},
      {"number": "2", "isbn13": "978-1-64505-XXX-X", "release_date": "2021-02-09", "contains": [2]}
    ],
    "source_url": "https://sevenseasentertainment.com/series/mushoku-tensei-roxy-gets-serious/",
    "reason": "Seven Seas edition; not on the en list article, publisher blocks robots",
    "checked": "2026-09-04"
  }
]

Required: work, market, medium, name, volumes, source_url, checked. Optional: publisher, reason, origin_line.

work is the OpenTome work id (series.tome_work_id in the published artifact, on every sibling line). market is one of JP EN FR DE KR IT ES BR CN TW HK. Every volume needs a number; contains lists the original-market volume numbers each volume collects — [1] for a straight translation, [1, 2] for a 2-in-1 — and is how the cross-market mapping and the status rule see the edition. Per-volume fields: isbn13, release_date, page_count, title, cover_url. An ISBN whose registration group is another market's (a 978-4 on an English line) is rejected: that is the single most common way a wrong row gets in.

The line receives exactly the id the pipeline would give the same edition, so if a source later carries it the two meet instead of duplicating, and the consumer's series id never changes.

origin_line (optional): pins this new line's orig_series_id to an exact, already-cataloged release line, for the case the export's own name-key matching (export/to_mangarr.py's origin_line()) cannot pair the two. export/to_mangarr.py matches a licensed line to its origin-market counterpart by an EXACT string match on the two lines' names — which fails when a work has more than one line sharing (work, medium, market): Mushoku Tensei's JP manga has both the main serial and a "Roxy Gets Serious" spin-off, and the JP spin-off's own line_name claim is not Japanese at all — it is the French string cross-parsed from the FR Wikipedia table ("Mushoku Tensei : Les Aventures de Roxy"), the same string the FR spin-off line carries (which is why FR pairs with JP correctly today). A new EN line named "Mushoku Tensei: Roxy Gets Serious" cannot exact-match that string, so without a pin it silently falls back to the JP work's MAIN manga line — the new line then reads 12 of 25 volumes against a still-running series and exports stalled instead of completed. Value: an existing release line id (rl_...), which must belong to the SAME work and the SAME medium, and sit in an origin market (JP KR CN TW) — tier2/corrections.py validates all three and refuses a stale, cross-work, cross-medium or non-origin-market target the same way a medium/market override refuses a stale line. It is also used for composition.ref_line_id (the cross-market volume mapping) in place of the same naive "any origin-market line for this work+medium" query the whole-edition entry already relies on, which has the identical ambiguity when more than one origin-market line exists.

lines.json — medium override (a narrower entry shape, same file)

For a line the pipeline already has, but has classified as the wrong medium. The case: Denma is a Korean webtoon (Naver, printed and licensed as a manga), but every one of its three lines was tagged plain manga upstream (no medium hint applies to it), which made the origin-market picker compare release dates instead of trusting the medium — and the Japanese edition's earlier print date won, so the Korean line resolved to its own Japanese translation as "the origin".

[
  {
    "line": "rl_266a70c679ed",
    "medium": "manhwa",
    "source_url": "https://en.wikipedia.org/wiki/Denma",
    "reason": "a Korean webtoon tagged plain 'manga' upstream; see the other two entries for the same work",
    "checked": "2026-09-23"
  }
]

Required: line, medium, source_url, checked. This entry has no volumes key at all — that absence is exactly what distinguishes it from a normal lines.json entry above (a normal entry always requires a non-empty one). Do not add a work, market or name to one of these; it targets an EXISTING line by its own id, not a natural key.

line is an OpenTome release-line id (series.tome_id in the published artifact) — not the work id, and not something you compute: copy it from the pipeline or the published artifact. medium is one of the pipeline's own names (manga, light_novel, manhwa, manhua, novel, artbook — tier0/release_lines.py's MEDIUM_HINTS; not webtoon, which tier0 always canonicalises to manhwa, so no line ever carries that value).

Retag every line of the work that should move together, not just one. The origin picker groups a work's lines by (work, medium) and only ever compares candidates inside the same group: retagging only the Korean line would split it into a group of its own (an origin of one market is trivially itself) and leave the English line still resolving to the Japanese one. Denma needed all three of its lines (Japanese, English, Korean) retagged together.

Applied as a plain UPDATE release_line SET medium = ... — the line's id never changes, so a rebuild that re-detects the same medium upstream (nothing changed there) recomputes the same id and the correction keeps applying cleanly; if the line is ever renamed or reclassified upstream, the id changes and this correction fails loudly (STALE CORRECTION) instead of silently attaching to the wrong line.

excluded.json — a whole work that should not be in the catalogue at all

For a work that entered through a source but is not in scope: The Walking Dead (comic book) is a US comic that came in through an English Wikipedia list-of-volumes page, not manga/light-novel/manhwa/manhua.

[
  {
    "work": "w_5e8c089527e9",
    "source_url": "https://en.wikipedia.org/wiki/The_Walking_Dead_(comic_book)",
    "reason": "a US comic; out of scope",
    "checked": "2026-09-23"
  }
]

Required: work, source_url, checked. Optional: reason.

work is the OpenTome work id (series.tome_work_id in the published artifact, on every sibling line) -- the same identity a medium override keys its line on, and stable the same way: a rebuild that reprocesses the same Wikipedia article recomputes the same id, so the exclusion keeps applying.

Applied before every other correction (tier2/corrections.py's apply_exclusions, stage 5b, first): every release line the work owns, their volumes, compositions, and every claim/override/external_id on any of those entities or the work itself is deleted outright. Aliases and volumes "go with the line" -- once the release_line row is gone there is nothing left for a later stage, including the exporter's own alias fan-out, to read. Be certain: this is broader than a single line, and removes every market edition of the work at once. Do not use it on a work that is legitimately in scope just because one of its lines is bad -- Arrietty (Comics), a real Japanese Studio Ghibli film comic that also came in through a list-of-volumes page, stays.

check() accepts a work that resolves against the published artifact's series.tome_work_id as usual, OR one recorded in the artifact's meta.excluded_works (a JSON array the exporter writes for every exclusion it applied): once a publish actually removes the work, series no longer carries it, and treating that as a stale correction would fail every future corrections PR for a correction that is working exactly as intended.

lines.json — market override (a third entry shape, same file)

For a line the pipeline has correctly found but tagged the wrong MARKET. The case: Denma's line tagged ja is not a Japanese print edition at all -- its titles are hangul and its numbering is the Naver webtoon's own episode-arc list (2010-01 to 2012-01), i.e. the same Korean web serialization the ko line's print volumes collect, not a translation of it.

[
  {
    "line": "rl_266a70c679ed",
    "market": "KR",
    "source_url": "https://en.wikipedia.org/wiki/Denma",
    "reason": "hangul titles, Naver webtoon episode-arc numbering -- the same Korean serialization the ko line collects, not a JP translation of it",
    "checked": "2026-09-23"
  }
]

Required: line, market, source_url, checked. Like the medium override, this entry has no volumes key -- and, to keep the two override shapes unambiguous, no medium key either: an entry with neither volumes nor medium but a market is a market override, checked against the pipeline's own MARKET_LANG table (schema/load.py). It targets an EXISTING line by its own id; do not add work, medium or name.

Applied as a plain UPDATE release_line SET market=?, language=? (language follows MARKET_LANG[market] automatically) -- the line's id never changes, so a rebuild that re-detects the same market upstream (nothing changed there) recomputes the same id and the correction keeps applying cleanly. Retagging a line's market can change which line the origin picker treats as another line's counterpart (export/to_mangarr.py's origin_line, matched by exact name within a market) -- read the result, do not assume it.

lines.json — link_work: a reviewed library line (a fourth entry shape, same file)

For a line built from a library record (DNB, and in the KR/CN round LoC and BnF) that the linker could not place with confidence and sent to review (build/dnb-review.tsv, build/krcn-review.tsv): a person checks which work it is, and the line ships under that work.

[
  {
    "line_key": "dnb:1380595053",
    "link_work": "w_0123456789ab",
    "source_url": "https://d-nb.info/1380595053",
    "reason": "Bastard (Carnby Kim) is the Korean manhwa",
    "checked": "2026-09-28"
  }
]

Required: line_key, link_work, source_url, checked. Optional: reason. An entry with a link_work key is this shape; it must not also carry volumes, medium, market, line or origin_line (--check refuses it).

line_key is the library line's NATURAL key, copied from the review file's dnb_key column (build/dnb-review.tsv) or line_key column (build/krcn-review.tsv): dnb:<IDN>, loc:<LCCN> or bnf:<ark>. It is not an rl_ id: a library line's id is not known before it links. link_work is the OpenTome work id (series.tome_work_id in the published artifact).

Applied by the linking stages, 3e (tier0/build_dnb.py) and 3f (tier0/build_krcn.py), not by 5b: linking happens before the corrections stage, and 5b's apply_line_corrections skips these entries. The line gets role linked, via correction, under the named work. The scope rules still apply after it: a line corrected onto a Korean/Chinese work in the German JP round is still out_of_scope. A line that shares ISBNs with a Wikipedia line (merged / sibling) follows the ISBNs, not the entry; the build log prints one link_work <key>: line is merged|sibling by ISBN under <work> line for it, marked CONFLICT when that work is not the entry's. A link_work naming a work the catalogue does not have prints STALE CORRECTION at build time and the line stays as the linker left it.

Lifting the KR/CN JP guard. Stage 3f sends a KR/CN line whose work is Japanese (a JP line and no KR/CN line) to review even when a link_work names that work. When a person has checked that the work IS Korean / Chinese (Black God / Kurokami: Korean creators, first published in Japan), the entry adds "override_jp_guard": true and a non-empty "why"; 3f then links the line and lists it in build/krcn-report.json (jp_guard_overrides). --check refuses an override that is not true, one without a why, and one on an entry that is not a link_work entry. The German JP round's out-of-scope rule is not the JP guard and is never lifted.

export/test_artifact.py checks every entry whose line_key the build has: the line must ship under the named work -- linked, kept, or merged / sibling by ISBN (a merged line counts under its Wikipedia line's work). Under any other work, or not shipped at all (out of scope, absorbed, still in review), it fails. A line_key the build no longer has (DNB can renumber a set) is reported as stale, not failed. --check refuses a key that is not a library key, a link_work that is not a work id, a key corrected twice, an entry that also carries another shape's keys, and a work that is not in the published artifact.

lines.json — cluster_with and review: before a library work is created (fifth and sixth entry shapes, same file)

Stage 3f clusters the library lines the linker left unlinked (the pooled lines) and creates a work from a cluster that passes every bar (docs/krcn-design.md §9). These two shapes act on pooled lines before any work is created. A person writes them from the lists a build writes: build/krcn-duplicates.tsv and the containment_created list in build/krcn-report.json.

[
  {
    "line_key": "dnb:1393101849",
    "cluster_with": "bnf:ark:/12148/cb46807564x",
    "source_url": "https://d-nb.info/1393101849",
    "checked": "2026-10-01",
    "why": "the German and French editions of Who Can Define Popularity?"
  },
  {
    "line_key": "dnb:1386543211",
    "review": "may be Who Made Me a Princess under another title; not settled",
    "source_url": "https://d-nb.info/1386543211",
    "checked": "2026-10-01"
  }
]

cluster_with joins the two lines' clusters into one work, anchored on the joined cluster's comic line with the lowest key (bnf: sorts before dnb:, so the French line wins over the German one) and titled with that line's name. Use it for a German and a French cluster of one series: they cluster per library by design, because a romanised original title only joins lines of one library. Required: line_key, cluster_with, source_url, checked, why.

review sends the line's whole cluster to review (role review, reason correction: <text>), so no work is created. Use it for a suspected duplicate you cannot settle. Required: line_key, review (the reason, a non-empty string), source_url, checked. It is refused for a cluster that holds a published line: a published line never goes back to review (use link_work).

Rules for both:

• Every key is a library line's natural key (dnb:<IDN>, loc:<LCCN>, bnf:<ark>), as in link_work.
• The key must be a line the linker left unlinked in this build. A key that is linked, merged / sibling / adopting by ISBN, already in review, or deferred to the German JP round stops stage 3f with a correction error naming the key and its role. Exception: a published (carried) line the linker now places elsewhere is skipped with a printed CLUSTER_WITH / REVIEW SKIPPED line (a weekly build never stops on it). A key the build does not have prints STALE CORRECTION and the build goes on.
• A cluster that IS an existing work under another title is not this shape: give each of its lines a link_work.
• An entry carries one shape only (no link_work, volumes, medium, market, line or origin_line beside it), and a line_key appears in at most one link_work / cluster_with / review entry.
• Applied by 3f; 5b's apply_line_corrections skips them; --check validates the shape. There is nothing to resolve against the published artifact, because the keys are natural keys.
• Keep a cluster_with entry after the publish; it is not cleanup. Without it the two halves it joined are decided as separate clusters again (each frozen to the one work id).

A pair in build/krcn-duplicates.tsv that is NOT a duplicate gets a verdict row in export/fixtures/krcn_lift_duplicates_reviewed.tsv (work, other, not-a-duplicate, why). That fixture is the only file the build's duplicate gate reads; it needs a verdict for every title row at ratio 0.9 or more and every AniList-id row that no correction settled.

anilist.json — a hand-checked AniList id for a line

For a line export/resolve_anilist.py binds to the wrong AniList entry, where no safe rule can tell the right one from the wrong one. The case: the English Worst (3 volumes) has two AniList entries titled exactly "Worst" on its search page — 31741 (33 volumes) and 147044 (4 volumes). The volume rule rejects the 33-volume one (more than 4x the line) and binds the 4-volume one, a different work. The line's own orig_series_id is the Japanese Worst line, 33 volumes: the right entry is 31741, and a person can see that where the resolver cannot. Mangarr takes the series' poster and synopsis from this id and pins it on every add, so a wrong one is worth a correction.

[
  {
    "line": "rl_1cc5f2e75d35",
    "anilist_id": 31741,
    "source_url": "https://anilist.co/manga/31741",
    "reason": "orig_series_id is the 33-volume JP 'Worst'; 31741 has volumes: 33, the resolver's 147044 is a different 4-volume work",
    "checked": "2026-09-24"
  }
]

Required: line, anilist_id, source_url, checked. Optional: reason.

Unbind pin (2026-10-01). When the resolver bound a line to a wrong entry and AniList has no entry for the work at all, write "anilist_id": null (or "none"). --anilist writes NULL over the resolver's pick, and export/resolve_anilist.py reads the pins first and never searches that line again. source_url is the AniList search or page you checked; say in reason why no entry fits. A KR/CN binding is written to every main comic line of the work (the French and the German line of a lifted work share one id), so a wrong binding needs a pin, or an unbind pin, on EACH of those lines: a pin on one line leaves the resolver's id on the others.

line is an OpenTome release-line id (series.tome_id in the published artifact), the same key aliases.json and the lines.json overrides use: it is stable across rebuilds and names exactly one line, where a series name does not (many lines share one). anilist_id is a JSON integer (not a string), or null / "none" for an unbind pin; source_url is the AniList entry that was checked. Put the evidence in reason — which catalogue fact the entry matches (volume count of the origin line, author, dates).

Applied in stage 8a, right after export/resolve_anilist.py, by python3 tier2/corrections.py --anilist build/manga-metadata.sqlite.new: a plain UPDATE series SET anilist_id on the exported artifact, so it overrides whatever the resolver picked (the resolver only ever fills NULL ids). A pin whose line is not in the artifact stops the build (STALE CORRECTION), and export/test_artifact.py fails if a pinned line carries any other id.

Clean room: the pin is an id, the same lookup key the resolver writes — no AniList title, synonym or description enters the artifact through it.

Covers: build/anilist-covers.json (the site's fallback cover for a line with no ISBN-keyed volume cover) is filled last in stage 8a, by export/resolve_anilist.py --covers-only after the pins land, so a pinned id gets its fallback cover exactly like a resolved one. The display-only fallback (--display, series.display_anilist_id; see docs/schema-v1.md) runs between the two, so a pinned line never keeps a display id and a display id gets a cover too.

Checking before you open the pull request

Every key is resolved against the published artifact, not against a database you would have to build. Download it, then:

curl -fsSLO https://github.com/opentomedb/mangarr-metadata/releases/download/metadata/manga-metadata.sqlite
python3 tier2/corrections.py --check corrections/ --artifact manga-metadata.sqlite

It exits 0 and prints corrections check ok when every file is well-formed JSON, every entry has its required keys, every field and market is one the pipeline accepts, and every id or ISBN resolves. Otherwise it names the file, the index and the reason — STALE CORRECTION for a key that is not in the artifact — and exits 1. It never writes anything.

Opening a pull request that touches these files runs the same check automatically (.github/workflows/corrections-check.yml), plus the parser's unit tests. A merged correction lands in the next build.

Finding an id

Against the published artifact:

sqlite3 manga-metadata.sqlite "SELECT tome_id, tome_work_id, name, language, medium
  FROM series WHERE name LIKE 'Vinland%'"

sqlite3 manga-metadata.sqlite "SELECT v.tome_id, v.volume_number, v.isbn13, v.release_date
  FROM volumes v JOIN series s ON s.gcd_series_id = v.gcd_series_id
  WHERE s.tome_id = 'rl_13affd6972b3' ORDER BY v.volume_number"

Ids are a public contract — never reused, never re-keyed — so a correction keyed on one keeps applying across rebuilds. An id that no longer exists is reported by the check and by the loader as a stale correction rather than ignored.

How the catalogue is built

tier0/rebuild_all.sh runs the whole pipeline, in order:

1. Unit tests — a rebuild on a broken parser is worse than no rebuild.
2. Work identity — Wikipedia language links become cross-language work classes.
3. Corpus — volume lists from the English, French and German Wikipedias, plus the main article's titles, status, publishers and relations; then the German market from the Deutsche Nationalbibliothek (CC0): print manga and light novels of Japanese origin, linked to their works by title and author, only confident links exported.
4. Enrichment — every ISBN checked against openBD (JP), Open Library (EN, FR) and BnF (FR), each value stored as a claim with its source and licence.
5. Clean — date semantics (_precision, _type), malformed ISBNs.
6. Corrections — corrections/*.json applied, ranked above every source.
7. Resolve — claims become values with a confidence; disagreements are recorded, not averaged.
8. Audit — exits non-zero on any defect.
9. Export — the consumer-shaped manga-metadata.sqlite, with AniList ids for the English lines and ids carried forward from the previous build.
10. Gates — the artifact contract test, then a replay of a real library's series matching against a private library fixture (not in git; CI reads it from a secret, and its per-series report never leaves the build machine); a coverage failure fails the build before the new artifact replaces the old one.

Every source response is cached under .cache/ (never committed — it is raw third-party data), so a re-run costs minutes and zero requests; a cold build re-fetches everything under each source's rate limit and takes hours.

The build runs in GitHub Actions on a weekly schedule and on demand. A scheduled run builds and gates; publishing is a separate, human decision — a dispatched run with publish set, which runs export/publish.sh: it refuses any artifact whose label is not opentome-YYYY-MM-DD or whose aliases came from anywhere but this pipeline, creates the per-build release, and re-points metadata at it.

tier0/          Wikipedia extraction (one dialect entry per wiki language), DNB (German market), work identity, the rebuild script
tier1/          enrichment and cross-verification against independent sources
tier2/          clean, corrections, resolve (claims -> values + confidence), audit
schema/         schema.sql + loader, per-field provenance
export/         the consumer artifact, AniList ids, contract test, measure gate, publish
corrections/    hand-checked facts, as data
docs/           findings -- every design decision traces to one

From README.md, rendered as committed.