The Evidence Was There. The Parser Missed It.

The previous episode separated a skill’s internal consistency from its grounding and practical effect. The next problem was smaller in code but just as important to the publishing workflow: evidence could be present in a claim ledger while the parser failed to recognise the table carrying it.

I wanted factual claims in the blog to lead back to inspectable sources. That requires more than writing a references section. The publishing check has to recognise the author’s ledger, extract the concrete claim tokens the linter expects, and preserve the association between each claim and its source. A mismatch at the first step can make the later system judge a different input from the one I reviewed.

The record can be present while the sorting mechanism expects another heading. Conceptual candidate hero and placement.

This chapter covers July 8, 2026. It reconstructs the ledger-parser repair from the dated handover and the historical parser, linter and tests at revision a9918f56. I owned the publishing requirement and review; agents assisted with implementation and checking. Reported CI and verifier outcomes remain historical. The new examples and checks are isolated parser probes, not a replay of the original publishing run or a fresh assessment of every article’s truth.

The gate needed an adapter, not a second truth engine

The ledger was a small bridge between prose and a concrete-claim linter. An author could write a sentence containing several numbers, identifiers or other checkable tokens in one claim cell, then attach a source in the next cell. The adapter expanded the prose into the units the linter actually compared.

That separation is useful. The author can explain a claim in ordinary language; the checker can operate on a more precise representation. It also creates a failure boundary. If the table is not recognised, the linter cannot evaluate the supplied source associations as intended. A correct downstream comparison does not repair an empty or misread adapter result.

The repair did not add a model to decide whether evidence sounded plausible. It changed a deterministic Markdown parser. That narrow scope matters because accepting a table heading is not the same as accepting the source’s authority. A file path might exist and still contain an outdated design, a proposal or an irrelevant observation.

Stage	Input	Output	Separate question still open
Table recognition	Markdown lines	Recognised claim/source rows	Is this a supported ledger shape?
Claim expansion	A claim prose cell	Concrete tokens	Did extraction capture the intended claim?
Source association	A token and source cell	Linter evidence input	Does that source actually support the token?
Editorial review	Claim, evidence and scope	A bounded publication decision	Are important omissions and qualifications visible?

The table also explains why an apparently minor spelling variation could matter. The adapter was the boundary connecting an author-friendly format with the checker’s stricter input. Improving that boundary should make the author’s meaning easier to represent without silently changing the evidence standard.

A small vocabulary of real equivalents

The retained implementation recognises Claim as the first column and a bounded vocabulary for the second: Source, Sources, Verification source, Verification or Evidence. Comparisons use lower-case text after the surrounding whitespace has been stripped.

The key word is bounded. The code does not accept any second column just because the first one says Claim. In particular, a status field such as Verified? is not an evidence location. It may say that someone considers a claim checked, but it does not tell the reader what they could inspect.

ledger.py · header predicates · platform a9918f56

def _is_claim_header(cells: list[str]) -> bool:
    return len(cells) >= 2 and cells[0].lower() == "claim"


def _is_claim_source_header(cells: list[str]) -> bool:
    return _is_claim_header(cells) and cells[1].lower() in _SOURCE_SYNONYMS

Explain this code line by line

• The first helper requires at least two cells before reading either column.
• It compares the first cell, in lower case, with the expected claim role.
• The second helper keeps that requirement and checks the source-role vocabulary for the second cell.
• A supported synonym is therefore accepted only inside a claim-header shape. A status label is not added to the vocabulary.

The short predicate contains a useful design decision. The first-column role must agree as well as the source-column label. An unrelated numerical table can contain a column called Evidence without becoming a claim ledger. Conversely, a claim ledger with an unsupported second heading should not disappear quietly into the same outcome as no ledger at all.

A bounded set of equivalents is different from accepting every shape. Conceptual illustration of the supported-header vocabulary.

The handover says the synonyms came from the real ledger shapes under investigation. That is a more defensible rule than adding an open-ended list of plausible English words. A future new heading can be considered deliberately, with a source example and a test, instead of broadening the parser until unrelated content starts to match.

It is also important not to confuse a tolerant header rule with a complete Markdown implementation. The inspected parser works line by line on pipe-delimited rows. It does not claim to implement every table feature a full Markdown renderer can display. Escaped pipes and richer syntax need explicit support and tests if the workflow begins to rely on them.

Follow one source through a synthetic ledger

Here is a deliberately small Markdown table. The path is an illustrative label, not a claim that a benchmark file with that name existed in the original run:

| Claim | Verification source |
|---|---|
| The controlled probe returned 12 records. | retained-probe.json |

The parser first sees the header and waits for a valid separator. The next line confirms the table shape. The data row then yields a claim cell and a source cell. The later linter adapter can extract a concrete token such as 12 from the prose and associate it with retained-probe.json.

Replace Verification source with Evidence and the supported intent is the same. Replace it with Verified?, and the table now supplies a status-shaped column instead of the source role the adapter needs. The parser should raise a clear error rather than pretend that this is a valid source mapping.

The distinction remains even if the status cell contains yes. A yes is a review assertion. It is not the underlying observation. The source may be a report, test result or other artifact, but it has to be named and then assessed for relevance. The parser’s job is to preserve that opportunity, not declare the claim true.

Follow the mechanism

Pause flowExpand diagram

Close expanded diagram

This diagram follows the ledger boundary, not the full publication workflow. The error outcome is intentionally visible. A useful failure should identify an unsupported claim-table header, rather than leave the author guessing whether the entire article lacks evidence.

Why the separator was part of the meaning

The parser maintains two states. After a recognised header it waits for a separator row. Once the separator is valid, it accepts subsequent rows as data until a non-table line resets the state. A header-looking sentence or an isolated pipe row is therefore not enough to enter a recognised ledger.

That stateful detail protects more than formatting. Without it, arbitrary text that happened to contain Claim could be treated as the start of evidence data. A deterministic contract needs enough structure to distinguish the author declaring a table from the author merely mentioning a word.

The error detector mirrors that idea. It looks for a claim header with an unsupported source column, then confirms it with a separator before reporting the problem. The presence of unrelated tables elsewhere does not satisfy that condition.

ledger.py · _unrecognised_source_header_present, state loop · platform a9918f56

awaiting_separator = False
for line in markdown.splitlines():
    cells = _parse_table_cells(line)
    if cells is None:
        awaiting_separator = False
        continue

    if _is_claim_header(cells) and not _is_claim_source_header(cells):
        awaiting_separator = True
        continue

    if _is_separator(cells):
        if awaiting_separator:
            return True
        continue

    awaiting_separator = False

return False

Explain this code line by line

• The flag begins false, so an ordinary table or paragraph is not already an error.
• Each line is parsed into cells; a non-table line clears the pending header state.
• A Claim header with an unsupported source label sets the flag and waits for the next structural evidence.
• A valid separator while that flag is set confirms a malformed claim-table header and returns true.
• Other content clears the pending state. Reaching the end returns false.
• The caller converts true into a descriptive ValueError; this helper does not decide whether a factual claim is true.

A naive replacement would have been “raise an error whenever a non-empty document produces zero ledger rows.” That would be easier to explain but wrong for articles with no claim ledger, ordinary numerical tables, or drafts without concrete claims. The implemented detector asks a more precise question: did the author appear to supply this kind of table, but use a source heading the parser cannot interpret?

That precision keeps the gate useful. A stricter error is justified when the author has declared a claim ledger with an unsupported contract. It is not justified merely because every document does not contain the same structure. A check that turns every absence into a failure encourages authors to add meaningless boilerplate just to make it stop.

Extraction still had to preserve the source relationship

After table recognition, the adapter iterates the recognised rows. Empty source cells are not expanded into evidence entries. Non-empty sources are attached to each concrete claim token extracted from the corresponding claim cell.

ledger.py · ledger_rows_for_linter, row expansion · platform a9918f56

rows: list[dict[str, str]] = []
for claim_cell, source in parse_ledger_tables(markdown):
    if not source:
        continue
    for token in claim_linter.extract_concrete_claims(claim_cell):
        rows.append({"claim": token.text, "source": source})
return rows

Explain this code line by line

• The output list starts empty.
• The loop receives paired claim/source cells only from recognised tables.
• An empty source is skipped instead of turned into a source assertion.
• The claim linter extracts concrete tokens from the prose cell.
• Each extracted token gets the same supplied source association.
• Returning these rows supplies the later linter; it does not independently verify the source content.

This is a mapping operation, not a fact check by itself. A claim cell can contain more than one concrete token, and the same source label can be associated with all of them. That source still needs to support each relevant assertion. One report containing a number somewhere is not enough to justify every number in the sentence.

The adapter also exposes an important ambiguity in ordinary prose. A source cell may describe a historical report, an independently inspected implementation or an intended design. Those are different kinds of support. The article’s ledger needs to say which it means; the parser should not silently promote all of them to observed runtime truth.

The new draft checks therefore exercise the transformation with synthetic tables and inspect the retained source. They do not treat “a row was emitted” as evidence that the row’s claim is true. This is the same boundary the blog now uses for its own claim ledger: mechanical structure and editorial truth assessment remain separate.

What the historical checks established

The July handover reports the repair merged through MR !312 and identifies both the branch and merge revisions. It reports 53 passing content-gate tests and a broader slice of 86 passing tests with two skips. Those are historical test results. The retained source and tests make the intended cases inspectable, but this reconstruction does not substitute a fresh local parser probe for the entire original CI environment.

The most meaningful cases are quite specific: a supported synonym is accepted; a real claim table with an unknown source header fails loudly; an unrelated table does not trigger that error; and existing supported ledgers continue to parse. A parser can pass hundreds of other tests while still missing one of these boundary cases.

A clear empty result and a recognisable contract error should not look identical. Conceptual artwork; not a photograph of the publishing system.

There are useful remaining limits. A case-insensitive heading check does not handle every spelling or Unicode variation. The separator rule requires at least three hyphens per cell in the inspected implementation. A simple row splitter can be confused by unsupported pipe syntax. None of these observations means the patch failed its stated scope. They define the supported language so that the next author does not infer a general Markdown parser from a narrow adapter.

The publication boundary remains broader still. A parser pass cannot prove that an agent used the cited source, that a historical report is accurate, that a screenshot belongs to the described day, or that a claim’s scope matches its evidence. Those are separate editorial and provenance obligations. The adapter repair helps the gate see the intended evidence; it does not complete those obligations automatically.

Before parsing, the ledger still had to be the right one

The earlier July 8 handover records a separate failure in the surrounding publishing work: some Season 4 draft numbers had changed, while their older ledger filenames had not. A batch harness paired by episode number and produced two invalid green results. The report says those pairings were corrected by matching the episode titles and reconciling their actual claims.

That is a different boundary from recognising a source-column heading. A parser can correctly read the wrong ledger. Fixing the header vocabulary cannot establish that the supplied evidence belongs to this article. The practical order is to identify the draft and its evidence by verified subject and revision, then parse the ledger, then check whether its sources support the claims. I have kept the pairing failure visible here because a more tolerant parser would otherwise make the surrounding workflow look more complete than the record establishes.

Lessons I would keep

I would start a parser repair with a real failing input and the expected output. The supported vocabulary should come from observed formats, and the rejection cases should be just as concrete. That keeps a convenience fix from turning into an unbounded guess about author intent.

I would separate “not this kind of table” from “this kind of table, malformed.” Both can produce no usable rows, but they should lead to different feedback. One is ordinary absence; the other is an actionable contract error.

I would also treat status and evidence as different data. A Verified? column can be useful in a review record, but its answer cannot replace the source field. Preserving both roles is more honest than accepting a tick as proof.

Finally, I would test the whole adapter boundary while keeping its claim modest. Recognition, separator handling, token expansion and source association all matter. Passing those tests establishes a dependable representation for later review. It does not establish the truth of every represented statement.

Next in the sequence

The next chapter returns to the graph backlog from Episode 4. The later July records describe a route-level schema-enforcement repair and an opt-in concurrency limit. The interesting question is what survives across the whole route, and which new bottleneck appears when more work finally reaches the database.

Glossary — plain-English definitions

Ledger

A structured mapping of a claim to its supporting source. Episode evidence and limits (retained historical evidence).

Predicate

A function returning whether a condition holds. Episode evidence and limits (retained historical evidence).

State machine

A parser whose current state changes how the next input is interpreted. Episode evidence and limits (retained historical evidence).

Separator row

The Markdown row confirming the header structure of a table. Episode evidence and limits (retained historical evidence).

Concrete token

A specific checkable part extracted from prose for the linter. Episode evidence and limits (retained historical evidence).

Fail loudly

Report a clear contract error instead of silently returning an ambiguous empty result. Episode evidence and limits (retained historical evidence).

Source support

Whether the cited evidence actually establishes the claim at its stated scope. Episode evidence and limits (retained historical evidence).

Sources and limits

The source and claim ledger (retained in review ledger) retains the July 8 handover, adjacent publishing context and the historical parser, linter and tests. The synthetic ledger is a teaching example. Its file label is not a recovered experiment. No production publication is claimed here, and no historical CI pass is silently converted into a fresh browser or truth-verification result.