The Schema Had to Survive the Whole Route

Episode 4 paused the graph backfill after a defensive parser repair left too much extraction output in doubt. The intervening work improved how agents found history, assessed their instructions and represented evidence for publishing. The graph backlog remained. Returning to it required a better generation contract, not another claim that readable JSON was enough.

I wanted documents to enter the graph with a response structure the ingestion code could rely on. The later records describe a decision to keep the remote extraction route and repair enforcement across it. The application then gained an opt-in concurrency limit. That made progress possible, but also exposed how easily a faster stage can move the pressure into the next one.

A requirement has to survive every hand-off before the receiving stage can rely on it. Conceptual illustration.

This chapter covers July 9–10, 2026, with the detailed implementation and runtime account centred on July 10. It is a curated reconstruction from the backfill handover, later edge-coercion handover and historical client/ingestion source at 0dff3571. I owned direction and review; agents assisted with implementation and checking. Reported provider probes, deployment checks and job progress have not been independently replayed. New examples and checks use isolated synthetic inputs without model calls, credentials or cluster changes.

A schema request was only one boundary

The output contract travelled through more than one component. Rootweaver’s client called a gateway route, that route reached OpenRouter, and OpenRouter selected a provider endpoint serving the requested model. The final response returned through the same chain before Graphiti attempted to validate it.

Every link mattered. A client could ask for a JSON schema while an intermediate configuration dropped an unsupported parameter. A provider could return valid JSON without enforcing the requested response structure. A compatible endpoint name therefore did not establish compatibility with every optional feature the caller intended to use.

The July account identifies two route-level settings: drop_params: false and provider.require_parameters: true. The reported purpose was to preserve requested parameters and require a compatible provider route rather than silently accept a different capability set. The platform also pinned the generic client’s structured-output mode explicitly.

OpenRouter’s structured-output documentation describes the schema feature and its support requirements. That primary reference explains the contract; it is not a new test of the July route. The runtime evidence in this episode remains the dated report that a schema-shaped probe succeeded through the deployed gateway configuration.

Boundary	Required agreement	Failure that an earlier pass would miss
Application client	Request the intended structured mode	A default changes or the wrong setting is read
Gateway route	Preserve the relevant parameters	Unsupported fields disappear before the provider sees them
Provider selection	Choose an endpoint supporting those parameters	The chosen model route lacks the intended enforcement
Response validation	Check the returned value against the contract	Syntactically valid JSON has the wrong shape
Task review	Inspect the extracted content and graph result	Correctly shaped output contains poor or irrelevant entities

This is why the defensive coercion from Episode 4 was still useful without being the whole solution. It could recover a wrapper or an encoded entity when the information was present. It could not create the facts that an unconstrained response omitted. Enforcement tried to improve what arrived at that boundary; it did not make semantic review unnecessary.

The generic client got its own explicit mode

The historical implementation added a configuration reader for the generic client. The default was json_schema; the accepted alternative was json_object. Invalid input produced a warning and fell back to the schema default. This was separate from the guided-decoding setting used on a different client path.

client.py · _generic_structured_output_mode · platform 0dff3571

def _generic_structured_output_mode() -> str:
    raw_value = os.getenv(GRAPHITI_GENERIC_STRUCTURED_MODE_ENV, "json_schema")
    mode = raw_value.strip().lower()
    if mode in _GENERIC_STRUCTURED_OUTPUT_MODES:
        return mode

    logger.warning(
        "Invalid %s=%r; using default json_schema",
        GRAPHITI_GENERIC_STRUCTURED_MODE_ENV,
        raw_value,
    )
    return "json_schema"

Explain this code line by line

• The environment value defaults to json_schema, then whitespace and case are normalised.
• An accepted mode returns directly; the retained vocabulary contains json_schema and json_object.
• Other text is reported through a warning, preserving an observable configuration mistake.
• The fallback returns the schema mode. That is application intent, not proof a remote endpoint enforced it.

That separation prevents a familiar configuration mistake. A variable can exist, be set correctly and still be irrelevant if the active client does not read it. The new code made the active generic-client choice inspectable rather than relying on a nearby vLLM-specific option to imply behaviour elsewhere.

The two modes also have different meanings. Asking for a JSON object constrains the broad response form. Asking for a JSON schema requests a more specific structure. Neither name should be treated as evidence that every provider actually enforced it in every call. The route must support the option, and the result still has to be checked.

The retained constructor passes this mode into the generic client. The surrounding tests cover the configured choice and its default behaviour. Those tests are useful evidence about application intent. They do not replace the route-level probe, because a fake client cannot demonstrate what an external gateway or endpoint forwards.

Here is a synthetic teaching payload showing the two layers of intent. It is deliberately shortened and is not an executable reproduction of the original request:

{
  "response_format": {
    "type": "json_schema",
    "json_schema": {"name": "ExtractedEntities", "strict": true}
  },
  "provider": {"require_parameters": true}
}

The schema name identifies the expected response contract. strict expresses the requested constraint. require_parameters concerns provider compatibility. A real request also needs the complete schema, model, messages, route configuration and any required authentication. Omitting those here keeps the example focused; it must not be copied as a complete provider call.

Follow the mechanism

Pause flowExpand diagram

Close expanded diagram

The diagram follows the requirement across boundaries. Dots show direction, not a successful historical request. The final content-review step stays visible because conformance establishes shape, not whether the graph learned the right things.

More work in flight needed a bound

Once extraction could progress, the application also gained GRAPHITI_INGEST_CONCURRENCY. The historical source keeps a special sequential path when the configured value is one. Values above one opt into within-batch parallel episode writes. That opt-in matters because entity resolution and deduplication share graph state.

Concurrency is the number of operations allowed to overlap. It is not a promise of proportional throughput. Eight writers can spend their time waiting on the same database resources, repeating slow searches or contending over related entities. A setting that improves one stage can make a later boundary worse.

Several jobs can be ready while only a bounded number may hold the shared tools. Conceptual illustration of a semaphore limit.

The parallel branch constructs an asyncio semaphore, wraps each eligible record’s ingestion with it, and gathers the results. This is an exact excerpt from the retained ingestion method:

ingest.py · DocumentKnowledgeIngester.ingest_records, parallel branch · platform 0dff3571

semaphore = asyncio.Semaphore(self._ingest_concurrency)

async def ingest_one(record: DocumentKnowledgeRecord) -> bool:
    async with semaphore:
        return await self._add_episode_with_retry(record)

results = await asyncio.gather(
    *(ingest_one(record) for record in records_to_ingest)
)

Explain this code line by line

• The semaphore creates the configured number of permits for this batch.
• Each task acquires a permit before entering the existing add-and-retry operation.
• The async context releases its permit when the operation leaves that block.
• gather waits for the submitted eligible tasks. It does not create extra database capacity.
• The returned results are later associated with their records. The default-one sequential path is separate.

For a synthetic batch of ten eligible documents with concurrency eight, at most eight calls can hold the semaphore at once. If only three documents remain after processed markers are filtered out, effective parallelism cannot exceed three. The configured limit does not create more work than the batch contains.

A useful upper bound is therefore the smaller of configured concurrency and eligible records in that batch. It still does not predict duration. Per-document work can vary, retries can consume time, and the database can become the shared bottleneck. The July job used concurrency eight and batch size ten; those are reported settings, not a recommended default for today’s system.

The historical source defaulted to one for compatibility with the established sequential behaviour. That made the new concurrency path an explicit operational choice. A faster-looking setting was not silently promoted into the baseline merely because it passed unit tests.

Successful markers were the resumability boundary

The batch loop checks whether each document is already marked processed. In the parallel branch it builds a list of eligible records before creating the tasks. After the calls finish, it pairs each result with its record. Only successful results write processed markers; failures increment the failure count and remain candidates for a later run.

ingest.py · successful-result marker update · platform 0dff3571

for record, ok in zip(records_to_ingest, results):
    if ok:
        self._marker_store.mark_processed(record)
        stats["processed"] += 1
        batch_processed += 1

Explain this code line by line

• Each result is paired with its input record.
• Only a true success result enters this branch.
• The marker store records success so a later run can skip the processed record.
• The aggregate processed count and this batch count each increase once.
• This is not a demonstrated atomic transaction spanning remote generation, graph effects and marker persistence.

This design supports resumability, but it deserves a precise description. A later run can skip documents whose success marker was saved and retry those not marked successful. That does not establish a transaction across the model call, graph write and marker file. If a process stops after a graph effect but before a marker is persisted, some work may be attempted again. Deduplication and content-based identities need their own correctness evidence.

It would therefore be too strong to call the whole job exactly-once simply because a marker store exists. The narrower supported claim is that successful markers reduce repeat work across ordinary retries and restarts. The retained source shows where those markers are written; the historical report says the job relied on them to make later runs converge.

The result collection also matters. The inspected code gathers tasks and processes their results in the order associated with the input list. A returned false result is different from an unexpected exception escaping the surrounding logic. This draft does not claim that every possible provider, cancellation or persistence error is made harmless by the semaphore itself.

Progress exposed the next failure

The July 10 handover reports both enforcement legs shipped, then describes a running second backfill attempt. Its recorded graph count rose from 276 to 805 nodes by the checkpoint. The job was at batch 17 of 82, with timeouts returning under load despite a larger database timeout setting. The report expected the eight-hour deadline to be exceeded before the whole backlog completed.

Those figures are observations from a dated checkpoint, not the final job outcome. The difference of 529 nodes is a net count change in that window. It is not 529 independently verified new facts, a count of completed documents or proof that answers improved.

Increasing flow into the system can move contention into a shared downstream stage. Conceptual artwork; not a measured throughput chart.

The report’s apparent contradiction is useful: enforcement could be repaired while ingestion still failed. Earlier schema drift was no longer the only meaningful problem. Database search and deduplication queries now consumed the waiting time. More documents reaching the graph made an existing downstream limit easier to observe.

There were also two reported residual edge-model validation errors. The earlier adaptation handled entity and node-resolution shapes; edge extraction had another response contract. The adjacent July 10 handover records a later coercion repair for that gap. That is a separate scope, not evidence that the first schema change had already covered every Graphiti response model.

Tests, live probes and unfinished work

The implementation handover reports 117 tests and a fresh verifier pass, followed by merged platform and gateway changes. It also records a first attempt whose fixture used plain dictionaries where Graphiti expected message objects. That is a useful example of a test failing before it exercises the intended provider contract.

The new checks for this reconstruction inspect the selected mode, exercise synthetic bounded tasks and verify marker behaviour against the retained source. They do not call OpenRouter, repeat the original provider probe or test the July graph under contention. Those boundaries prevent a convenient local pass from becoming a false claim of end-to-end recovery.

The historical job was still running when the handover was written. Its prediction of a deadline failure is not the same as a retained final DeadlineExceeded result. The next chapter follows the later timeout investigation, where a proposed short-circuit initially missed the real runtime callable and a corrected patch produced a split outcome.

Lessons I would keep

I would make the requested capability explicit at the active client and check that every intermediary preserves it. A model name, a schema-shaped request and an HTTP success each describe a different boundary. The acceptance test should show where the requirement was actually enforced.

I would introduce parallelism as an opt-in experiment with a clear bound. The semaphore limits overlap; it does not certify the safety of shared state or guarantee speed. Effective concurrency and completed-document count should sit beside latency and error rates.

I would describe resumability without promising exactly-once processing. A marker is valuable evidence about what the job will skip, but it is only one part of the write path. The graph effect and marker persistence can fail at different times.

Finally, I would keep progress and usefulness separate. Node growth showed that more work reached the graph in the reported window. It did not settle correctness, total coverage or task benefit. A pipeline becomes easier to reason about when each improvement is allowed to have its own limited result.

Next in the sequence

The next record follows the returning graph-search timeouts. It explains an empty-candidate fast path, why the first patch was attached to a callable the runtime did not use, and why eliminating the timeout storm still did not complete document ingestion. That is the next episode’s mechanism and evidence boundary.

Glossary — plain-English definitions

JSON schema

A declared data structure and constraints for a response. Primary concept reference.

Provider route

The endpoint path selected to fulfil a model request. Episode evidence and limits (retained historical evidence).

Parameter passthrough

Preserving a requested option across an intermediary. Episode evidence and limits (retained historical evidence).

Semaphore

A limit on how many tasks can hold permits at the same time. Primary concept reference.

Concurrency

Work in flight at the same time, distinct from completed work per second. Episode evidence and limits (retained historical evidence).

Processed marker

A retained success record used to skip work on a later run. Episode evidence and limits (retained historical evidence).

Exactly-once

A stronger processing guarantee than ordinary retries plus a marker file. Episode evidence and limits (retained historical evidence).

Contention

Tasks waiting or interfering at a shared resource. Episode evidence and limits (retained historical evidence).

Sources and limits

The source and claim ledger (retained in review ledger) retains the July backfill and edge-coercion handovers, historical client/ingestion code and tests. The shortened payload is synthetic. Original provider request/response bundles, final logs for every backfill attempt and a ground-truth entity set are not available in this package. Current official documentation supplies concept context, not a new observation of the July deployment.