fix(biolink): make KGX output validate against the Biolink Model

[SkyeAv]

Context

Tablassert's *.{nodes,edges}.ndjson is meant to be consumed as a KGX passthrough by
NCATSTranslator/translator-ingests and by the local DINGO DAKP ingest. It did not work.

Measured against the real 2M-edge corpus in .tablassert/million-kg/ using the installed biolink-model 4.4.3 Pydantic classes:

	records	before	after
edges	2,000,085	0 (0.00%)	2,000,085 (100%)
nodes	424,141	0 (0.00%)	424,141 (100%)

Not a single record validated. This stayed invisible because nothing ever instantiated Association/NamedThing against an emitted record — biolink.py derived the vocabulary from the model but never checked the output.

What was wrong

1. Retrieval provenance was flat. upstream_resource_ids and source_record_urls have domain: retrieval source, so both were extra_forbidden on every edge; primary_knowledge_source was a list where Biolink wants a scalar. There was no sources list and no resource_role.
2. Predicate contradicted its own category on 48.2% of edges (964,019). The largest single case: 723,595 × GeneToDiseaseAssociation + biolink:gene_associated_with_condition, a predicate that class's enum forbids. edge_category() chose a class from (subject role, object role) and never consulted the predicate.
3. Qualifiers were entity-resolved even when enum-ranged, producing object_direction_qualifier: "UMLS:C0205217" where the vocabulary requires the token increased.
4. Qualifiers landed on classes that forbid them — species_context_qualifier (1,436,251), anatomical_context_qualifier (383,798), and others.
5. Slots with zero Pydantic owners were emitted anyway — statistical_significance_qualifier (1,191,276), supporting_study_size (1,191,032). They are declared in the LinkML schema but attached to no class.
6. The allow-list walked only the base Association MRO, so subclass-only evidence slots (clinical_approval_status, number_of_cases, FDA_regulatory_approvals) were demoted into supporting_text.
7. source / source_version were extra_forbidden on all 424,141 nodes.
8. p_value was written as "6.5200e-06" where Biolink types it float.
9. supporting_text was used as an untyped dump ("sheet_name: Table_S7"), though its Biolink meaning is a supporting sentence.
10. strip_nulls deleted 0 and false, so a p_value of 0 or number_of_cases: 0 vanished.
11. Annotations were scalar-only, so multivalued slots became joined strings that consumers iterate character by character.

What this changes

• retrieval_sources() emits a sources[] of RetrievalSource, mirroring build_association_knowledge_sources() in translator-ingests; primary_knowledge_source is now scalar.
• resolve_association_class() walks the MRO and gives up specificity only as far as the predicate requires.
• Enum-ranged qualifiers are passed through unresolved and validated against their vocabulary both at config time and at emit time (a column-encoded qualifier carries whatever the sheet holds, so config-time checking alone is not enough).
• prune_to_class() masks per row what the resolved class does not declare, and hands those values to the inlined study rather than deleting them.
• inline_supporting_study() populates has_supporting_studies (Study → StudyResult), the COHD/ICEES pattern, replacing the supporting_text key/value dump.
• The allow-list unions the whole association family; slots no Pydantic class can hold are rejected at config time with an actionable error.
• Nodes emit provided_by / in_taxon / in_taxon_label; resolve_node_class() demotes the 48 categories that cannot be emitted (Publication requires publication_type, ClinicalAttribute requires has_attribute_type, GenomicEntity is a mixin that rejects its own name).
• New tablassert validate-kgx constructs every record as its own Biolink class, groups failures by field and error type, and exits non-zero — so a 0% pass rate can never ship again. Also asserted in the e2e build test.

The second commit is unrelated housekeeping that was already in the working tree: dspy moves out of [agent] into a new [optimize] extra, since it powers only the GEPA --optimize path.

⚠️ Open question: effect_size / effect_type

main (6f8456d) emits effect_size and effect_type as top-level edge fields for
biolink-model#1774. Neither is on Association in the pinned 4.4.3, so edges carrying them fail validation with extra_forbidden — the same class of problem this PR fixes, in a feature merged three days ago.

I deliberately did not change that behaviour: it is a recently shipped feature and the call is yours. Two options:

• Leave as-is. effect_size/effect_type reach the final edges; validate-kgx reports those edges as invalid until a biolink-model release ships #1774.
• Route them like supporting_study_*. Drop them from TABLASERT_EDGE_EXTRAS so the existing derivation carries them on the inlined StudyResult until the model ships them, at which point they become flat edge fields automatically. Fully valid today, feature deferred.

Say which and I will apply it.

On biolink-model#1770

Nothing is hardcoded to a model version. UNSATISFIABLE_EDGE_FIELDS is derived from the installed package at import time. Verified by generating the Pydantic model from biolink/biolink-model#1770 (dbd60110f): with it applied, all six supporting_study_* slots become real Association fields inherited by every subclass, drop out of the unsatisfiable set, and are emitted flat on the edge instead of routed to the StudyResult — with no code change. Note supporting_study_size is Optional[int] there, so it must be a JSON integer.

Verification

• pytest: 728 passed, 34 skipped (includes main's 32 new effect_size/effect_type tests).
• cargo test: 68 + 10 passed.
• ruff check / ruff format / cargo fmt / cargo clippy: clean.
• Full-corpus replay of the shipped transformations over all 2,000,085 edges and 424,141 nodes: 100% valid, zero remaining errors.

Rebased onto main, so this contains only the two commits above.

🤖 Generated with Claude Code

https://claude.ai/code/session_01V6KH4PXbz68F3DSFNAFa63

[coderabbitai]

Warning

Review limit reached

@SkyeAv, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 5 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: c2596caf-efca-447d-95ce-bd6f57c927e5

📥 Commits

Reviewing files that changed from the base of the PR and between 6a54349 and 4ec1cd5.

⛔ Files ignored due to path filters (1)

• uv.lock is excluded by !**/*.lock

📒 Files selected for processing (18)

• CHANGELOG.md
• README.md
• docs/agent.md
• docs/cli.md
• docs/installation.md
• pyproject.toml
• rust/src/json.rs
• src/tablassert/agent.py
• src/tablassert/biolink.py
• src/tablassert/cli.py
• src/tablassert/errors.py
• src/tablassert/lib.py
• src/tablassert/models.py
• src/tablassert/rig.py
• tests/test_agent_lazy.py
• tests/test_biolink.py
• tests/test_docs_cli_coverage.py
• tests/test_lib.py

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[SkyeAv]

@coderabbitai review

[coderabbitai]

⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.