Skip to content

NaturalLanguage entity tagging and CI#18

Merged
zac merged 5 commits into
mainfrom
linguistic-tagging
Jul 13, 2026
Merged

NaturalLanguage entity tagging and CI#18
zac merged 5 commits into
mainfrom
linguistic-tagging

Conversation

@zac

@zac zac commented Jul 12, 2026

Copy link
Copy Markdown
Member

Summary

Stacked on #16 (retargets to main when that merges).

NaturalLanguage entity tagging

  • New EntityTagger provider protocol on MemoryConfiguration, with an NLTagger-backed NLEntityTagger as the default under the MemoryNaturalLanguage trait (nil otherwise), mirroring the query-side NLQueryAnalyzer.
  • Tagger results merge additively into heuristic extraction: they upgrade generic entity labels to person/location/organization and append entities the heuristics missed in well-capitalized text. Heuristic behavior remains the floor — lowercase chat text leaves the tagger silent, and entityTagger: nil reproduces pre-change behavior exactly. Facet inference now sees merged entities, so location/person facets fire beyond the built-in lexicon.
  • Ambiguous self-reported locations ("i'm in lisbon") become durable profile memories only when the tagger confirms the phrase names a place, validated by title-casing it inside a carrier sentence — empirically the high-precision formulation (bare-phrase tagging mislabels; carrier junk like "a meeting"/"fear" is rejected). Travel qualifiers ("i'm in boston this week") mark the mention transient and skip the residence write. Residence statements ("i live in ...") never require tagger confirmation, keeping eval gates OS-stable.
  • save/ingest entity normalization consults the tagger when candidates supply no entities.

CI

  • ci.yml runs on every PR commit and pushes to main: swift test with default traits plus the leakage check and strict dataset validation, and a separate CoreML+NaturalLanguage trait build job. SwiftPM build dirs cached per trait flavor; in-progress runs cancel on new commits.
  • evals.yml is workflow_dispatch-only: extraction-affected suites under coreml_default, pressure/smoke baseline gates, reports uploaded as artifacts. Deliberately not PR-blocking since NaturalLanguage models vary by macOS version.

Testing

  • 120 tests pass locally, including new coverage for tagger-confirmed locations, transient-state rejection, label upgrades, lowercase no-op behavior, and the no-tagger heuristic floor.
  • All five extraction-affected eval suites reproduce baseline metrics exactly under coreml_default (96.49/85.10/95.29 on schema gold; 100% elsewhere; 0% false-writes) — the default-on tagger causes zero gate drift.
  • Leakage checker and CoreML-trait build clean. This PR also exercises the new CI on itself; if runner NaturalLanguage models diverge from dev machines, the NL-dependent tests will surface it here.

@zac
zac changed the base branch from agentmemory-rename to main July 12, 2026 23:34
zac added 3 commits July 12, 2026 16:35
- New EntityTagger provider protocol on MemoryConfiguration with an
  NLTagger-backed NLEntityTagger default under the MemoryNaturalLanguage
  trait (nil otherwise), mirroring the query-side NLQueryAnalyzer.
- Tagger results merge additively into heuristic extraction: they upgrade
  generic entity labels to person/location/organization and append entities
  the heuristics missed; heuristic behavior remains the floor. Facet
  inference now sees merged entities, so location/person facets fire for
  tagger-recognized names beyond the built-in lexicon.
- Ambiguous self-reported locations ("i'm in lisbon") are accepted when the
  tagger confirms the phrase names a place, validated by title-casing inside
  a carrier sentence where NLTagger place recognition is precise. Travel
  qualifiers ("i'm in boston this week") mark the mention transient and
  skip the residence write. Residence statements ("i live in ...") never
  require tagger confirmation, keeping eval gates OS-stable.
- ingest/save entity normalization consults the tagger when candidates
  supply no entities.
- ci.yml runs on every pull_request event and pushes to main: build and
  test with default traits, benchmark-leakage check, strict dataset
  validation, plus a separate CoreML+NaturalLanguage trait build job.
  SwiftPM build directories are cached per trait flavor; in-progress runs
  for a ref are cancelled on new commits.
- evals.yml is workflow_dispatch-only: runs the extraction-affected eval
  suites under coreml_default, gates the pressure and smoke baselines,
  and uploads run reports. Manual because eval results are OS-sensitive;
  the release gate stays local per AGENTS.md.
Swift 6.2 (the CI toolchain) rejects capturing the caller's closure
inside the actor-isolated transaction closure with a sending-risk
diagnostic; Swift 6.3 accepts it. The temp FTS table is session-scoped,
so seeding commits first and the caller's queries run after, with
identical observable behavior.
@zac
zac force-pushed the linguistic-tagging branch from d7496ec to 308d16e Compare July 12, 2026 23:36
zac added 2 commits July 12, 2026 16:40
The Swift Testing workaround hardcoded /Applications/Xcode.app framework
paths. On CI runners the selected Xcode differs from that symlink, so
the @test macro from the selected toolchain expanded against another
Xcode's Testing framework and failed to build. When DEVELOPER_DIR is
set, use only the selected toolchain's paths; CI now exports it from
the Xcode selection step.
@zac
zac merged commit 7fa57bc into main Jul 13, 2026
2 checks passed
@zac
zac deleted the linguistic-tagging branch July 13, 2026 00:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant