Add linguistic competence metrics for spoken-output analysis - #49
Merged
Conversation
Privacy-preserving, pure functions that analyse AAC *spoken output* (phrase history) across the four dimensions of linguistic competence (Light, 1989), grounded in AssistiveWare's 'Measuring AAC user linguistic competence' and Frisch/Wade et al. 'It's Complicated' (arXiv:2606.24854): - Semantic: MATTR-30 lexical diversity (headline, sample-length independent) - Syntactic: MA-UPC-TWR-30 preposition + conjunction diversity - Morphological: MA-UMORPH-TLWR-30 heuristic proxy - Phonological: spelling validity (optional, needs a dictionary) - Activity: utterances/words/unique words/active days/words-per-utterance All measures use 30-word moving-average windows (Covington & McFall, 2010), binned by calendar month with a weighted trend. No I/O, no platform deps - runs anywhere (browser included) and emits only aggregate statistics (no raw text, no word lists). Unit tested.
… no hardcoded lists Addresses review feedback: - No hardcoded word lists. Removed the inlined EN/NL preposition+conjunction sets and the morphology guard list. The core is now data-free; language resources (closed-class words, an inflection classifier) are injected via LanguageResources. Adding a language = provide resources, no core change. - Any language, no silent degradation. When a resource is missing, the affected measure is reported unavailable with a reason, and the report gains a support block + warnings list, so language gaps are explicit. - Source-agnostic. analyzeTimeline already took generic utterances; now historyEntriesToCompetenceUtterances adapts ANY HistoryEntry (Grid 3, Snap, OBF/OBFL, ...) into that stream. Added an OBF/OBFL test proving any speech-history source plugs in. - Removed non-null assertions; lint-clean. Semantic (MATTR-30) stays available for every language out of the box.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds a pure, platform-agnostic module that measures linguistic competence from AAC spoken output (phrase history), across the four dimensions (Light, 1989). This complements the existing pageset-structure metrics with analysis of what users actually say.
Grounded in:
Measures (per the AssistiveWare findings)
All diversity measures use 30-word moving-average windows, making them sample-length independent and usable for the tiny, highly-variable samples typical of AAC. MLU is intentionally not a headline (it conflates linguistic/operational/strategic/social competence); it is reported only as a distribution.
Design
Privacy rationale
This implements the AssistiveWare guidance directly: compute on-device, aggregate over 7+ days (we use a month), and never report fringe-word frequency. It is the analysis engine behind a deployable Grid 3 competence exporter, but it is useful to any project that needs privacy-safe linguistic analysis of AAC output.
Tests
New \ est/competence.test.ts\ — 24 unit tests covering tokenisation, MATTR (incl. sample-length insensitivity), syntactic/morphological diversity, spelling, activity stats, and the timeline engine (privacy assertions confirm no raw text leaks). All passing; typecheck clean.