Add: R (.r/.R) language support via bespoke extract_r#1932
Closed
Sirhan1 wants to merge 2 commits into
Closed
Conversation
Add _score_query() producing combined ranking + per-term singleton winners in a single graph traversal. _pick_seeds consumes precomputed winners instead of rescoring each term via _score_nodes. Behavior byte-identical to the legacy T+1 path. Supersedes the rescoring loop from Graphify-Labs#1596 while preserving its per-term coverage guarantee; folds in coverage scaling from Graphify-Labs#1724 and multiplicity penalty from Graphify-Labs#1832. Orthogonal to the trigram prefilter from Graphify-Labs#1431. Benchmark (full sweep in Graphify-Labs#1889): 1k-100k nodes, 1-10 terms: 1.43x-2.43x median speedup Single-term: no regression (1.70x-1.89x improvement) Traversals: T+1 -> 1 regardless of term count Tests: 3221 passed, 3 skipped. Ruff clean. Graphify graph updated. Refs Graphify-Labs#1889
Adds a new bespoke R extractor under graphify/extractors/, following the Julia/Elixir precedent. Handles the five function-assignment forms (<-, <<-, =, ->, ->>), nested function scoping, top-level vs function-level call attribution, member ($/@) and package-qualified (::/:::) raw_calls, pipes (|>/%>%), static library/require/requireNamespace imports, and static source() cross-file linkage. Unqualified bare calls route through the shared raw_calls resolver with an R language-family guard that blocks binding to non-R definitions. Wires .r into _DISPATCH/_EXTRA_FOR_EXTENSION/_LANG_FAMILY_BY_EXT and Rscript into _SHEBANG_DISPATCH; .R reuses the existing case-insensitive suffix fallback. Ships under optional [r] extra via tree-sitter-language-pack (no maintained tree-sitter-r wheel fits the per-language model); the pack may fetch/cache the R grammar on first use (no source uploaded, R not executed). Fixes Graphify-Labs#1689 for .R (no longer a no-AST-extractor case) and reuses the existing Graphify-Labs#1745 install-extra warning with pip install "graphifyy[r]". Adds tests/fixtures/sample.r + 3 cross-file fixtures, 22 R tests in test_languages.py following the DM optional-grammar skip-when-unavailable convention, and 2 dispatch/missing-dep tests in test_extract.py. R package metadata (DESCRIPTION/NAMESPACE/exports) and package-qualified call resolution are deferred to a follow-up (Graphify-Labs#9).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds R (
.r/.R) language support via a new bespokeextract_rundergraphify/extractors/, following the Julia (dd8235e) and Elixir (3d5da60) precedent. Scoped strictly to.Rcode structure + call topology — package metadata and package-qualified call resolution are deferred to a follow-up.Why
R was the motivating case for the
#1689"no AST extractor" warning (.r/.Ris inCODE_EXTENSIONSbut had no dispatch entry). This PR wires a real extractor and reuses the existing#1745install-extra warning path for the missing-dependency case (pip install "graphifyy[r]").Named functions in R are assignments whose right- or left-hand side is an anonymous
function_definition, so the generic extractor is not a good fit. A bespoke extractor undergraphify/extractors/matches the established home for language-specific extractors (not theextract.pygod node).No maintained
tree-sitter-rwheel fits the repo's per-language wheel model, so the grammar ships viatree-sitter-language-pack>=1.12,<2under the optional[r]extra. The pack may fetch/cache the R grammar on first use; no source is uploaded and R itself is not executed.How
.r→_DISPATCH(.Rreuses the existing case-insensitive suffix fallback);.r→_EXTRA_FOR_EXTENSION(names the[r]extra in#1745);.r→_LANG_FAMILY_BY_EXTas the"r"family (blocks cross-language binding);Rscript→_SHEBANG_DISPATCH(Rscriptwas already in_SHEBANG_CODE_INTERPRETERS, so R-shebang scripts were already detected as code — they now dispatch instead of hitting#1689). No new node types, relations, resolver interfaces, or package-ownership semantics;CODE_EXTENSIONS,_CASE_INSENSITIVE_EXTS, and the analysis family map unchanged.extract_r()emits the existingnodes/edges/raw_callscontract:<-,<<-,=,->,->>), scoped IDs for nested fns beneath the enclosing fn.=used as a named call argument.$/@receiver calls and::/:::package-qualified calls becomeraw_callswithis_member_call=True, so the shared resolver won't bind them by method name.|>,%>%) traverse normally so contained calls are captured.EXTRACTED; unresolved bare calls go toraw_callsfor the shared cross-file resolver, which resolves a unique cross-file candidateEXTRACTEDwith import evidence orINFERREDwithout, leaving ambiguous names unresolved.library(pkg),require(pkg),requireNamespace("pkg")with a literal identifier or string →importsedges;pkg::fn/pkg:::fndouble as import evidence forpkg. Dynamic expressions (call args, variables) are skipped.source("helper.r"): only a static string ending in.r(case-insensitive), resolved against the caller file's directory (mirrors the Bash static-sourcepolicy). Emitsimports_fromonly when the target exists on disk. URLs, connections, variables, computed paths, and missing files are skipped.Tests
tests/fixtures/sample.r+ 3 cross-file fixtures (r_other.r,r_extra.r,r_caller.r).tests/test_languages.pyfollowing the DM optional-grammar skip-when-unavailable convention (@_needs_rfor real-grammar tests; dispatch/missing-dep tests stay network-free via monkeypatchedsys.modules).tests/test_extract.py(test_extract_warns_when_r_extra_missing,test_extract_no_warning_when_r_extra_installed).test_extract.pytests that asserted.R/.rtriggered#1689now use.ejs(the canonical no-extractor case), since.Ris now dispatched.Real-grammar tests skip when
tree-sitter-language-packis absent. Tests must not download a parser implicitly; missing-dependency and dispatch tests are network-free.Verification
test_label_communities_batches_when_over_batch_sizeis order-dependent and fails on a clean stashed tree too — unrelated to R; there's already fix: make label batching test independent of order #1872 tracking it).ruff checkclean on touched files.pyrightclean ongraphify/extractors/r.py(0 errors with the venv pythonpath; the import-resolution noise ontree_sitter_language_pack/tree_sitteris repo-wide pre-existing and affects every extractor identically).graphify update .run to refresh the graphify-out graph.Scope deferred to follow-up (#9)
R package metadata (
DESCRIPTIONDCF ingestion,NAMESPACEimports/exports), package-qualified call resolution once package-to-symbol ownership exists, S3/S4/R6 semantics,.Rmdchunk extraction, NSE/formula handling, higher-order callback inference,renv.lock, and runtime evaluation.