fix(detect): let .graphifyinclude rescue generic keyword skips (#1225)#1921
Open
HerenderKumar wants to merge 1 commit into
Open
fix(detect): let .graphifyinclude rescue generic keyword skips (#1225)#1921HerenderKumar wants to merge 1 commit into
HerenderKumar wants to merge 1 commit into
Conversation
…ify-Labs#1225) The secrets keyword heuristic (stem ends in token/secret/credential/...) silently dropped benign files like derive-locked-tokens.md, and the .graphifyinclude allowlist could not override it: the include machinery was loaded in detect() but never consulted after dot-dirs became indexed by default, so the allowlist literally could not rescue anything. Wire the allowlist into the sensitive check as an explicit escape hatch: - A wildcard-free .graphifyinclude entry now rescues a file from the keyword stage. Glob entries deliberately do not count, and the specific patterns (secrets dirs, .env, key files) are never overridden, so a broad * cannot start ingesting api_token.txt. - Keyword skips are annotated in skipped_sensitive and surfaced once per scan on stderr, naming the escape hatch. - GRAPH_REPORT.md lists every skipped file by name (previously the report did not mention skips at all), so a swallowed file is greppable.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1225.
Problem
The generic secrets keyword heuristic drops any file whose stem ends in
token(s)/secret/credential/… — swallowing benign docs likereferences/prompts/derive-locked-tokens.md— and nothing can override it. The drop is also invisible: neither the CLI output nor GRAPH_REPORT.md mentions skipped files at all on current v8.Root cause of the missing escape hatch: the
.graphifyincludemachinery (_load_graphifyinclude,_is_included) is still loaded indetect()but never consulted — orphaned when dot-dirs became indexed by default — so the allowlist literally cannot rescue anything, matching its own docstring's disclaimer.Fix
Suggested fixes 2 + 3 from the issue:
.graphifyincludeentry beats the keyword heuristic._is_sensitiveis split into a reason-returning core (_sensitive_reason) so the scan can tell the heuristic stage apart from the specific ones; the bool wrapper keeps the existing API and tests binding. Only wildcard-free entries count as explicit intent, and only the keyword stage can be overridden — secrets dirs,.env, key files,id_rsa, … are never rescued, so a broad*glob written to opt hidden files into traversal cannot silently start ingestingapi_token.txt.skipped_sensitivewith the escape hatch spelled out, one aggregated stderr note per scan names the affected files (capped at 6, like theDockerfile/Gemfile/Makefile/Rakefile/LICENSEand other extensionless, non-shebang project files currently produce no visible trace in graphify's output — not classified, not counted anywhere #1692 unclassified note), and GRAPH_REPORT.md gains a## Skipped Filessection listing every skipped file by name so a swallowed file is greppable.detect_incrementaldelegates todetect(), so incremental scans inherit all of this.The
_credentials.pysource-file case from the follow-up comment is already handled on v8 by the #1666 source-code exemption; this covers the remaining docs/data case.Tests
Six new tests, written first and watched fail: exact-path rescue, annotation + stderr note, glob entries don't rescue, credential stores are never rescued, report lists names, report omits the section when empty. Full detect/report/hypergraph suites pass (186/186); the repo-wide run matches the pre-change baseline exactly (this env is missing tree-sitter/watch/dedup extras, identical failures with and without the change).