feat: opt-in PII filter to strip sensitive columns from responses - #56
Open
AnilRh wants to merge 1 commit into
Open
feat: opt-in PII filter to strip sensitive columns from responses#56AnilRh wants to merge 1 commit into
AnilRh wants to merge 1 commit into
Conversation
Add a feature-flagged filter that prevents the AI client from receiving PII values in tool responses. Off by default; enabled per-environment via PII_FILTER_ENABLED=true and PII_CONFIG_PATH pointing at a YAML blocklist. The filter has two layers: - Pre-flight AST check on execute_sql refuses queries that would project a blocked column (including SELECT * over a table with any blocked column). Lenient on WHERE/JOIN ON/GROUP BY/ORDER BY since those do not produce PII in the result. - Post-filter on result rows is the actual security guarantee: blocked columns are stripped from every returned row, even if the pre-flight check is bypassed by an unparseable query. Schema endpoints (list_tables, get_table_schema*) also hide blocked columns so the AI never learns their names. When disabled, every filter function short-circuits in its first line: no YAML load, no SQL parsing, no per-row work. Blocklist YAML is loaded lazily and cached for the process lifetime. Names match case-insensitively. The literal "*" marks a fully-blocked table (hidden from list_tables). Files: - src/pii_filter.py: new filter module (~350 lines) - src/config.py: PII_FILTER_ENABLED + PII_CONFIG_PATH env vars - src/server.py: hooks into list_tables, get_table_schema, execute_sql - examples/pii_blocklist.example.yaml: synthetic example - src/tests/test_pii_filter.py: 32 unit tests, all passing - README.md: documentation - pyproject.toml: sqlglot and pyyaml dependencies
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add a feature-flagged filter that prevents the AI client from receiving PII values in tool responses. Off by default; enabled per-environment via PII_FILTER_ENABLED=true and PII_CONFIG_PATH pointing at a YAML blocklist.
The filter has two layers:
Schema endpoints (list_tables, get_table_schema*) also hide blocked columns so the AI never learns their names.
When disabled, every filter function short-circuits in its first line: no YAML load, no SQL parsing, no per-row work.
Blocklist YAML is loaded lazily and cached for the process lifetime. Names match case-insensitively. The literal "*" marks a fully-blocked table (hidden from list_tables).
Files: