fix(advise): inverted IS NOT NULL on current sqlglot, and the CI gaps that hid it - #13
Merged
Merged
Conversation
…d guard the gap that hid it
sqlglot moved `IS NOT NULL` from a wrapping `Not(Is(...))` node to a `negate=True`
flag on the `Is` node itself, in 30.13 — inside the range this package declares
(`sqlglot>=30.12,<31`). `_role()` read only the parent, so on any newer sqlglot
every `IS NOT NULL` was classified as a `NULL_CHECK`: the exact inverse.
That is not a cosmetic misclassification. ADV004 turns these roles straight into a
partial index's WHERE clause, so it proposed
CREATE INDEX ON ... (col) WHERE guard IS NULL
for a workload filtering `guard IS NOT NULL` — an index over precisely the
complement of the rows the queries touch, at MEDIUM confidence. `_is_negated` now
accepts both encodings; the flag is additive, so neither the version floor nor the
ceiling has to move and users are not pushed to a particular sqlglot to get
correct DDL.
Two CI gaps let this through, and both are now closed:
- Every job ran `uv sync`, which installs what uv.lock pins (30.12), so CI tested
exactly one point in the declared ranges while `pip install sqlquality` resolved
the newest 30.x. A `highest-deps` job now installs with `--resolution highest`,
so an upstream release that breaks a declared range fails here instead of in
someone's generated DDL.
- Every job also ran `--all-extras`, so nothing verified the documented promise
that a plain `uv run pytest` works with no extras and no Docker. A test that
acquires such a dependency becomes a *skip*, which reads as success in a green
run, so the new `no-extras` job fails on any skip and prints the reason.
The regression test builds both trees directly instead of parsing, because parsing
only ever exercises whichever encoding the installed sqlglot produces — which is
why the existing parse-based test could not fail on 30.12. Verified: reverting to
parent-only polarity turns the new test RED under 30.12 *and* under 30.14, while
the old test only reddens under 30.14. Suite passes on both (578 passed).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…thing was checking Deferred Minor findings from the Batch 2 reviews, triaged fix-before-merge or fix-later. No emitted DDL changes; two rationale sentences are added. - ADV001 and ADV007 both reached MEDIUM in silence when the NDV catalog had nothing for the column, which reads as a considered middling judgement rather than what it is: the selectivity check did not run. Disclosing a check that could not run is the discipline this whole rule set turns on, and this was the one confidence level that stayed quiet about its own reason. Both now say so, in the same words, and only at that rung -- with an unknown row count the proposal is already LOW for a different reason and a second sentence would misattribute it. - Four behaviours had no test that noticed them: ADV001's and ADV007's new disclosures, ADV007's `(-cost_ms, column)` tiebreak (two equally hot join keys reshuffled between runs without it), and `_coverage_warning`'s `<=` boundary -- flipping it to `<` so the warning fires at exactly 20% passed the entire suite. The boundary test asserts at, above and below the threshold so it fails whichever way the comparison is flipped. - Three comments that were no longer true. `_identifier_pattern` named ADV006's wide-table detection as its main caller, which stopped being true when ADV006 moved to parsing; the real callers are the expression-index disclosures in ADV001/ADV007/ADV008. `_dedupe_by_ddl` said tied proposals "carry no information that distinguishes them" when `rationale` and `title` can differ -- what they lack is anything a tie-break could act on. ADV008's unreadable-index-list sentence omitted the "-- check before applying" its two siblings carry. Verified: each of the four new pins turns RED under a mutation of exactly the line it claims to pin, from purged caches, and the ADV001/ADV007 pair reddens independently. Also confirmed against a live PostgreSQL 16 under `track = all` that the CAP_WORKLOAD comment's two examples are what the server really records -- a COPY's nested row keeps its wrapper, a PL/pgSQL body is recorded bare. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…em Python The runner's Python is Debian-managed and refuses pip install (PEP 668), so `uv pip install --system` exited 2 before resolving anything. `uv run` is not an alternative either: it syncs from uv.lock, which is exactly what this job exists to bypass. Rehearsed the fixed commands locally against the highest allowed resolution (sqlglot 30.14) before pushing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-ups from the Batch 2 reviews, plus one shipped correctness bug the follow-up work
uncovered.
The bug:
IS NOT NULLwas read asIS NULLon any current sqlglotsqlglot moved
IS NOT NULLfrom a wrappingNot(Is(...))node to anegate=Trueflag on theIsnode itself, in 30.13 — inside the range this package declares (sqlglot>=30.12,<31)._role()read only the parent, so on any newer sqlglot everyIS NOT NULLwas classified as aNULL_CHECK: the exact inverse.That is not a cosmetic misclassification. ADV004 turns these roles straight into a partial
index's
WHEREclause, so it proposedfor a workload filtering
guard IS NOT NULL— an index over precisely the complement of therows the queries touch, at MEDIUM confidence.
uv.lockpins 30.12, so development, CI and every review on the Batch 2 branch ran the oneversion where the code was correct. A fresh
pip install sqlqualityresolved 30.14 and got theinverted predicate.
_is_negatednow accepts both encodings — the flag is additive, so neitherthe version floor nor the ceiling has to move and users are not pushed to a particular sqlglot
to get correct DDL.
The regression test builds both trees directly rather than parsing, because parsing only
exercises whichever encoding the installed version produces — which is why the existing
parse-based test could not fail on 30.12. Verified: reverting to parent-only polarity turns the
new test RED under 30.12 and 30.14, while the old test reddens only under 30.14.
Two CI blind spots, both closed
The bug above was invisible to CI for a structural reason, and so was a second invariant:
highest-depsinstalls with--resolution highestinstead of readinguv.lock, so everyjob no longer tests the same single point in the declared dependency ranges. An upstream
release that breaks a declared range now fails here rather than in someone's generated DDL.
no-extrasruns the default suite after a plainuv sync— no extras, no container — andfails on any skip. Every existing job passes
--all-extras, so nothing verified thedocumented promise that
uv run pytestworks out of the box. A test that acquires such adependency becomes a skip, which reads as success in a green run.
I verified the
no-extrasguard both ways: it passes today (576 → 583 passed, no psycopginstalled, zero skips), and it fires on a deliberately introduced
importorskip.Review follow-ups
The MEDIUM rung now says why. ADV001 and ADV007 both reached MEDIUM in silence when the NDV
catalog had nothing for the column — which reads as a considered middling judgement rather than
what it is: the selectivity check did not run. Both now disclose it, in the same words, and only
at that rung (with an unknown row count the proposal is already LOW for a different reason, and a
second sentence would misattribute it).
Four behaviours nothing was checking, each now pinned and each verified RED under a mutation
of the line it claims to pin:
(-cost_ms, column)tiebreak — two equally hot join keys reshuffled between runswithout it
_coverage_warning's<=boundary — flipping it to<, so the warning fires at exactly 20%,passed the entire suite. The test asserts at, above and below the threshold, so it fails
whichever way the comparison is flipped.
Three comments that were no longer true.
_identifier_patternnamed ADV006's wide-tabledetection as its main caller, which stopped being true when ADV006 moved to parsing.
_dedupe_by_ddlclaimed tied proposals "carry no information that distinguishes them" whenrationaleandtitlecan differ — what they lack is anything a tie-break could act on.ADV008's unreadable-index-list sentence omitted the "— check before applying" its two siblings
carry.
Verification
583 passed, 14 deselected— zero skips14 passedintegration against livepostgres:16ruff check,ruff format --check,mypy src/sqlqualitycleanStill open
rule that never calls
_covered. Pre-existing; the fix is a design question, not wiring.another project on at least one dev machine. When that happens
docker compose updoes notbind and the suite talks to whatever else is listening. Worth either picking a less common
port or failing loudly on a version/database mismatch.
🤖 Generated with Claude Code