fix: preserve '=' inside --filter values - #442
Open
jacalata wants to merge 5 commits into
Open
Conversation
Previously `apply_filter_value` called `value.split("=")`, so a filter
like `Notes=x=y` silently truncated to name=Notes, value=x. Split on the
first '=' only (maxsplit=1) so multi-'=' values round-trip intact, and
raise a clear error when the input isn't in name=value form.
Verified end-to-end against a real workbook: `Product Name=x=y Config Kit`
now filters correctly instead of returning empty results.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
3 tasks
Dataset with product names containing '=', '&', ',', backslashes, and metacharacter samples for exercising --filter parsing edge cases against a real server. Also includes columns whose names contain '&', '#', and '\' so future tests can exercise special characters on both sides of the name=value pair. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…rity)
The earlier tightening in this PR made apply_filter_value raise a clean
error when a clause isn't in name=value form. That's the right behavior
for the --filter flag, but URL-syntax exports inherited from tabcmd
Classic scripts often contain literal '&' inside filter values (e.g.
`?Product Name=AT&T 841000 Phone`), which get split into fragments the
parser can't understand.
Classic silently skipped such fragments. Verified end-to-end against a
Tableau Server today: with the strict path, tabcmd 2 errored out
("Filter clause 'T 841000 Phone' must be in name=value form") where
Classic silently continued and let the server return whatever the well-
formed portion matched.
Add strict: bool = True to apply_filter_value. --filter callers stay
strict; apply_encoded_filter_value (the URL path) passes strict=False,
which logs a WARNING and skips the fragment instead of exiting.
Adds three tests: strict path still exits, non-strict path skips
silently, and end-to-end apply_values_from_url_params tolerates a value
with '&'.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds more robust parsing for filter clauses (especially when values contain = or URL fragments are malformed) and expands test coverage and fixtures for these edge cases.
Changes:
- Update filter parsing to split on the first
=only and supportstrictvs non-strict behavior. - Make URL-embedded filter parsing non-strict to match tabcmd Classic’s “skip invalid fragments” behavior.
- Add tests (and a CSV fixture) covering
=in values, empty values, and&-split URL fragments.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| tests/commands/test_datasources_and_workbooks_command.py | Adds regression tests for = inside filter values and non-strict URL fragment handling. |
| tests/assets/filter_test_data.csv | Adds fixture data with special characters used in filter parsing scenarios. |
| tabcmd/commands/datasources_and_workbooks/datasources_and_workbooks_command.py | Updates filter parsing logic and introduces strict mode to control behavior for malformed clauses. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| # fragment. Match tabcmd Classic's silent-skip behavior so drop-in | ||
| # migration of scripts that contain literal '&' in filter values | ||
| # (which the parser splits on) doesn't hard-fail. | ||
| logger.warning("Skipping unparseable filter clause from URL: %r", value) |
Black 22 (pinned in pyproject.toml) reformats these two files; the CI 'Check formatting with black' step was failing.
The prior fix for `?Product Name=AT&T 841000 Phone` matched Classic by
skipping the "T 841000 Phone" fragment, but it also applied the truncated
first fragment (`Product Name=AT`) as a real filter. On any dataset with
a matching "AT" row that produces silent wrong data with only a WARNING
log the user might miss.
Detect the pattern: if a fragment after '&' has no '=' and isn't a
tabcmd options key (":..."), assume it's a continuation of the previous
value and rejoin. Warns loudly that the URL should encode '&' as '%26'
so the ambiguity is visible.
Tests cover the AT&T case, multiple '&' in a value, options mixed with
filters, and multiple legitimate filters.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Two related client-side filter-parser bugs, both caught while porting
Classic scripts:
=inside a filter value was silently truncated (Notes=x=ybecameNotes=x).apply_filter_valuesplit on every=; fix issplit("=", maxsplit=1).&inside a URL-syntax filter value used to hard-error under the newstrict path introduced by the above fix (
?Product%20Name=AT&T%20841000%20Phone-- a real Classic script pattern -- got split on
&into a bogusT%20841000%20Phonefragment). Classic silently skipped suchfragments. On the URL path we now rejoin fragments where the fragment
after
&has no=and isn't an options key, warning loudly tosuggest
%26encoding.Behavior change
For users:
Product%20Name=Widget%20Plain(baseline)Product Name=x=y Config Kit(via--filter)Product%20Name=AT&T%20841000%20Phone(URL syntax)Note that spaces in URL query strings must be percent-encoded as
%20;literal spaces fail at tabcmd's URL parser (
cannot include spaces)before reaching this code path. The URL-path rejoin is stricter than
Classic's silent skip but avoids Classic's own footgun of silently
returning wrong data when the truncated prefix happens to match rows.
Adds
tests/assets/filter_test_data.csvwith product names containing=,&,,, backslashes, unicode, and quotes -- staged for future e2efilter tests, not consumed by tests in this PR yet.
Test plan
pytest tests/commands/test_datasources_and_workbooks_command.py— 34 passed
pytest tests/— 346 passed, 2 skippedtabcmd export --csv --filter "Product Name=x=y Config Kit"returns the expected row against a live servertabcmd get "views/escapeyvalues/Sheet1.csv?Product%20Name=AT&T%20841000%20Phone"returns the
AT&T 841000 Phonerow; WARNING logged suggesting%26encoding
🤖 Generated with Claude Code