[SYNPY-1840] Add ability to set column order when creating file and record based tasks. - #1443
Merged
Conversation
andrewelamb
marked this pull request as draft
August 13, 2026 14:45
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
andrewelamb
marked this pull request as ready for review
August 13, 2026 15:04
BryanFauble
approved these changes
Aug 13, 2026
| Synapse.get_client(synapse_client=syn).logger.exception( | ||
| f"Could not delete the created EntityView {view.id}. It " | ||
| "must be deleted manually." | ||
| ) |
Member
There was a problem hiding this comment.
What does "deleted manually" mean here?
Contributor
Author
There was a problem hiding this comment.
I'm changing this to
f"Could not delete the created EntityView {view.id}. Delete it "
"yourself, either from the Synapse web UI, or with the Python "
f"client: EntityView(id='{view.id}').delete()"
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem:
The curator metadata task creators gave callers no control over the left-to-right order of the columns that contributors see in the grid.
create_record_based_metadata_taskbuilt its CSV template from the JSON Schema properties in whatever orderextract_schema_properties_from_webreturned them, and only moved theupsert_keysto the front.create_file_based_metadata_taskhard-coded a single ordering in_create_json_schema_entity_view:name,id,createdBywere reordered to positions 0, 1, 2 and everything else kept the order Synapse gave it.JSON Schema property order is not reliably preserved by downstream applications, so there was no dependable way to make contributor-relevant metadata appear first. Pinning
createdByin third position made this worse for file-based tasks — a system column was always placed ahead of every curation column.Solution:
Add an optional keyword-only
column_order: list[str] | Noneparameter to bothcreate_record_based_metadata_taskandcreate_file_based_metadata_task.Ordering semantics are shared by both functions and live in two new helpers in
synapseclient/extensions/curator/utils.py:validate_column_order_list()— the cheap shape check (is a list, entries are non-empty strings, no duplicates). Called at the top of both public functions so malformed input fails before any entity is created in Synapse.resolve_column_order_list()— produces the final order: pinned columns, then the requested columns that were not already pinned, then the remaining available columns in their existing relative order. A column never appears twice, so naming a pinned column is a no-op rather than a duplication. This is also where the unknown-column check happens.The result is that callers only name the columns that need intentional placement; everything else stays visible and is appended afterwards. Pinned columns are the
upsert_keysfor record-based tasks andname+idfor file-based tasks.Notable details:
createdByis no longer pinned for file-based tasks. Onlynameandidare.createdBy,modifiedOn, and the other Synapse-managed columns are now orderable like any other column, which is what lets curation metadata precede system metadata.Docs: a new "Controlling the order of the columns" section in
docs/guides/extensions/curator/metadata_curation.md, plus a worked example in each function's docstring.Testing:
Unit tests added to
tests/unit/synapseclient/extensions/unit_test_curator.py:validate_column_order_list()—None, non-list input, non-string and empty-string entries, duplicates, valid input passthrough.resolve_column_order_list()— pinned-only, partial request, request naming a pinned column (no duplication, stays pinned), unknown column raisesValueError, remaining columns keep relative order.column_orderproduces the expected CSV template header (patientId,specimenID,diagnosis,age,assayfrom a partial order that redundantly names an upsert key), and a malformedcolumn_orderraises before the schema is fetched.column_orderis forwarded to_create_json_schema_entity_view, the resultingview.columnsis in the expected order, a malformedcolumn_orderraises before the entity view is created, and an unknown column name deletes the created view before raising — including the case where the cleanup delete itself fails, which logs and still raises theValueError.Existing call-site assertions were updated for the new
column_order=Noneargument.Two existing
unittest.TestCaseclasses in this file (TestRecordBasedHelperFunctions,TestFileBasedHelperFunctions) were converted to plain pytest classes so the new cases could usepytest.mark.parametrizeand fixtures, in line with the test conventions intests/CLAUDE.md.