GTID 4/5: Persist GTID coordinates in state, enable resume, test both modes in CI#448
Open
driv3r wants to merge 3 commits into
Open
GTID 4/5: Persist GTID coordinates in state, enable resume, test both modes in CI#448driv3r wants to merge 3 commits into
driv3r wants to merge 3 commits into
Conversation
driv3r
force-pushed
the
gtid-stage3-streamer
branch
from
July 22, 2026 00:17
7d3a39d to
4abcb71
Compare
driv3r
force-pushed
the
gtid-stage4-state-resume
branch
2 times, most recently
from
July 22, 2026 13:50
5e4845c to
aa1df3b
Compare
grcooper
approved these changes
Jul 22, 2026
Make Ghostferry's serialized state and state tracker mode-aware so a GTID-mode run can be interrupted and resumed, and route writer/verifier state updates through GTID coordinates. - SerializableState gains BinlogCoordinateMode plus GTID counterparts of the three binlog positions (pointer + omitempty, so file/position states serialize unchanged). StateTracker stores GTID coordinates alongside file/position and selects the read path by mode. - Add mode-aware coordinate accessors and a GTID-aware MinSourceBinlogCoordinate that returns the intersection of the writer and inline-verifier GTID sets (the safe resume floor). Add intersectGTIDSets since go-mysql has no intersection primitive. - DMLEventBase can carry explicit coordinates; the BinlogStreamer stamps GTID coordinates onto events in GTID mode so BinlogWriter, inline verifier, and target verifier advance GTID-based state. - Ferry resume now uses the mode-aware coordinates and no longer rejects GTID resume. - Tests: GTID state round-trip, file/position backward-compatibility, GTID intersection resume, and tracker update/serialize/resume. File/position behavior and on-disk format are unchanged.
Allow the integration suite to run against file/position or GTID coordinate mode, and exercise both in CI. - integrationferry and the Go test helper read GHOSTFERRY_BINLOG_COORDINATE_MODE so every test can run in either mode. - Ruby harness accepts a per-test :binlog_coordinate_mode and falls back to the suite-wide GHOSTFERRY_BINLOG_COORDINATE_MODE env var. - GitHub Actions ruby-test matrix runs file_position on 5.7/8.0/8.4 and gtid on 8.0/8.4 only (GTID mode is MySQL 8+). - test_helper gains binlog_coordinate_mode / gtid_coordinate_mode? helpers; dumped-state and progress assertions in callbacks and interrupt/resume tests are made mode-aware. - Progress reports mode-aware LastSuccessfulBinlogCoordinate and FinalBinlogCoordinate (alongside the legacy file/position fields), and BinlogStreamer exposes GetStopBinlogCoordinate, so GTID runs report coherent progress. Verified locally against MySQL 8.0: full Go and Ruby suites pass in both file_position and gtid modes.
…tion - Reject resume when the serialized state's coordinate mode differs from the configured mode, preventing cross-mode mis-routing. - Check target-verifier coordinate presence explicitly (HasTargetVerifierBinlogCoordinate) rather than via IsZero(), so an empty (but valid) GTID set on resume is not mistaken for "absent" and does not skip pre-interrupt target events. - Make intersectGTIDSets and MinSourceBinlogCoordinate fail closed: a parse/set-math error returns an error instead of silently falling back to one side, which could advance the resume floor and skip events. Ferry.Start propagates the error. - Remove SetCoordinates from the exported DMLEvent interface; stamp GTID coordinates via an internal coordinateStamper type assertion so external DMLEvent implementers are unaffected. Use the cached NewGTIDCoordinateFromSet when stamping. - CI: run go-test in gtid mode on MySQL 8.0/8.4 (file_position on all). - Tests: multi-UUID intersection, fail-closed parse error, target verifier presence with empty GTID set.
driv3r
force-pushed
the
gtid-stage3-streamer
branch
from
July 22, 2026 19:00
2132529 to
e3d40fb
Compare
driv3r
force-pushed
the
gtid-stage4-state-resume
branch
from
July 22, 2026 19:00
aa1df3b to
53678c2
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked PR 4 of 5 — GTID support
Base: #447. Review/merge in stack order. Contains two commits.
Commit 1 — Persist GTID coordinates in state and enable GTID resume
SerializableStategainsBinlogCoordinateModeplus GTID counterparts of the three binlog positions (pointer + omitempty, so file/position states serialize unchanged).StateTrackerstores GTID coordinates alongside file/position and selects the read path by mode.MinSourceBinlogCoordinatereturning the intersection of the writer and inline-verifier GTID sets (safe resume floor); addsintersectGTIDSets(go-mysql has no intersection primitive).DMLEventBasecan carry explicit coordinates;BinlogStreamerstamps GTID coordinates onto events in GTID mode soBinlogWriter, inline verifier, and target verifier advance GTID-based state.Commit 2 — Test both binlog coordinate modes in Ruby suite and CI
GHOSTFERRY_BINLOG_COORDINATE_MODE.:binlog_coordinate_mode+ suite-wide env fallback.ruby-testrunsfile_positionon 5.7/8.0/8.4 andgtidon 8.0/8.4 only (GTID is MySQL 8+).test_helpergainsbinlog_coordinate_mode/gtid_coordinate_mode?; dumped-state and progress assertions made mode-aware.LastSuccessfulBinlogCoordinate/FinalBinlogCoordinate(alongside legacy fields);BinlogStreamer.GetStopBinlogCoordinateadded.File/position behavior and on-disk format are unchanged. Verified locally on MySQL 8.0: full Go and Ruby suites pass in both modes.