e2e: Fix the flaky test races and re-enable the composer giphy test#6570
Draft
andremion wants to merge 8 commits into
Draft
e2e: Fix the flaky test races and re-enable the composer giphy test#6570andremion wants to merge 8 commits into
andremion wants to merge 8 commits into
Conversation
waitToAppear waited for hasObject once and then read findObject once. Under active recomposition the node the wait just saw can be recycled before the single read reaches it, failing in a few hundred ms with a misleading timeout message; the failure screenshots show the app in the correct state. Both overloads now poll at the existing 50ms interval within the same timeout budget and absorb StaleObjectException between polls.
assertMessageDeliveryStatus and assertMessageFailedIcon used wait().isDisplayed(), the same wait-once-read-once race in a second shape; the historic failures of test_deliveryStatusShownForTheLastMessageInGroup died in 460-572ms. assertVisibility gains a timeout parameter (the READ branch keeps its 30s) and these asserts go through it. On a genuine timeout they now surface as IllegalStateException (Allure broken) instead of AssertionError (failed).
Back-ports from stream-video-android PR 1746: the recording file uses methodName (displayName's parentheses break when recording commands go through the device shell, which depends on the uiautomator version), and a recording stop failure logs and skips the video attachment instead of replacing the test result or skipping the failure reporting.
test_userObservesAnimatedGiphy_afterAddingGiphyThroughComposerMenu was ignored pending the composer redesign (AND-1018), which shipped in January. The redesign removed the dedicated commands button, so the flow now types '/' to open the command suggestion list. Text entry uses the direct UiObject2.typeText: the shell-based input leaks its wrapping quotes into the field on this uiautomator version (the composer literally contained '/'), which also silently affected the command branch of uploadGiphy. The dead Stream_ComposerCommandsButton selector is removed.
No usages remain, and on this uiautomator version the shell command's wrapping quotes reach the input field as literal text. Direct UiObject2.typeText is the working path everywhere.
Contributor
PR checklist ✅All required conditions are satisfied:
🎉 Great job! This PR is ready for review. |
Contributor
Author
|
@CodeRabbit review |
✅ Action performedReview finished.
|
Contributor
SDK Size Comparison 📏
|
WalkthroughThe E2E test updates replace the composer command button flow with slash-triggered suggestions, standardize message visibility waits, harden retry screen recording, move text entry to ChangesE2E test behavior
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related PRs
Suggested labels: Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
The command branch of uploadGiphy tapped the send button with a one-shot findObject right after setting the input text. The send button only renders once the input is non-empty, and on a slow CI emulator the recomposition can still be in flight, so the read returned null and all three attempts failed. tapOnSendButton already waits and is what every other send path uses.
Selecting a command suggestion activates command mode, where the composer's trailing button is the save button instead of the send button. Whether the suggestion tap lands depends on the animation state (CI disables animations, local emulators usually do not), so the giphy command flow saw the send button locally and the save button on CI. With the sample's configuration both buttons build the same message, so the robot taps whichever the composer settled on.
|
A node returned by waitToAppear can be recycled by recomposition before the caller reads visibleCenter, so assertTrue(waitToAppear().isDisplayed()) still threw StaleObjectException after the wait itself was fixed. waitDisplayed polls find-and-check as one operation and reports the outcome, so asserts never hold a node across reads. All 14 assert sites move to it; assertGiphyImage's stale-retry wrapper becomes redundant. waitToAppear stays for interaction flows that need the node.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Goal
Remove the two client-side race mechanisms behind the flaky e2e tests, harden the RetryRule screen recording, and re-enable the first of the ignored tests. Part of AND-1303; the remaining flake family (mock server behavior under WebSocket reconnects) is addressed in GetStream/stream-chat-test-mock-server#43.
Root causes
Diagnosed from the per-attempt artifacts (logcat, screenshot, hierarchy, step timings) that AND-1302's retry reporting attached to every flake of the 2026-07-13 nightly, plus the step durations of historic failures in TestOps:
waitToAppearStaleObjectException; screenshots show the correct app statethreadIsNotLocked...,userAddsReactionWhileOfflinewait().isDisplayed()AssertionError(460 to 572ms)deliveryStatusShown...teststhreadReplyIsRemovedEverywhere...,originalQuoteIsDeleted...,channelPreviewShowsPreviousMessage...,deletesReaction— tracked in the mock repoImplementation
waitToAppear(both overloads) pollsfindObjectat the existing 50ms interval within the same timeout budget and absorbsStaleObjectExceptionbetween polls.waitDisplayedpredicate: the find-and-check happens inside the poll, so asserts never hold aUiObject2across reads (a held node can be recycled by recomposition beforevisibleCenteris read, which CI demonstrated after the first two fixes).assertMessageDeliveryStatusandassertMessageFailedIconroute through the same helper (the READ branch keeps its 30s timeout).RetryRule: the recording file is named bymethodName(the display name's parentheses can break when the recording commands go through the device shell), andstopRecordingSafelymakes sure a recording infrastructure failure cannot change a test result or skip the failure reporting.test_userObservesAnimatedGiphy_afterAddingGiphyThroughComposerMenure-enabled: its blocking ticket (AND-1018, composer redesign) shipped in January. The flow is adapted to the redesigned composer (typing/opens the command suggestion list; the dedicated commands button no longer exists).UiDevice.typeTextis removed: on this uiautomator version the shell command's wrapping quotes reach the input field as literal text, and nothing uses it after the flow fix.Testing
All on a local emulator against the mock server:
MessageListTests#test_threadIsNotLocked_afterParentMessageDeletedByParticipant(the race-prone test): 3/3 green, first attempt each time.MessageDeliveryStatusTests#test_singleCheckmarkShown_whenMessageIsSentand#test_deliveryStatusShownForTheLastMessageInGroup: green through the new assert routing.GiphyTests#test_userObservesAnimatedGiphy_afterAddingGiphyThroughComposerMenu(re-enabled): 2/2 green on the adapted flow.To reproduce any of them:
./gradlew :stream-chat-android-compose-sample:connectedE2eDebugAndroidTest -Pandroid.testInstrumentationRunnerArguments.class=<class>#<method>with the mock server driver running.No UI changes.
Summary by CodeRabbit
New Features
/in the message field.Bug Fixes
Tests