Skip to content

TEZ-4723: TestMapProcessor is flaky#504

Open
abstractdog wants to merge 2 commits into
apache:masterfrom
abstractdog:TEZ-4723
Open

TEZ-4723: TestMapProcessor is flaky#504
abstractdog wants to merge 2 commits into
apache:masterfrom
abstractdog:TEZ-4723

Conversation

@abstractdog
Copy link
Copy Markdown
Contributor

No description provided.

@abstractdog abstractdog requested a review from ayushtkn June 2, 2026 09:54
output.collect(key, value);
if (counter.incrementAndGet() % N == 0) {
try {
Thread.sleep(1);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will not fix it I believe, it will increase the probability of it passing I believe. Can we wait in the Test's monitoring thread for some X condition instead? Like maybe using GenericTestUtils.waitFor()

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agreed, the current solution is just a sophisticated Thread.sleep :) let me improve it

@tez-yetus
Copy link
Copy Markdown

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 5m 38s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 detsecrets 0m 1s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ master Compile Tests _
+1 💚 mvninstall 3m 12s master passed
+1 💚 compile 2m 25s master passed
+1 💚 checkstyle 0m 20s master passed
+1 💚 javadoc 0m 22s master passed
+0 🆗 spotbugs 0m 35s tez-mapreduce in master has 97 extant spotbugs warnings.
_ Patch Compile Tests _
+1 💚 mvninstall 2m 8s the patch passed
+1 💚 codespell 0m 53s No new issues.
+1 💚 compile 2m 24s the patch passed
+1 💚 javac 2m 24s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 14s the patch passed
+1 💚 javadoc 0m 18s the patch passed
+1 💚 spotbugs 0m 39s the patch passed
_ Other Tests _
+1 💚 unit 58m 27s root in the patch passed.
+1 💚 asflicense 0m 20s The patch does not generate ASF License warnings.
78m 56s
Subsystem Report/Notes
Docker ClientAPI=1.54 ServerAPI=1.54 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-504/1/artifact/out/Dockerfile
Optional Tests dupname compile unit asflicense javac javadoc spotbugs checkstyle codespell detsecrets
uname Linux 0518a0fadd0b 5.15.0-173-generic #183-Ubuntu SMP Fri Mar 6 13:29:34 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality tez-personality.sh
git revision master / 330fdc8
Default Java Eclipse Adoptium-21.0.11+10-LTS
Test Results https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-504/1/testReport/
Max. process+thread count 1344 (vs. ulimit of 5500)
modules C: tez-mapreduce U: tez-mapreduce
Console output https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-504/1/console
versions git=2.43.0 maven=3.9.15 spotbugs=4.9.3 codespell=2.4.1
Powered by Apache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

@abstractdog abstractdog force-pushed the TEZ-4723 branch 2 times, most recently from 5a07af8 to 8521d5d Compare June 3, 2026 11:39
@abstractdog
Copy link
Copy Markdown
Contributor Author

@ayushtkn : created LatchMapper, please let me know if it's fine

@tez-yetus
Copy link
Copy Markdown

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 2m 20s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ master Compile Tests _
+1 💚 mvninstall 3m 3s master passed
+1 💚 compile 2m 27s master passed
+1 💚 checkstyle 0m 23s master passed
+1 💚 javadoc 0m 24s master passed
+0 🆗 spotbugs 0m 35s tez-mapreduce in master has 97 extant spotbugs warnings.
_ Patch Compile Tests _
+1 💚 mvninstall 2m 11s the patch passed
+1 💚 codespell 0m 48s No new issues.
+1 💚 compile 2m 31s the patch passed
+1 💚 javac 2m 31s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 18s tez-mapreduce: The patch generated 0 new + 2 unchanged - 2 fixed = 2 total (was 4)
+1 💚 javadoc 0m 22s the patch passed
+1 💚 spotbugs 0m 43s the patch passed
_ Other Tests _
+1 💚 unit 58m 48s root in the patch passed.
+1 💚 asflicense 0m 21s The patch does not generate ASF License warnings.
76m 17s
Subsystem Report/Notes
Docker ClientAPI=1.54 ServerAPI=1.54 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-504/5/artifact/out/Dockerfile
Optional Tests dupname compile unit asflicense javac javadoc spotbugs checkstyle codespell detsecrets
uname Linux 15ea1b7fa1d1 5.15.0-173-generic #183-Ubuntu SMP Fri Mar 6 13:29:34 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality tez-personality.sh
git revision master / 330fdc8
Default Java Eclipse Adoptium-21.0.11+10-LTS
Test Results https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-504/5/testReport/
Max. process+thread count 2127 (vs. ulimit of 5500)
modules C: tez-mapreduce U: tez-mapreduce
Console output https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-504/5/console
versions git=2.43.0 maven=3.9.15 spotbugs=4.9.3 codespell=2.4.1
Powered by Apache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants