Skip to content

Flaky UT Fix - #380

Merged
yashnap merged 2 commits into
masterfrom
flaky_ut_fix
Aug 24, 2026
Merged

Flaky UT Fix#380
yashnap merged 2 commits into
masterfrom
flaky_ut_fix

Conversation

@yashnap

@yashnap yashnap commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

DESCRIPTION BY COPILOT AGENT:
The issue I fixed was a flaky unit test (test_update_file) that depended on:
sleep() timing, and file mtime comparisons.
On some CI filesystems, timestamp granularity is coarse (e.g., 1 second), so the file modification time didn’t always change as expected, causing intermittent failures even when behavior was correct.

I changed the test to be deterministic by validating status file content instead:

Confirm update_file("test1") does not change test.status.
Confirm update_file(file_name) updates test.status content as expected.
Description update
Reworked test_update_file in Test_ExtOutputStatusHandler.py to remove timing/mtime-based assertions that caused CI flakiness due to filesystem timestamp granularity.
The test now validates behavior using status-file content comparisons, ensuring deterministic verification that:

updating a different sequence number does not affect the original status file, and
updating the target sequence number correctly updates status payload fields.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adjusts a unit test timing delay intended to reduce flakiness when asserting that a status file’s mtime changes after an update.

Changes:

  • Increased time.sleep() in test_update_file from 0.03 to 0.08 seconds to try to exceed filesystem mtime granularity.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/extension/tests/Test_ExtOutputStatusHandler.py Outdated
@codecov

codecov Bot commented Aug 13, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.87%. Comparing base (488ab8e) to head (f0d6e9d).

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #380      +/-   ##
==========================================
- Coverage   94.87%   94.87%   -0.01%     
==========================================
  Files         111      111              
  Lines       20855    20844      -11     
==========================================
- Hits        19787    19776      -11     
  Misses       1068     1068              
Flag Coverage Δ
python27 94.87% <100.00%> (-0.01%) ⬇️
python312 94.87% <100.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Co-authored-by: yashnap <267671466+yashnap@users.noreply.github.com>
@yashnap yashnap changed the title Flaky UT Fix - TEST Flaky UT Fix Aug 13, 2026
Comment thread src/extension/tests/Test_ExtOutputStatusHandler.py

@kjohn-msft Koshy John (kjohn-msft) left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment inline

@yashnap
yashnap merged commit adc7934 into master Aug 24, 2026
10 checks passed
@yashnap
yashnap deleted the flaky_ut_fix branch August 24, 2026 19:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants