Skip to content

ci: fast CI#207

Merged
maxday merged 15 commits into
mainfrom
maxday/fast-ci
Jun 18, 2026
Merged

ci: fast CI#207
maxday merged 15 commits into
mainfrom
maxday/fast-ci

Conversation

@maxday

@maxday maxday commented Jun 17, 2026

Copy link
Copy Markdown
Member

Description of changes:

  • Parallelize integration tests using GitHub Actions matrix strategy instead of running all distro/version combos
    sequentially via make test-integ
  • Split the monolithic amazonlinux job into amazonlinux2 (3.10, 3.11) and amazonlinux2023 (3.12, 3.13, 3.14)
  • Remove Python 3.9 from the test matrix (deprecated)
    Add explicit permissions: contents: read to the workflow
  • Fix Python 3.14 CI: exclude unavailable Docker images (alpine 3.19/3.20, debian bullseye) and add explicit combos for
    available ones (alpine 3.21, debian bookworm)
  • Fix test_success_and_failure_isolation on Python 3.14 by using explicit fork multiprocessing context (3.14 changed
    default start method to forkserver)
  • Add retry with exponential backoff for ECR Public image pulls to handle rate limiting

Target (OCI, Managed Runtime, both):
both

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Comment thread .github/workflows/test-on-push-and-pr.yml Fixed
@maxday maxday marked this pull request as ready for review June 17, 2026 18:08
@anzheyazzz

anzheyazzz commented Jun 17, 2026

Copy link
Copy Markdown

Why don't we add support for python3.14?

@maxday

maxday commented Jun 17, 2026

Copy link
Copy Markdown
Member Author

that's a good catch! adding it now!

@anzheyazzz anzheyazzz left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

run: DISTRO=alpine make test-integ
- name: Run integration test (alpine ${{ matrix.distro_version }} / python ${{ matrix.runtime_version }})
run: |
docker build -t codebuild-agent - < tests/integration/codebuild-local/Dockerfile.agent

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is repeated across all jobs, having it in a common job would reduce time further

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

true, but this is going away with #208 so if that looks good to you let's merge both?

Comment thread tests/test_concurrency.py
@@ -18,8 +18,9 @@ def setUp(self):
self.socket = "/tmp/sock"

def test_success_and_failure_isolation(self):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this test needs to be updated to reflect what are actually checking, using fork explicitly hides what's actually being used, so in the context of python 3.14 we force the test to use fork while we know the default now is to use forkserver. I'd use multiprocessing.Manager to track the shared data and this way we won't need to force usage of fork

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, will change this

Comment thread tests/test_concurrency.py Outdated
Comment on lines +45 to +47
), patch(
"awslambdaric.lambda_multi_concurrent_utils.multiprocessing.Process",
threading.Thread,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What i meant is that we should mock this at all, and rely on multiprocessing manager to track shared data, so here we're testing something different

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, I've changed back to no mock, but I still needed to add set_start_method as the default behaviour changed in python 3.14 (see: https://docs.python.org/3.14/whatsnew/3.14.html#concurrent-futures + python/cpython#84559)

@maxday maxday merged commit 296b39f into main Jun 18, 2026
36 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants