Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
0877d12
Add matrix:generate and deprecate older endpoints
ArBridgeman Jun 9, 2026
ec7f6f9
Add matrix.yml and remove older ones
ArBridgeman Jun 9, 2026
233eba4
Update PTB's workflows
ArBridgeman Jun 9, 2026
b0012bd
Fix missed workflow to matrix.yml
ArBridgeman Jun 9, 2026
c552e5e
Switch poetry.urls to project.urls
ArBridgeman Jun 10, 2026
814f2a0
Switch to a JSON array
ArBridgeman Jun 10, 2026
3fc086b
Alter so runs when workflow file is modified to reduce delayed errors
ArBridgeman Jun 10, 2026
15a8d40
Switch to jq and bash
ArBridgeman Jun 11, 2026
ed980e2
Put behind slow approval for PR too
ArBridgeman Jun 11, 2026
fdd7433
Merge branch 'main' into feature/654_add_and_use_general_matrix
ArBridgeman Jun 11, 2026
b9bf47b
Add summary description
ArBridgeman Jun 11, 2026
9eca9d6
Add documentation
ArBridgeman Jun 11, 2026
0be697e
Update .gitattributes for matrix.yml
ArBridgeman Jun 12, 2026
e4b9a78
Apply reviewer corrections to unreleased.md
ArBridgeman Jun 12, 2026
9b08dcf
Apply review improvement to create_and_update.rst
ArBridgeman Jun 12, 2026
a4257a7
Apply review improvements to workflow_variables.rst
ArBridgeman Jun 12, 2026
6edd52e
Fix indentation
ArBridgeman Jun 12, 2026
8fe1025
Improve docstring to give the pydantic context
ArBridgeman Jun 12, 2026
185f956
Switch to generate instead of dump
ArBridgeman Jun 12, 2026
54a47ff
Switch _ci.py to _matrix.py
ArBridgeman Jun 12, 2026
c62cdcb
Switch to deprecated_matrix_output and add docstring
ArBridgeman Jun 12, 2026
bfd8217
Align names once more
ArBridgeman Jun 12, 2026
bd7b518
Use format:fix
ArBridgeman Jun 12, 2026
3cb1e7e
Remove duplicate key check
ArBridgeman Jun 12, 2026
8d2c065
Remove unneeded test
ArBridgeman Jun 12, 2026
b9f9681
Remove unneeded call
ArBridgeman Jun 12, 2026
91754d4
Apply reviewer fix and remove somewhat duplicate text from the example
ArBridgeman Jun 12, 2026
a03e934
Switch to dictionary comprehension
ArBridgeman Jun 12, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitattributes
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ doc/changes/changelog.md linguist-generated=true
.github/workflows/dependency-update.yml linguist-generated=true
.github/workflows/fast-tests.yml linguist-generated=true
.github/workflows/gh-pages.yml linguist-generated=true
.github/workflows/matrix-*.yml linguist-generated=true
.github/workflows/matrix.yml linguist-generated=true
.github/workflows/merge-gate.yml linguist-generated=true
.github/workflows/periodic-validation.yml linguist-generated=true
.github/workflows/pr-merge.yml linguist-generated=true
Expand Down
41 changes: 0 additions & 41 deletions .github/workflows/matrix-exasol.yml

This file was deleted.

41 changes: 0 additions & 41 deletions .github/workflows/matrix-python.yml

This file was deleted.

25 changes: 16 additions & 9 deletions .github/workflows/matrix-all.yml → .github/workflows/matrix.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 12 additions & 0 deletions .github/workflows/merge-gate-extension.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,20 @@ on:
workflow_call:

jobs:
approve-run-slow-tests:
name: Approve Running Slow Tests?
runs-on: "ubuntu-24.04"
permissions:
contents: read
environment: manual-approval
steps:
- name: Tests
run: echo "Slow tests approved"

test-python-environment:
name: Test python-environment Action
uses: ./.github/workflows/test-python-environment.yml
needs:
- approve-run-slow-tests
permissions:
contents: read
10 changes: 6 additions & 4 deletions .github/workflows/slow-checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,14 @@ on:
jobs:
build-matrix:
name: Build Matrix
uses: ./.github/workflows/matrix-python.yml
uses: ./.github/workflows/matrix.yml
Comment thread
ckunki marked this conversation as resolved.
with:
matrix_keys_json: '["python_versions"]'
permissions:
contents: read

run-integration-tests:
name: Run Integration Tests (Python-${{ matrix.python-version }})
name: Run Integration Tests (Python-${{ matrix.python_versions }})
needs:
- build-matrix
runs-on: "ubuntu-24.04"
Expand All @@ -33,7 +35,7 @@ jobs:
id: set-up-python-and-poetry-environment
uses: exasol/python-toolbox/.github/actions/python-environment@v8
with:
python-version: ${{ matrix.python-version }}
python-version: ${{ matrix.python_versions }}
poetry-version: "2.3.0"

- name: Run Integration Tests
Expand All @@ -44,7 +46,7 @@ jobs:
id: upload-artifacts
uses: actions/upload-artifact@v7
with:
name: coverage-python${{ matrix.python-version }}-slow
name: coverage-python${{ matrix.python_versions }}-slow
path: .coverage
include-hidden-files: true
overwrite: false
16 changes: 9 additions & 7 deletions .github/workflows/test-python-environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@ jobs:

# Otherwise, check if any relevant files were changed in this PR
else
if git diff --quiet "origin/$BASE_REF...HEAD" -- .github/actions/python-environment; then
echo "No changes in .github/actions/python-environment, skipping."
if git diff --quiet "origin/$BASE_REF...HEAD" -- .github/actions/python-environment .github/workflows/test-python-environment.yml; then
echo "No changes in .github/actions/python-environment or .github/workflows/test-python-environment.yml, skipping."
echo "should_run=false" >> $GITHUB_OUTPUT
else
echo "Changes detected in directory, running."
Expand All @@ -41,7 +41,9 @@ jobs:
needs:
- check-changes
if: needs.check-changes.outputs.should_run == 'true'
uses: ./.github/workflows/matrix-all.yml
uses: ./.github/workflows/matrix.yml
with:
matrix_keys_json: '["python_versions"]'
permissions:
contents: read

Expand All @@ -60,8 +62,8 @@ jobs:
- int-linux-x64-4core-gpu-t4-ubuntu24.04-1
- int-linux-x64-4core-ubuntu24.04-1
- int-linux-x64-2core-ubuntu24.04-1
python-version: ${{ fromJson(needs.build-matrix.outputs.matrix).python-version }}
name: Verify Poetry Setup for ${{ matrix.runner }} (Python-${{ matrix.python-version }})
python-versions: ${{ fromJson(needs.build-matrix.outputs.matrix).python_versions }}
name: Verify Poetry Setup for ${{ matrix.runner }} (Python-${{ matrix.python-versions }})
runs-on:
labels: ${{ matrix.runner }}
steps:
Expand All @@ -75,7 +77,7 @@ jobs:
id: set-up-python-and-poetry-environment
uses: ./.github/actions/python-environment
with:
python-version: "${{ matrix.python-version }}"
python-version: "${{ matrix.python-versions }}"
poetry-version: "2.3.0"

- name: Check Poetry Version
Expand All @@ -85,7 +87,7 @@ jobs:
- name: Validate Python Version
id: validate-python-version
env:
PYTHON_VERSION: ${{ matrix.python-version }}
PYTHON_VERSION: ${{ matrix.python-versions }}
run: |
poetry run which python
poetry run python --version
Expand Down
26 changes: 25 additions & 1 deletion doc/changes/unreleased.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,32 @@

## Summary

This major version introduces `matrix.yml` whose results are generated by the nox session `matrix:generate`.

Projects can extend `BaseConfig` with additional matrix values when they need to expose
more entries to the workflows.

```python
class Config(BaseConfig):
extra_matrix_value: str = "extra"

@computed_field # type: ignore[misc]
@property
def computed_matrix_value(self) -> str:
# This can be requested when generating the matrix. If it is a simple string value,
# like is shown here, then the code will automatically wrap it in an array.
return f"{self.project_name}-computed"
```

The corresponding nox sessions (`matrix:all`, `matrix:exasol`, and `matrix:python`) will
remain available until September 15, 2026, to provide a transition period for existing projects.

At the same time, the workflows `matrix-all.yml`, `matrix-exasol.yml`, and `matrix-python.yml`
are deprecated and are no longer maintained by the exasol-toolbox. You can still use
these workflows in your project until you transition fully to using `matrix.yml`.

## Feature

* #730: Added support to extend GitHub workflow `cd.yml`
* #864: Modified PTB workflow templates to not persist credentials and to use pinned SHAs
* #864: Modified PTB workflow templates to not persist credentials and to use pinned SHAs
* #654: Added and used general matrix `matrix.yml` for PTB-provided workflows
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,10 @@ update existing workflows.

The workflows are based on Jinja templates with variables populated by the
PTB. The PTB reads the values from various attributes and properties of your
project's config, see :ref:`template_variables`.
project's config, see :ref:`workflow_templates`.
Comment thread
ArBridgeman marked this conversation as resolved.

Please note that the PTB only updates the values in the GitHub workflows when
*updating* the workflows. So, after updating the :ref:`list of Python versions
updating the workflows in general. So, after updating the :ref:`list of Python versions
<python_versions>` in file ``noxconfig.py``, you need to :ref:`re-generate
<update_workflows>` the GitHub workflows.

Expand Down Expand Up @@ -49,6 +49,11 @@ The PTB has a default for these versions, but you can override it in the
Some workflows are expected to not depend on a specific Python version and
will use only the lowest Python version in the list specified above.

If you need more than the built-in matrix inputs, you can extend
:class:`exasol.toolbox.config.BaseConfig` with extra properties or computed fields
and consume them from ``matrix.yml`` via ``matrix:generate``. See
:ref:`workflow_matrix` for an example.

.. _customize_workflows:

Customize Workflows for Your Project
Expand Down
22 changes: 10 additions & 12 deletions doc/user_guide/features/github_workflows/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ GitHub Workflow Templates

github_project_configuration
create_and_update
template_variables
workflow_variables
workflow_patcher

The PTB ships with configurable GitHub workflow templates covering the most common
Expand Down Expand Up @@ -44,6 +44,12 @@ The PTB has three categories of workflows:
Maintained by the PTB
^^^^^^^^^^^^^^^^^^^^^

.. note::
The ``matrix.yml`` workflow replaces the older ``matrix-all.yml``, ``matrix-exasol.yml``,
and ``matrix-python.yml`` workflows. The associated nox sessions (``matrix:all``,
``matrix:exasol``, and ``matrix:python``) are deprecated and will be removed
2026-09-15.

.. list-table::
:widths: 25 25 50
:header-rows: 1
Expand Down Expand Up @@ -79,18 +85,10 @@ Maintained by the PTB
* - ``gh-pages.yml``
- Workflow call
- Builds the documentation and deploys it to GitHub Pages.
* - ``matrix-all.yml``
- Workflow call
- Calls Nox session ``matrix:all``, which typically evaluates ``exasol_versions``
and ``python_versions`` from the ``PROJECT_CONFIG``.
* - ``matrix-exasol.yml``
- Workflow call
- Calls Nox session ``matrix:exasol`` to get the ``exasol_versions`` from the
``PROJECT_CONFIG``.
* - ``matrix-python.yml``
* - ``matrix.yml``
- Workflow call
- Calls Nox session ``matrix:python`` to get the ``python_versions`` from the
``PROJECT_CONFIG``.
- Calls Nox session ``matrix:generate`` to build a custom matrix from the
``PROJECT_CONFIG``. See :ref:`workflow_matrix`.
* - ``merge-gate.yml``
- Workflow call
- Acts as a final status check (gatekeeper) to ensure all required CI steps have
Expand Down
15 changes: 0 additions & 15 deletions doc/user_guide/features/github_workflows/template_variables.rst

This file was deleted.

Loading