Skip to content

fix: publish shared capa fixtures from a testing package - #286

Open
AhtishamShahid wants to merge 1 commit into
openedx:mainfrom
AhtishamShahid:ahtisham/fix-capa-fixtures-wheel-packaging
Open

fix: publish shared capa fixtures from a testing package#286
AhtishamShahid wants to merge 1 commit into
openedx:mainfrom
AhtishamShahid:ahtisham/fix-capa-fixtures-wheel-packaging

Conversation

@AhtishamShahid

@AhtishamShahid AhtishamShahid commented Aug 6, 2026

Copy link
Copy Markdown

The problem

openedx-platform imports capa test fixtures from this package in 17 of its own test modules. No 1.0.x wheel contains them, so those modules fail at collection:

ModuleNotFoundError: No module named 'xblocks_contrib.problem.capa.tests'

The cause is that two different files decide two different artifacts:

  • MANIFEST.in re-includes the fixtures, and says exactly the right thing about why — "We do this because openedx-platform imports these specific test helpers" — but it only governs the sdist.
  • The wheel is built from [tool.setuptools.packages.find], whose exclude = ["*tests*"] (added in Modernize Python repo: pyproject.toml + uv + semantic-release #251) drops the whole capa.tests package.

So the sdist has the fixtures, the wheel does not, and consumers install the wheel.

That is why openedx-platform is pinned back to 0.17.0 (openedx-platform@2d315ebd69) and why no 1.0.x release can reach a deployment — including the grading fix from #272, which course teams are still waiting on: a problem added to a graded subsection displays as 0 points / ungraded to the author who created it, while every other user sees it correctly.

The fix

Move the two fixtures consumers actually import into a capa/testing/ package. It is deliberately not under a tests/ directory, so the existing exclusion cannot match it — no packaging configuration changes at all.

fixture new location
response_xml_factory.py capa/testing/response_xml_factory.py — pure rename, zero content change
UseUnsafeCodejail capa/testing/codejail.py
helpers.py unchanged — stays internal, see below

Two details worth knowing:

  • tests/test_util.py mixes a real test case (UtilTest) with UseUnsafeCodejail, which only consumers use — so it is split, not moved. UtilTest stays put.
  • helpers.py does not move despite its MANIFEST.in entry: nothing outside this repo imports it, and its load_fixture resolves paths relative to tests/test_files. The stale MANIFEST.in block is removed — one of the three files it names no longer exists, and none of them need re-including.

Verification

  • Repo suite with relocated imports: 698 passed, 1 skipped.
  • Built wheel contains all three capa/testing/ modules and zero tests/ files.
  • Wheel installed into openedx-platform (Py3.12) with imports updated: the 17 previously-erroring modules collect cleanly (0 errors, was 17 erroring), and xmodule/tests/test_capa_block.py passes 201/201.
  • fix: fix the problem score when added in graded sub-section #272's fix then works: the author of a new problem in a graded subsection gets Progress(0, 1) instead of None.

For consumers

capa.tests.response_xml_factorycapa.testing.response_xml_factory
capa.tests.test_util.UseUnsafeCodejailcapa.testing.codejail.UseUnsafeCodejail

Deliberately not included

Kept out to keep this reviewable as a single fix — happy to follow up on any of them:

  • A CI check asserting wheel contents. This class of bug is invisible to the test suite (imports resolve fine from a source checkout while the wheel is missing modules), so nothing here stops capa/testing/ being silently re-excluded by a future glob edit — exclude = ["*test*"], one character shorter, would do it.
  • xblock_pdf/js_tests/ is shipped in released wheels (vitest config, tsconfig, spec helpers) — a pre-existing leak, unrelated to this bug.
  • Three undeclared dependencies are imported by shipped modules: six (capa/inputtypes.py, capa/responsetypes.py), symmath (capa/responsetypes.py), and web_fragments (xblock_pdf/pdf.py + 8 others). They work only because installers pull them in transitively.

Merge checklist:
Check off if complete or not applicable:

  • Version bumped (semantic-release, via fix:)
  • Changelog record added (generated by semantic-release)
  • Documentation updated (not only docstrings) — rationale in testing/__init__.py
  • Fixup commits are squashed away
  • Manual testing instructions provided
  • Noted any: Concerns, dependencies, migration issues, deadlines, tickets

This PR was created using Claude Opus 5

@openedx-webhooks openedx-webhooks added open-source-contribution PR author is not from Axim or 2U core contributor PR author is a Core Contributor (who may or may not have write access to this repo). labels Aug 6, 2026
@openedx-webhooks

openedx-webhooks commented Aug 6, 2026

Copy link
Copy Markdown

Thanks for the pull request, @AhtishamShahid!

This repository is currently maintained by @openedx/axim-engineering.

Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review.

🔘 Get product approval

If you haven't already, check this list to see if your contribution needs to go through the product review process.

  • If it does, you'll need to submit a product proposal for your contribution, and have it reviewed by the Product Working Group.
    • This process (including the steps you'll need to take) is documented here.
  • If it doesn't, simply proceed with the next step.
🔘 Provide context

To help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:

  • Dependencies

    This PR must be merged before / after / at the same time as ...

  • Blockers

    This PR is waiting for OEP-1234 to be accepted.

  • Timeline information

    This PR must be merged by XX date because ...

  • Partner information

    This is for a course on edx.org.

  • Supporting documentation
  • Relevant Open edX discussion forum threads
🔘 Get a green build

If one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green.

Details
Where can I find more information?

If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources:

When can I expect my changes to be merged?

Our goal is to get community contributions seen and reviewed as efficiently as possible.

However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:

  • The size and impact of the changes that it introduces
  • The need for product review
  • Maintenance status of the parent repository

💡 As a result it may take up to several weeks or months to complete a review and merge your PR.

@github-project-automation github-project-automation Bot moved this to Needs Triage in Contributions Aug 6, 2026
AhtishamShahid added a commit to AhtishamShahid/edx-platform that referenced this pull request Aug 6, 2026
Not for merge. The previous commit pinned the released 1.0.3 and CI failed
across every unit-test shard with

    ModuleNotFoundError: No module named 'xblocks_contrib.problem.capa.tests'

because no 1.0.x wheel ships the shared capa fixtures this repo imports from
its own test suite.

This commit repoints the requirement at the branch behind
openedx/xblocks-core#286, which fixes that packaging bug, so CI can
demonstrate that the version bump is otherwise clean.

Revert to a released pin (1.0.4) once edly-io#286 is merged and released.
@AhtishamShahid

Copy link
Copy Markdown
Author

Downstream verification: openedx/openedx-platform#38960 demonstrates both sides of this.

Its first commit pins the released 1.0.3 and every unit-test shard fails at collection with ModuleNotFoundError: No module named 'xblocks_contrib.problem.capa.tests' (run 31097321942). Its second commit points the requirement at this branch, after which the same modules collect cleanly (940 tests collected, 0 errors) and xmodule/tests/test_capa_block.py passes 201/201.

That PR is parked in draft until this one is merged and released.

@AhtishamShahid
AhtishamShahid marked this pull request as draft August 6, 2026 11:36
@AhtishamShahid
AhtishamShahid force-pushed the ahtisham/fix-capa-fixtures-wheel-packaging branch from 16cd54f to f7575a3 Compare August 6, 2026 11:47
AhtishamShahid added a commit to AhtishamShahid/edx-platform that referenced this pull request Aug 6, 2026
openedx/xblocks-core#286 moves the capa fixtures this repo imports out of
`xblocks_contrib.problem.capa.tests` and into a `capa.testing` package that
actually ships in the wheel:

    capa.tests.response_xml_factory -> capa.testing.response_xml_factory
    capa.tests.test_util.UseUnsafeCodejail -> capa.testing.codejail.UseUnsafeCodejail

Update all 17 importers accordingly, and move the VCS reference to that PR's
latest commit.

Verified with the resulting wheel installed: the previously-erroring modules
collect cleanly (1281 tests collected, 0 errors) and
xmodule/tests/test_capa_block.py passes 201/201.
@AhtishamShahid AhtishamShahid changed the title fix: ship shared capa test fixtures in the wheel fix: publish shared capa fixtures from a testing package Aug 6, 2026
@AhtishamShahid
AhtishamShahid force-pushed the ahtisham/fix-capa-fixtures-wheel-packaging branch from f7575a3 to dd39587 Compare August 6, 2026 11:57
AhtishamShahid added a commit to AhtishamShahid/edx-platform that referenced this pull request Aug 6, 2026
…ixup

No functional change; retargets the VCS reference at the amended commit on
openedx/xblocks-core#286.
@AhtishamShahid
AhtishamShahid force-pushed the ahtisham/fix-capa-fixtures-wheel-packaging branch from dd39587 to 22a6c1e Compare August 6, 2026 12:30
Downstream test suites import capa fixtures from this distribution, but no
1.0.x wheel contains them: `[tool.setuptools.packages.find]`'s `*tests*`
exclusion drops `xblocks_contrib.problem.capa.tests` entirely. MANIFEST.in
tries to hold them back --

    # We do this because openedx-platform imports these specific test helpers.
    include xblocks_contrib/problem/capa/tests/test_util.py
    ...

-- but MANIFEST.in only governs the sdist, so the intent never reached the
wheel. openedx-platform consequently fails to collect ~17 of its own test
modules against any 1.0.x release, and stays pinned to 0.17.0.

Move the two fixtures consumers actually import into a `capa/testing/`
package. It is deliberately not under a `tests/` directory, so the existing
exclusion cannot match it -- no packaging configuration changes at all:

* `capa/testing/response_xml_factory.py` -- moved verbatim; a pure factory
  module with no tests in it.
* `capa/testing/codejail.py` -- `UseUnsafeCodejail`, lifted out of
  `tests/test_util.py`. That module mixes a real test case (`UtilTest`) with a
  fixture only consumers use, so it is split rather than moved; `UtilTest`
  stays behind.

`capa/tests/helpers.py` stays internal: despite MANIFEST.in listing it, nothing
outside this repo imports it, and its `load_fixture` resolves paths relative to
`tests/test_files`. The now-stale MANIFEST.in block is removed -- one of the
three files it names no longer exists, and none of them need re-including.

Consumers change import paths as follows:

    from xblocks_contrib.problem.capa.tests.response_xml_factory import X
    -> from xblocks_contrib.problem.capa.testing.response_xml_factory import X

    from xblocks_contrib.problem.capa.tests.test_util import UseUnsafeCodejail
    -> from xblocks_contrib.problem.capa.testing.codejail import UseUnsafeCodejail

No released 1.0.x wheel exposes the old paths, so nothing installable can break.
@AhtishamShahid
AhtishamShahid force-pushed the ahtisham/fix-capa-fixtures-wheel-packaging branch from 22a6c1e to f532c45 Compare August 6, 2026 12:41
AhtishamShahid added a commit to AhtishamShahid/edx-platform that referenced this pull request Aug 6, 2026
openedx/xblocks-core#286 moves the capa fixtures this repo imports out of
`xblocks_contrib.problem.capa.tests`, which no 1.0.x wheel ships, into a
`capa.testing` package that does:

    capa.tests.response_xml_factory          -> capa.testing.response_xml_factory
    capa.tests.test_util.UseUnsafeCodejail   -> capa.testing.codejail.UseUnsafeCodejail

Update all 17 importers accordingly.
AhtishamShahid added a commit to AhtishamShahid/edx-platform that referenced this pull request Aug 6, 2026
Picks up the CapaBlock grading fix released in 1.0.3
(openedx/xblocks-core#272), which restores the correct score and graded state
for a problem added to a graded subsection when viewed by its author
(mitodl/hq#12023). Reverses the pin from 2d315eb.

Points at the branch behind openedx/xblocks-core#286 while that is unreleased,
because no released 1.0.x wheel ships the capa fixtures this repo's tests
import. Becomes `xblocks-contrib==1.0.4` once edly-io#286 is released.
AhtishamShahid added a commit to AhtishamShahid/edx-platform that referenced this pull request Aug 6, 2026
openedx/xblocks-core#286 moves the capa fixtures this repo imports out of
`xblocks_contrib.problem.capa.tests`, which no 1.0.x wheel ships, into a
`capa.testing` package that does:

    capa.tests.response_xml_factory          -> capa.testing.response_xml_factory
    capa.tests.test_util.UseUnsafeCodejail   -> capa.testing.codejail.UseUnsafeCodejail

Update all 17 importers accordingly.
AhtishamShahid added a commit to AhtishamShahid/edx-platform that referenced this pull request Aug 6, 2026
Picks up the CapaBlock grading fix released in 1.0.3
(openedx/xblocks-core#272), which restores the correct score and graded state
for a problem added to a graded subsection when viewed by its author
(mitodl/hq#12023). Reverses the pin from 2d315eb.

Points at the branch behind openedx/xblocks-core#286 while that is unreleased,
because no released 1.0.x wheel ships the capa fixtures this repo's tests
import. Becomes `xblocks-contrib==1.0.4` once edly-io#286 is released.
AhtishamShahid added a commit to AhtishamShahid/edx-platform that referenced this pull request Aug 6, 2026
Picks up the CapaBlock grading fix released in 1.0.3
(openedx/xblocks-core#272), which restores the correct score and graded state
for a problem added to a graded subsection when viewed by its author.
Reverses the pin from 2d315eb.

Points at the branch behind openedx/xblocks-core#286 while that is unreleased,
because no released 1.0.x wheel ships the capa fixtures this repo's tests
import. Becomes `xblocks-contrib==1.0.4` once edly-io#286 is released.
AhtishamShahid added a commit to AhtishamShahid/edx-platform that referenced this pull request Aug 6, 2026
openedx/xblocks-core#286 moves the capa fixtures this repo imports out of
`xblocks_contrib.problem.capa.tests`, which no 1.0.x wheel ships, into a
`capa.testing` package that does:

    capa.tests.response_xml_factory          -> capa.testing.response_xml_factory
    capa.tests.test_util.UseUnsafeCodejail   -> capa.testing.codejail.UseUnsafeCodejail

Update all 17 importers accordingly.
AhtishamShahid added a commit to AhtishamShahid/edx-platform that referenced this pull request Aug 6, 2026
Picks up the CapaBlock grading fix released in 1.0.3
(openedx/xblocks-core#272), which restores the correct score and graded state
for a problem added to a graded subsection when viewed by its author.
Reverses the pin from 2d315eb.

Points at the branch behind openedx/xblocks-core#286 while that is unreleased,
because no released 1.0.x wheel ships the capa fixtures this repo's tests
import. Becomes `xblocks-contrib==1.0.4` once edly-io#286 is released.
@AhtishamShahid
AhtishamShahid marked this pull request as ready for review August 7, 2026 06:57
@AhtishamShahid
AhtishamShahid requested a review from farhan August 7, 2026 06:58
@mphilbrick211 mphilbrick211 moved this from Needs Triage to Ready for Review in Contributions Aug 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core contributor PR author is a Core Contributor (who may or may not have write access to this repo). open-source-contribution PR author is not from Axim or 2U

Projects

Status: Ready for Review

Development

Successfully merging this pull request may close these issues.

3 participants