Skip to content

fix(grading): stop autograder from silently crashing under bash 5 - #9

Merged
lassebenni merged 1 commit into
mainfrom
fix/grader-arithmetic-crash
Jul 12, 2026
Merged

fix(grading): stop autograder from silently crashing under bash 5#9
lassebenni merged 1 commit into
mainfrom
fix/grader-arithmetic-crash

Conversation

@lassebenni

Copy link
Copy Markdown
Contributor

Summary

  • ((var += expr)) reports a failing exit status whenever the resulting arithmetic value is 0 -- a well-known bash gotcha. Combined with set -euo pipefail, the grader silently dies mid-run the moment a student's running score hits exactly 0 at a checkpoint (e.g. a mostly-empty early submission).
  • This only shows up on GitHub Actions' Ubuntu runners (bash 5+); it's invisible when testing locally on macOS (bash 3.2 ships without this strictness), which is how it went unnoticed.
  • Also guards bare check_*() helper calls (which legitimately return 1 to signal warn/fail, not a script error) with || true so they don't trip set -e either.

Test plan

  • bash -n .hyf/test.sh
  • Reproduced against an empty scaffold repo with GNU bash 5: exit code 1 before the fix, exit code 0 after.
  • Re-ran against a known-good local solution: score/pass output unchanged (regression-free).
  • Confirm a real PR run in CI now completes the "Run tests" step for a low/zero-scoring submission.

🤖 Generated with Claude Code

`((score += l1))`-style compound assignment is a false command in bash
whenever the resulting value is 0 -- exit status 1 even though the
assignment succeeded. Under `set -e` (used by every autograder here),
that silently kills the script mid-run on GitHub Actions' Ubuntu
runners (bash 5), while working fine locally on macOS (bash 3.2, which
does not enforce this). Any student whose running score hits exactly 0
at a checkpoint -- most commonly an early, mostly-empty submission --
got a grading run that failed with a bare "exit code 1" and no
score.json update, not a real 0 score.

Reproduced locally with bash 5 against an empty scaffold; confirmed
exit 0 after the fix, and re-verified real solution runs still score
correctly. Fix: rewrite every `((var += expr))`/`((var -= expr))` as
`var=$((var + expr))`, and guard bare check_*() calls that can
legitimately return non-zero (a warn/fail signal, not a script error)
with `|| true` so `set -e` does not treat them as fatal.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

📝 HackYourFuture auto grade

Assignment Score: 18 / 100 ❌

Status: ❌ Not passed
Minimum score to pass: 60
🧪 The auto grade is experimental and still being improved

Test Details

=== Week 6 Autograder ===
  ✓ PASS  found Dockerfile
  ✓ PASS  found requirements.txt
  ✓ PASS  found src/pipeline.py
  ✓ PASS  found AI_ASSIST.md
  ✓ PASS  found README.md
  ✓ PASS  found docs/ directory
  ✓ PASS  Level 1: required files (10/10 pts)
  ✗ FAIL  requirements.txt does not pin azure-storage-blob (expected line like 'azure-storage-blob==12.x.y')
  ✗ FAIL  requirements.txt does not pin psycopg2-binary (expected line like 'psycopg2-binary==2.x.y')
  ✓ PASS  Level 2: pinned dependencies (0/10 pts)
  ✓ PASS  Dockerfile uses a python:3.11 base image
  ✗ FAIL  Dockerfile does not copy requirements.txt before src/ (cache-unfriendly)
  ✗ FAIL  Dockerfile CMD is still the placeholder — replace it with the pipeline entry point
  ✓ PASS  Level 3: Dockerfile (3/10 pts)
  ✓ PASS  pipeline.py reads POSTGRES_URL and AZURE_STORAGE_CONNECTION_STRING from env
  ✗ FAIL  pipeline.py does not use 'from contextlib import closing' + 'with closing(...)' (Chapter 4 pattern)
  ✗ FAIL  pipeline.py does not silence the azure SDK logger (logging.getLogger("azure").setLevel(...))
  ⚠ WARN  pipeline.py still contains 'raise NotImplementedError' — finish the stubs before submitting
  ✓ PASS  Level 4: pipeline shape (5/15 pts)
  ✗ FAIL  pipeline.py does not use ON CONFLICT ... DO UPDATE (idempotent upsert)
  ✗ FAIL  pipeline.py does not use %s placeholders for parameterised SQL
  ✓ PASS  Level 5: idempotent upsert (0/15 pts)
  ✗ FAIL  no mention of sslmode=require in the repo — Azure Postgres rejects connections without it
  ✗ FAIL  pipeline.py does not use the azure-storage-blob SDK (BlobServiceClient)
  ✓ PASS  Level 6: connection + SDK (0/10 pts)
  ✗ FAIL  AI_ASSIST.md has section headers but is not filled in (665 chars, 3 TODO line(s); target 1800+ chars, 0 TODOs)
  ✓ PASS  Level 7: AI report (0/10 pts)
  ✗ FAIL  README.md is missing the '## Verification' heading (Task 5)
  ✓ PASS  Level 8: README verification (0/10 pts)
  ✗ FAIL  docs/execution_history.png not found (Task 5 deliverable)
  ✓ PASS  Level 9: execution screenshot (0/10 pts)
  ✓ PASS  .gitignore correctly excludes __pycache__/, *.pyc, and .env

Score: 18 / 100  (passing: 60)  pass=false

@lassebenni
lassebenni merged commit c88c5a4 into main Jul 12, 2026
1 check failed
@lassebenni
lassebenni deleted the fix/grader-arithmetic-crash branch July 12, 2026 21:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant