From 950a6200dc8c2dba379b039610b45772e576089b Mon Sep 17 00:00:00 2001 From: whisper67265 Date: Tue, 14 Jul 2026 13:54:39 -0600 Subject: [PATCH 1/3] Weblate outbound POST JSON schema + schema-driven bats --- .github/workflows/lint.yml | 8 ++- docs/endpoint-contract.md | 28 ++++++++- .../weblate-add-or-update.request.schema.json | 46 +++++++++++++++ scripts/ensure_check_jsonschema.sh | 50 ++++++++++++++++ scripts/lint.sh | 16 +++++ scripts/test.sh | 5 ++ .../weblate-add-or-update-request-valid.json | 8 +++ tests/helpers/schema.bash | 58 +++++++++++++++++++ tests/helpers/test_helper.bash | 2 + tests/test_trigger_weblate.bats | 14 ++--- 10 files changed, 225 insertions(+), 10 deletions(-) create mode 100644 docs/schemas/weblate-add-or-update.request.schema.json create mode 100755 scripts/ensure_check_jsonschema.sh create mode 100644 tests/helpers/fixtures/weblate-add-or-update-request-valid.json create mode 100644 tests/helpers/schema.bash diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml index ee12781..fdee197 100644 --- a/.github/workflows/lint.yml +++ b/.github/workflows/lint.yml @@ -18,7 +18,11 @@ jobs: with: persist-credentials: false - - name: ShellCheck and actionlint + - name: Install Python for schema validation + if: runner.os == 'Linux' + run: sudo apt-get update && sudo apt-get install -y python3-venv python3-pip + + - name: ShellCheck, actionlint, and JSON Schema run: scripts/lint.sh test: @@ -31,7 +35,7 @@ jobs: persist-credentials: false - name: Install test dependencies - run: sudo apt-get update && sudo apt-get install -y bats jq curl + run: sudo apt-get update && sudo apt-get install -y bats jq curl python3-pip python3-venv - name: Run tests run: make test diff --git a/docs/endpoint-contract.md b/docs/endpoint-contract.md index 07adb73..9b1177e 100644 --- a/docs/endpoint-contract.md +++ b/docs/endpoint-contract.md @@ -52,7 +52,7 @@ Semver applies to documented changes in: | `repository_dispatch` **`event_type`** values | `add-submodules`, `start-translation`, `sync-translation` | | **`client_payload`** field names, optionality, semantics | [README](../README.md) workflow tables; sections below | | Dispatch HTTP contract | URL, auth headers, success = HTTP **204** | -| Outbound Weblate POST | `organization`, `version`, `extensions`, `add_or_update`; success **200** or **202** | +| Outbound Weblate POST | Request schema [weblate-add-or-update.request.schema.json](schemas/weblate-add-or-update.request.schema.json); success **200** or **202** | | Shell batch return codes **0 / 1 / 2** and job collapse | [ARCHITECTURE §6](ARCHITECTURE.md#6-shell-return-codes) — code **2** never propagates to GitHub Actions step exit | | Branch/path constants affecting behavior | `MASTER_BRANCH`, `LOCAL_BRANCH_PREFIX`, `TRANSLATION_BRANCH_PREFIX`, `WEBLATE_ENDPOINT_PATH` in [`env.sh`](../.github/workflows/assets/env.sh) | @@ -134,3 +134,29 @@ submodule pointer updates, and Boost release refs in `client_payload.version`. | Auth | `Authorization: Token {WEBLATE_TOKEN}` | | Invoked by | `.github/workflows/start-translation.yml` (`start-local` job) | | Success | HTTP **200** or **202** (async accepted) | + +### Payload + +Field names, types, and constraints are defined by +**[weblate-add-or-update.request.schema.json](schemas/weblate-add-or-update.request.schema.json)** +(source of truth). Summary: + +| Field | Shape | +| ----- | ----- | +| `organization` | Non-empty string (`MODULE_ORG`) | +| `version` | Non-empty string (Boost libs ref) | +| `extensions` | Array of dot-prefixed strings (may be `[]`) | +| `add_or_update` | Object: language code → non-empty array of submodule basenames | + +Built by `trigger_weblate` in `.github/workflows/assets/translation.sh`. The call is +omitted when `add_or_update` would be empty. + +### Success response bodies + +| HTTP status | Body | +| ----------- | ---- | +| **202** | JSON object with required string `task_id` (async acceptance) | +| **200** | JSON object with required string `status` equal to `"ok"` (sync completion) | + +Orchestration treats both statuses as success and prints the response body to logs; +it does not otherwise act on response fields. diff --git a/docs/schemas/weblate-add-or-update.request.schema.json b/docs/schemas/weblate-add-or-update.request.schema.json new file mode 100644 index 0000000..68ea4c5 --- /dev/null +++ b/docs/schemas/weblate-add-or-update.request.schema.json @@ -0,0 +1,46 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://github.com/boostorg/boost-docs-translation/docs/schemas/weblate-add-or-update.request.schema.json", + "title": "Weblate add-or-update request", + "description": "Outbound POST body built by trigger_weblate for WEBLATE_ENDPOINT_PATH.", + "type": "object", + "additionalProperties": false, + "required": ["organization", "version", "extensions", "add_or_update"], + "properties": { + "organization": { + "type": "string", + "minLength": 1, + "description": "GitHub org hosting library mirror repos (MODULE_ORG)." + }, + "version": { + "type": "string", + "minLength": 1, + "description": "Boost libs ref (e.g. develop or boost-1.90.0)." + }, + "extensions": { + "type": "array", + "description": "Dot-prefixed file extensions filter for Weblate; empty means no filter.", + "items": { + "type": "string", + "pattern": "^\\..+" + } + }, + "add_or_update": { + "type": "object", + "minProperties": 1, + "description": "Map of language code to non-empty arrays of submodule basenames.", + "propertyNames": { + "type": "string", + "pattern": "^[A-Za-z]{2,3}([_-][A-Za-z0-9]{2,8})*$" + }, + "additionalProperties": { + "type": "array", + "minItems": 1, + "items": { + "type": "string", + "pattern": "^[a-z][a-z0-9._-]*$" + } + } + } + } +} diff --git a/scripts/ensure_check_jsonschema.sh b/scripts/ensure_check_jsonschema.sh new file mode 100755 index 0000000..620c854 --- /dev/null +++ b/scripts/ensure_check_jsonschema.sh @@ -0,0 +1,50 @@ +#!/usr/bin/env bash +# Bootstrap pinned check-jsonschema into .cache/ (shared by lint.sh and test helpers). +# Safe to source: does not enable set -euo (callers own shell options). +# shellcheck shell=bash + +CHECK_JSONSCHEMA_VERSION="${CHECK_JSONSCHEMA_VERSION:-0.37.4}" + +ensure_check_jsonschema() { + local root version cache_dir venv_dir bin marker + root="${REPO_ROOT:-}" + if [[ -z "$root" ]]; then + root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" + fi + version="$CHECK_JSONSCHEMA_VERSION" + cache_dir="$root/.cache/check-jsonschema" + venv_dir="$cache_dir/${version}" + bin="$venv_dir/bin/check-jsonschema" + marker="$venv_dir/.installed" + + if [[ -x "$bin" && -f "$marker" ]]; then + CHECK_JSONSCHEMA_BIN="$bin" + export CHECK_JSONSCHEMA_BIN + return 0 + fi + + if ! command -v python3 >/dev/null 2>&1; then + echo "ensure_check_jsonschema: python3 is required" >&2 + return 1 + fi + + echo "ensure_check_jsonschema: installing check-jsonschema==${version} into ${venv_dir}..." >&2 + rm -rf "$venv_dir" + python3 -m venv "$venv_dir" || return 1 + "$venv_dir/bin/pip" install --disable-pip-version-check --quiet \ + "check-jsonschema==${version}" || return 1 + if [[ ! -x "$bin" ]]; then + echo "ensure_check_jsonschema: expected binary missing at $bin" >&2 + return 1 + fi + printf '%s\n' "$version" >"$marker" + CHECK_JSONSCHEMA_BIN="$bin" + export CHECK_JSONSCHEMA_BIN +} + +# When sourced, define the function; when executed, run it and print the bin path. +if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then + set -euo pipefail + ensure_check_jsonschema + printf '%s\n' "$CHECK_JSONSCHEMA_BIN" +fi diff --git a/scripts/lint.sh b/scripts/lint.sh index 5b6faec..f9d0eb3 100755 --- a/scripts/lint.sh +++ b/scripts/lint.sh @@ -177,3 +177,19 @@ echo "lint: actionlint ${ACTIONLINT_VERSION} ($("$ACTIONLINT_BIN" -version | hea tests/helpers/*.bash "$ACTIONLINT_BIN" -color + +# shellcheck source=scripts/ensure_check_jsonschema.sh +source "$ROOT/scripts/ensure_check_jsonschema.sh" +ensure_check_jsonschema + +echo "lint: check-jsonschema ${CHECK_JSONSCHEMA_VERSION}" >&2 +shopt -s nullglob +schema_files=(docs/schemas/*.schema.json) +if [[ ${#schema_files[@]} -eq 0 ]]; then + echo "lint: no JSON Schema files under docs/schemas/" >&2 + exit 1 +fi +"$CHECK_JSONSCHEMA_BIN" --check-metaschema "${schema_files[@]}" +"$CHECK_JSONSCHEMA_BIN" --schemafile \ + docs/schemas/weblate-add-or-update.request.schema.json \ + tests/helpers/fixtures/weblate-add-or-update-request-valid.json diff --git a/scripts/test.sh b/scripts/test.sh index acdcd68..daff295 100755 --- a/scripts/test.sh +++ b/scripts/test.sh @@ -35,4 +35,9 @@ ensure_bats() { } ensure_bats + +# shellcheck source=scripts/ensure_check_jsonschema.sh +source "$ROOT/scripts/ensure_check_jsonschema.sh" +ensure_check_jsonschema + "$BATS_BIN" tests/ diff --git a/tests/helpers/fixtures/weblate-add-or-update-request-valid.json b/tests/helpers/fixtures/weblate-add-or-update-request-valid.json new file mode 100644 index 0000000..390c566 --- /dev/null +++ b/tests/helpers/fixtures/weblate-add-or-update-request-valid.json @@ -0,0 +1,8 @@ +{ + "organization": "testorg", + "version": "develop", + "extensions": [".adoc"], + "add_or_update": { + "en": ["algorithm", "system"] + } +} diff --git a/tests/helpers/schema.bash b/tests/helpers/schema.bash new file mode 100644 index 0000000..d0d25a8 --- /dev/null +++ b/tests/helpers/schema.bash @@ -0,0 +1,58 @@ +# shellcheck shell=bash +# JSON Schema helpers for bats (check-jsonschema + request/response extractors). +# shellcheck disable=SC1091 + +SCHEMAS_DIR="${SCHEMAS_DIR:-$REPO_ROOT/docs/schemas}" + +# shellcheck source=../../scripts/ensure_check_jsonschema.sh +source "$REPO_ROOT/scripts/ensure_check_jsonschema.sh" + +# Validate a JSON string against a schema file. Writes a temp instance file. +validate_json_against_schema() { + local json="$1" schema_path="$2" tmp + ensure_check_jsonschema || return 1 + tmp="$(mktemp)" + printf '%s\n' "$json" >"$tmp" + "$CHECK_JSONSCHEMA_BIN" --schemafile "$schema_path" "$tmp" + local rc=$? + rm -f "$tmp" + return "$rc" +} + +# Extract POST body from the Weblate mock request log (BODY_START … BODY_END). +extract_weblate_request_body_from_log() { + local log_file="$1" + sed -n '/^BODY_START$/,/^BODY_END$/p' "$log_file" | sed '1d;$d' +} + +# Extract the last JSON object from a stderr/log file (pretty-printed or one-line). +extract_json_object_from_log() { + local log_file="$1" + python3 -c ' +import json, sys +text = open(sys.argv[1], encoding="utf-8", errors="replace").read() +# Prefer scanning for brace-balanced objects from the end. +candidates = [] +depth = 0 +start = None +for i, ch in enumerate(text): + if ch == "{": + if depth == 0: + start = i + depth += 1 + elif ch == "}" and depth: + depth -= 1 + if depth == 0 and start is not None: + candidates.append(text[start : i + 1]) + start = None +for blob in reversed(candidates): + try: + obj = json.loads(blob) + except json.JSONDecodeError: + continue + if isinstance(obj, dict): + print(json.dumps(obj)) + sys.exit(0) +sys.exit(1) +' "$log_file" +} diff --git a/tests/helpers/test_helper.bash b/tests/helpers/test_helper.bash index e876d28..7ba7f98 100644 --- a/tests/helpers/test_helper.bash +++ b/tests/helpers/test_helper.bash @@ -10,5 +10,7 @@ source "$_HELPER_DIR/common.bash" source "$_HELPER_DIR/git_fixtures.bash" # shellcheck source=tests/helpers/http_mock.bash source "$_HELPER_DIR/http_mock.bash" +# shellcheck source=tests/helpers/schema.bash +source "$_HELPER_DIR/schema.bash" unset _HELPER_DIR diff --git a/tests/test_trigger_weblate.bats b/tests/test_trigger_weblate.bats index 2a13652..78325cf 100644 --- a/tests/test_trigger_weblate.bats +++ b/tests/test_trigger_weblate.bats @@ -33,13 +33,11 @@ teardown() { [ -f "$MOCK_WEBLATE_REQUEST_LOG" ] grep -q "HEADER:Authorization: Token $WEBLATE_TOKEN" "$MOCK_WEBLATE_REQUEST_LOG" grep -q "HEADER:Content-Type: application/json" "$MOCK_WEBLATE_REQUEST_LOG" - body_json=$(sed -n '/^BODY_START$/,/^BODY_END$/p' "$MOCK_WEBLATE_REQUEST_LOG" | sed '1d;$d') - [ "$(echo "$body_json" | jq -r '.organization')" = "$MODULE_ORG" ] - [ "$(echo "$body_json" | jq -r '.version')" = "$libs_ref" ] - echo "$body_json" | jq -e --argjson expected_exts "$exts_json" \ - '.extensions | type == "array" and . == $expected_exts' >/dev/null - echo "$body_json" | jq -e \ - '.add_or_update | type == "object" and has("en") and (.en | type == "array") and .en == ["algorithm","system"]' >/dev/null + body_json=$(extract_weblate_request_body_from_log "$MOCK_WEBLATE_REQUEST_LOG") + validate_json_against_schema "$body_json" \ + "$SCHEMAS_DIR/weblate-add-or-update.request.schema.json" + response_json=$(extract_json_object_from_log "$BATS_TMPDIR/weblate-202.stderr") + echo "$response_json" | jq -e '.task_id == "abc123"' >/dev/null grep -q "boost-endpoint/add-or-update" "$MOCK_WEBLATE_REQUEST_LOG" } @@ -56,6 +54,8 @@ teardown() { [ "$status" -eq 0 ] grep -q "Weblate returned HTTP 200" "$BATS_TMPDIR/weblate-200.stderr" + response_json=$(extract_json_object_from_log "$BATS_TMPDIR/weblate-200.stderr") + echo "$response_json" | jq -e '.status == "ok"' >/dev/null } @test "trigger_weblate: auth failure on HTTP 403" { From 343e4be009d816038635bf78767fb4ffa0c22329 Mon Sep 17 00:00:00 2001 From: whisper67265 Date: Tue, 14 Jul 2026 14:37:59 -0600 Subject: [PATCH 2/3] fix the coderabbitai review comments --- scripts/ensure_check_jsonschema.sh | 7 +++++++ tests/helpers/schema.bash | 31 ++++++++++++++---------------- 2 files changed, 21 insertions(+), 17 deletions(-) diff --git a/scripts/ensure_check_jsonschema.sh b/scripts/ensure_check_jsonschema.sh index 620c854..293da7d 100755 --- a/scripts/ensure_check_jsonschema.sh +++ b/scripts/ensure_check_jsonschema.sh @@ -12,6 +12,13 @@ ensure_check_jsonschema() { root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" fi version="$CHECK_JSONSCHEMA_VERSION" + # Reject path separators / .. so version cannot escape cache_dir via venv_dir. + case "$version" in + '' | */* | *\\* | *..*) + echo "ensure_check_jsonschema: invalid CHECK_JSONSCHEMA_VERSION: ${version}" >&2 + return 1 + ;; + esac cache_dir="$root/.cache/check-jsonschema" venv_dir="$cache_dir/${version}" bin="$venv_dir/bin/check-jsonschema" diff --git a/tests/helpers/schema.bash b/tests/helpers/schema.bash index d0d25a8..9f9d58e 100644 --- a/tests/helpers/schema.bash +++ b/tests/helpers/schema.bash @@ -31,28 +31,25 @@ extract_json_object_from_log() { python3 -c ' import json, sys text = open(sys.argv[1], encoding="utf-8", errors="replace").read() -# Prefer scanning for brace-balanced objects from the end. +decoder = json.JSONDecoder() candidates = [] -depth = 0 -start = None -for i, ch in enumerate(text): - if ch == "{": - if depth == 0: - start = i - depth += 1 - elif ch == "}" and depth: - depth -= 1 - if depth == 0 and start is not None: - candidates.append(text[start : i + 1]) - start = None -for blob in reversed(candidates): +i = 0 +n = len(text) +while i < n: + start = text.find("{", i) + if start < 0: + break try: - obj = json.loads(blob) + obj, end = decoder.raw_decode(text, start) except json.JSONDecodeError: + i = start + 1 continue if isinstance(obj, dict): - print(json.dumps(obj)) - sys.exit(0) + candidates.append(obj) + i = end +for obj in reversed(candidates): + print(json.dumps(obj)) + sys.exit(0) sys.exit(1) ' "$log_file" } From ca50f58a46d0f0eed748b925c967200e4b897417 Mon Sep 17 00:00:00 2001 From: whisper67265 Date: Wed, 15 Jul 2026 01:01:06 -0600 Subject: [PATCH 3/3] fix the first reviewer comments --- CHANGELOG.md | 14 ++++++++++++++ .../weblate-add-or-update.request.schema.json | 2 +- tests/test_trigger_weblate.bats | 2 ++ 3 files changed, 17 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 51fd5ce..297c4b8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,11 +11,25 @@ are a separate namespace — see [README](README.md#releases) and ## [Unreleased] +### Added + +- JSON Schema for the outbound Weblate add-or-update POST request: + [weblate-add-or-update.request.schema.json](docs/schemas/weblate-add-or-update.request.schema.json) + (`organization`, `version`, `extensions`, `add_or_update`). +- Documented Weblate success response bodies in + [endpoint-contract.md](docs/endpoint-contract.md): HTTP **202** (`task_id`), + HTTP **200** (`status: "ok"`). +- Lint/CI schema checks via pinned `check-jsonschema` (metaschema + fixture + validation); bats validates captured Weblate request bodies against the schema + and asserts success response fields. + ### Changed - Renamed and expanded operator quick reference to [GETTING-STARTED.md](docs/GETTING-STARTED.md): end-to-end walkthrough with per-step verification and create-tag coverage. +- [endpoint-contract.md](docs/endpoint-contract.md) Outbound Weblate section: + request schema is now the source of truth for payload fields. ## [1.0.0] - 2026-07-07 diff --git a/docs/schemas/weblate-add-or-update.request.schema.json b/docs/schemas/weblate-add-or-update.request.schema.json index 68ea4c5..7763ed7 100644 --- a/docs/schemas/weblate-add-or-update.request.schema.json +++ b/docs/schemas/weblate-add-or-update.request.schema.json @@ -1,6 +1,6 @@ { "$schema": "https://json-schema.org/draft/2020-12/schema", - "$id": "https://github.com/boostorg/boost-docs-translation/docs/schemas/weblate-add-or-update.request.schema.json", + "$id": "https://github.com/cppalliance/boost-docs-translation/docs/schemas/weblate-add-or-update.request.schema.json", "title": "Weblate add-or-update request", "description": "Outbound POST body built by trigger_weblate for WEBLATE_ENDPOINT_PATH.", "type": "object", diff --git a/tests/test_trigger_weblate.bats b/tests/test_trigger_weblate.bats index 78325cf..ad4ed78 100644 --- a/tests/test_trigger_weblate.bats +++ b/tests/test_trigger_weblate.bats @@ -36,6 +36,8 @@ teardown() { body_json=$(extract_weblate_request_body_from_log "$MOCK_WEBLATE_REQUEST_LOG") validate_json_against_schema "$body_json" \ "$SCHEMAS_DIR/weblate-add-or-update.request.schema.json" + [ "$(echo "$body_json" | jq -r '.organization')" = "$MODULE_ORG" ] + [ "$(echo "$body_json" | jq -r '.version')" = "$libs_ref" ] response_json=$(extract_json_object_from_log "$BATS_TMPDIR/weblate-202.stderr") echo "$response_json" | jq -e '.task_id == "abc123"' >/dev/null grep -q "boost-endpoint/add-or-update" "$MOCK_WEBLATE_REQUEST_LOG"