-
Notifications
You must be signed in to change notification settings - Fork 53
test: add Docker-based build and smoke test environment #92
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
fzipi
wants to merge
6
commits into
owasp-modsecurity:master
Choose a base branch
from
fzipi:test/docker-build-environment
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
84321bb
test: add Docker-based build and smoke test environment
fzipi 16d47bb
fix: use a standard HTTP status for the request-body test rule
fzipi 7d408a1
ci: add GitHub Actions workflow to build and smoke test the Dockerfile
fzipi 2dfc54a
fix: correct and slim the Docker test environment
fzipi 97436ec
docs: describe what the test environment actually does
fzipi 5c6d60c
docs: mark the sample output fence as text
fzipi File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| name: Docker build | ||
|
|
||
| on: | ||
| push: | ||
| branches: [master] | ||
| paths: | ||
| - Dockerfile | ||
| - docker-compose.yml | ||
| - test-connector.sh | ||
| - src/** | ||
| - .github/workflows/docker-build.yml | ||
| pull_request: | ||
| paths: | ||
| - Dockerfile | ||
| - docker-compose.yml | ||
| - test-connector.sh | ||
| - src/** | ||
| - .github/workflows/docker-build.yml | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| concurrency: | ||
| group: ${{ github.workflow }}-${{ github.ref }} | ||
| cancel-in-progress: true | ||
|
|
||
| jobs: | ||
| build-and-smoke-test: | ||
| name: Build and smoke test | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | ||
| with: | ||
| persist-credentials: false | ||
|
|
||
| # Compose is the only supported way to run this: the smoke tests read | ||
| # the ModSecurity debug log through the bind mount it sets up. | ||
| - name: Build and start container | ||
| run: docker compose up -d --build | ||
|
|
||
| - name: Run smoke tests | ||
| run: ./test-connector.sh | ||
|
|
||
| - name: Show container logs | ||
| if: always() | ||
| run: | | ||
| docker compose logs | ||
| cat logs/modsec_debug.log 2>/dev/null | tail -50 || true | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -5,3 +5,6 @@ | |
| .libs/* | ||
| src/.libs/* | ||
| t/htdocs/index.html | ||
|
|
||
| # Test harness output (docker-compose bind mount) | ||
| logs/ | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,104 @@ | ||
| # Docker Testing Guide for the ModSecurity Apache Connector | ||
|
|
||
| A smoke-test harness for the ModSecurity v3 Apache connector. It builds | ||
| libmodsecurity and the connector from source, loads a two-rule test set, and | ||
| lets connector behaviour be observed directly. It is not a production | ||
| configuration. | ||
|
|
||
| ## Quick Start | ||
|
|
||
| ```bash | ||
| docker compose up -d --build | ||
| ./test-connector.sh | ||
| ``` | ||
|
|
||
| Compose is the supported way to run this: the tests read the ModSecurity debug | ||
| log through the bind mount it sets up, so a bare `docker run` will not work. | ||
|
|
||
| ## Manual Testing | ||
|
|
||
| ```bash | ||
| # Normal request (200) | ||
| curl http://localhost:8080/ | ||
|
|
||
| # Query string rule, id 1001 (403) | ||
| curl -v "http://localhost:8080/?test=evil" | ||
|
|
||
| # Request body rule, id 1002 (403) | ||
| curl -X POST http://localhost:8080/ -d "data=malicious" | ||
|
|
||
| # Large body, no match (200) | ||
| curl -X POST http://localhost:8080/ -d "$(head -c 100000 /dev/zero | tr '\0' 'A')" | ||
|
|
||
| # Large body spanning multiple buckets, with a match at the end (403) | ||
| curl -X POST http://localhost:8080/ -d "$(head -c 100000 /dev/zero | tr '\0' 'A')malicious" | ||
| ``` | ||
|
|
||
| Bodies stay under the 128KB `SecRequestBodyNoFilesLimit` from the recommended | ||
| configuration; larger ones are rejected with 413 before the rules run. A 10KB | ||
| body arrives in a single bucket, so it does not exercise multi-bucket handling. | ||
|
|
||
| ## Observing rule evaluation | ||
|
|
||
| Denied requests are **not** written to the Apache error log — that is upstream | ||
| issue #67, not a misconfiguration here. Two other signals are available: | ||
|
|
||
| - `logs/modsec_audit.log` — one entry per transaction, showing which rule | ||
| matched. It does not tell you how many times a rule was evaluated. | ||
| - `logs/modsec_debug.log` — one line per phase invocation. This is the only | ||
| signal that shows how often a phase actually ran. | ||
|
|
||
| `test-connector.sh` uses the debug log to report how many times the | ||
| request-body phase ran for a single large POST: | ||
|
|
||
| ```text | ||
| request-body phase invocations for that request: 26 (KNOWN BUG: expected 1, ...) | ||
| ``` | ||
|
|
||
| A correct connector assembles the whole body and evaluates it once. The | ||
| current source re-runs the phase for every bucket, which is the defect behind | ||
| the request-body work; the count is reported rather than asserted so this | ||
| branch stays green. Once the fix lands it becomes a hard assertion. | ||
|
|
||
| ## Debugging | ||
|
|
||
| ```bash | ||
| # Live logs | ||
| docker compose logs -f | ||
|
|
||
| # Shell into the container | ||
| docker compose exec modsec3-apache bash | ||
|
|
||
| # Confirm the module loaded | ||
| apache2ctl -M | grep security3 | ||
|
|
||
| # Module dependencies | ||
| ldd /usr/lib/apache2/modules/mod_security3.so | ||
|
|
||
| # Active configuration | ||
| cat /etc/modsecurity/modsecurity.conf | ||
| cat /etc/modsecurity/test-rules.conf | ||
| ``` | ||
|
|
||
| ## Expected Results | ||
|
|
||
| All 6 checks in `test-connector.sh` pass: | ||
|
|
||
| 1. Normal request — 200 | ||
| 2. Query string block — 403 | ||
| 3. Request body block — 403 | ||
| 4. Normal POST — 200 | ||
| 5. Large POST — 200 | ||
| 6. Large POST with a match — 403 | ||
|
|
||
| Test 6 additionally reports the request-body phase count described above. | ||
|
|
||
| ## What's Included | ||
|
|
||
| - **libmodsecurity** v3.0.16, built from the pinned release tag | ||
| - **Apache HTTP Server** 2.4.68, from Debian bookworm | ||
| - **ModSecurity Apache Connector**, built from this working tree | ||
|
|
||
| The recommended ModSecurity configuration is copied out of the same | ||
| libmodsecurity source tree that was built, so it cannot drift from the | ||
| version in the image. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,145 @@ | ||
| # Dockerfile for testing the ModSecurity v3 Apache connector. | ||
| # Builds libmodsecurity3 and the connector against Debian's Apache. | ||
|
|
||
| FROM debian:bookworm-slim AS builder | ||
|
|
||
| ARG MODSECURITY_VERSION=v3.0.16 | ||
|
|
||
| RUN apt-get update && \ | ||
| apt-get install -y --no-install-recommends \ | ||
| # Build essentials | ||
| build-essential \ | ||
| ca-certificates \ | ||
| automake \ | ||
| autoconf \ | ||
| libtool \ | ||
| pkg-config \ | ||
| git \ | ||
| # Apache module build support (apxs2, plus the httpd binary configure probes for) | ||
| apache2 \ | ||
| apache2-dev \ | ||
| # libmodsecurity dependencies | ||
| libcurl4-openssl-dev \ | ||
| libyajl-dev \ | ||
| libgeoip-dev \ | ||
| liblmdb-dev \ | ||
| libxml2-dev \ | ||
| libpcre2-dev \ | ||
| libmaxminddb-dev \ | ||
| libfuzzy-dev && \ | ||
| rm -rf /var/lib/apt/lists/* | ||
|
|
||
| # Build libmodsecurity v3 from a pinned release tag | ||
| WORKDIR /build | ||
|
|
||
| RUN git clone --depth 1 --branch ${MODSECURITY_VERSION} \ | ||
| https://github.com/owasp-modsecurity/ModSecurity.git libmodsecurity && \ | ||
| cd libmodsecurity && \ | ||
| git submodule update --init --recursive && \ | ||
| ./build.sh && \ | ||
| ./configure \ | ||
| --prefix=/usr/local/modsecurity \ | ||
| --with-pcre2 \ | ||
| --with-yajl \ | ||
| --with-geoip \ | ||
| --with-lmdb && \ | ||
| make -j$(nproc) && \ | ||
| make install && \ | ||
| ldconfig | ||
|
|
||
| # Build the connector; configure finds Debian's apxs2 on its own | ||
| WORKDIR /build/connector | ||
|
|
||
| COPY . . | ||
|
|
||
| RUN ./autogen.sh && \ | ||
| ./configure --with-libmodsecurity=/usr/local/modsecurity && \ | ||
| make -j$(nproc) && \ | ||
| make install | ||
|
|
||
| FROM debian:bookworm-slim | ||
|
|
||
| LABEL description="Apache with the ModSecurity v3 connector, for smoke testing" | ||
|
|
||
| RUN apt-get update && \ | ||
| apt-get install -y --no-install-recommends \ | ||
| apache2 \ | ||
| wget \ | ||
| libcurl4 \ | ||
| libyajl2 \ | ||
| libgeoip1 \ | ||
| liblmdb0 \ | ||
| libxml2 \ | ||
| libpcre2-8-0 \ | ||
| libmaxminddb0 \ | ||
| libfuzzy2 && \ | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
| rm -rf /var/lib/apt/lists/* | ||
|
|
||
| COPY --from=builder /usr/local/modsecurity /usr/local/modsecurity | ||
| COPY --from=builder /usr/lib/apache2/modules/mod_security3.so /usr/lib/apache2/modules/ | ||
|
|
||
| RUN echo "/usr/local/modsecurity/lib" > /etc/ld.so.conf.d/modsecurity.conf && \ | ||
| ldconfig | ||
|
|
||
| # Take the recommended config from the same source tree we built, so it can | ||
| # never drift from the pinned libmodsecurity version. | ||
| COPY --from=builder /build/libmodsecurity/modsecurity.conf-recommended /etc/modsecurity/modsecurity.conf | ||
| COPY --from=builder /build/libmodsecurity/unicode.mapping /etc/modsecurity/unicode.mapping | ||
|
|
||
| RUN sed -i 's/SecRuleEngine DetectionOnly/SecRuleEngine On/' /etc/modsecurity/modsecurity.conf | ||
|
|
||
| RUN cat > /etc/modsecurity/test-rules.conf << 'EOF' | ||
| # Fires on the query string, to check phase 1 / ARGS handling | ||
| SecRule ARGS:test "@contains evil" \ | ||
| "id:1001,phase:2,deny,status:403,msg:'Test rule triggered'" | ||
|
|
||
| # Fires on the request body, to check that a multi-bucket body is assembled | ||
| # and evaluated exactly once | ||
| SecRule REQUEST_BODY "@rx malicious" \ | ||
| "id:1002,phase:2,deny,status:403,msg:'Request body rule triggered'" | ||
|
|
||
| # The connector does not write denied requests to the Apache error log | ||
| # (upstream issue #67), and the audit log records one entry per transaction | ||
| # rather than one per rule evaluation. The debug log is the only signal that | ||
| # shows how many times a phase actually ran, which is what the request-body | ||
| # tests need to check. | ||
| SecDebugLog /var/log/apache2/modsec_debug.log | ||
| SecDebugLogLevel 4 | ||
| SecAuditLog /var/log/apache2/modsec_audit.log | ||
| EOF | ||
|
|
||
| RUN cat > /etc/apache2/mods-available/security3.load << 'EOF' | ||
| LoadModule security3_module /usr/lib/apache2/modules/mod_security3.so | ||
|
|
||
| <IfModule security3_module> | ||
| modsecurity on | ||
| modsecurity_rules_file /etc/modsecurity/modsecurity.conf | ||
| modsecurity_rules_file /etc/modsecurity/test-rules.conf | ||
| </IfModule> | ||
| EOF | ||
|
|
||
| RUN a2enmod security3 && \ | ||
| sed -i 's/^Listen 80$/Listen 8080/' /etc/apache2/ports.conf && \ | ||
| sed -i 's/<VirtualHost \*:80>/<VirtualHost *:8080>/' \ | ||
| /etc/apache2/sites-available/000-default.conf && \ | ||
| echo "ServerName localhost" >> /etc/apache2/apache2.conf | ||
|
|
||
| RUN cat > /usr/local/bin/start.sh << 'EOF' | ||
| #!/bin/bash | ||
| set -e | ||
|
|
||
| if ! apache2ctl -M 2>&1 | grep -q security3_module; then | ||
| echo "ERROR: ModSecurity module not loaded!" | ||
| ldd /usr/lib/apache2/modules/mod_security3.so | ||
| exit 1 | ||
| fi | ||
|
|
||
| echo "ModSecurity module loaded, starting Apache on :8080" | ||
| exec apache2ctl -DFOREGROUND | ||
| EOF | ||
|
|
||
| RUN chmod +x /usr/local/bin/start.sh | ||
|
|
||
| EXPOSE 8080 | ||
|
|
||
| CMD ["/usr/local/bin/start.sh"] | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| services: | ||
| modsec3-apache: | ||
| build: . | ||
| container_name: modsec3-apache-test | ||
| ports: | ||
| - "8080:8080" | ||
| volumes: | ||
| - ./logs:/var/log/apache2:rw | ||
| healthcheck: | ||
| test: ["CMD", "wget", "-q", "-O", "/dev/null", "http://localhost:8080/"] | ||
| interval: 10s | ||
| timeout: 5s | ||
| retries: 3 | ||
| start_period: 5s | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.