feat: add Dockerfile-supabase and rework Dockerfile-multigres as layered images#2160
Open
mkindahl wants to merge 2 commits into
Open
feat: add Dockerfile-supabase and rework Dockerfile-multigres as layered images#2160mkindahl wants to merge 2 commits into
mkindahl wants to merge 2 commits into
Conversation
mkindahl
commented
May 20, 2026
mkindahl
force-pushed
the
feat/dockerfile-refactor
branch
3 times, most recently
from
8a8de35 to
645a765
Compare
PostgreSQL Extension Dependency Analysis: PR #2160
SummaryNo extensions had dependencies with MAJOR version updates. Full Analysis ResultsPostgreSQL 15 Extension DependenciesPostgreSQL 17 Extension DependenciesOrioleDB 17 Extension Dependencies |
PostgreSQL Package Dependency Analysis: PR #2160
SummaryNo packages had MAJOR version updates. Full Analysis ResultsPostgreSQL 15 Dependency ChangesExtracting PostgreSQL 15 dependencies...
Runtime Closure Size
Raw Dependency ClosurePostgreSQL 17 Dependency ChangesExtracting PostgreSQL 17 dependencies...
Runtime Closure Size
Raw Dependency Closure |
mkindahl
commented
May 27, 2026
Author
There was a problem hiding this comment.
Previous code sent Postgres output to stdout and this was mixed with pgctld output.
This patch changes that and now only pgctld output is written to stdout. This means that we need to capture the log output by reading the file or mounting a volume where we can read the data.
I am unsure how logs are collected, so we might either send all output to stdout again (which will mix pgctld and postgres output), or send one of them to stdout (depending on how the log collectors operate) and require the other to be read explicitly.
Contributor
|
This looks pretty good to me, I've got some follow ups that I think make sense but better to get this PR in and do so later (mostly minor nits, but I suspect we can drop all the Dockerfiles-* and have just the one multi-stage Dockerfile and use |
Contributor
|
@mkindahl I tried restarting the failed CI builds and noticed that the failures are real issues in Dockerfile-supabase (for example syntax should be 1.4 not 1.3 because the heredoc syntax was in 1.3-labs and graduated in 1.4, maybe some other issues too). I tried fixing it and then I ended up trying out putting everything all into one Dockerfile and fell down the rabbit hole here. It's not too bad IMO. I'm going to drop it into here so you can take a quick look if nothing stands out I can throw it into this PR. This one Dockerfile can be used to build PG15, PG17, PG17-orioledb, PG17-multigres, PG17-orioledb-multigres and there's no issue with running 2 commands for multigres. Unfortunately I think that means some of your changes need to backed off since we'd want to make use of Dockerfile# syntax=docker/dockerfile:1.4
# 1.4: minimum version for heredoc support in RUN (used for nix config below)
# Supabase PostgreSQL image with Nix extensions — parameterised by major version.
#
# Build (defaults to PostgreSQL 17):
# docker build --build-arg=PG_VERSION=17 --target=postgres -t supabase-postgres:17 .
#
# Build for a different PostgreSQL version:
# docker build --build-arg=PG_VERSION=15 --target=postgres -t supabase-postgres:15 .
#
# Build for Multigres:
# docker build --build-arg=PG_VERSION=17 --target=multigres -t supabase-multigres:17 .
# Put all ARGs up top for better discoverability
# Use non-versioned `ARG THE_ARG` in the layer that uses it to get the ref to this one
ARG ALPINE_VERSION=3.23
ARG GOSU_VERSION=1.19
ARG PGCTLD_REV=1e3bad798972600778ee27eb08ab7d34cc8be8e9 # Pinned to the commit that introduced --pg-initdb-sql-dirs (MUL-484)
ARG PG_VERSION=17
ARG VARIANT
###################################
# Postgres + Exts #
###################################
FROM alpine:${ALPINE_VERSION} AS pg
ARG PG_VERSION
ARG VARIANT
RUN <<EOF
case $PG_VERSION:$VARIANT in
15: | 17:) ;;
17:orioledb) VARIANT=$VARIANT-;;
15:orioledb) echo "OrioleDB not supported on PG17" >&2 && exit 1;;
17:*) echo "Unknown VARIANT ($VARIANT) only orioledb supported" >&2 && exit 1;;
*) echo "Unknown PG_VERSION ($PG_VERSION) only 15,17 supported" >&2 && exit 1;;
esac
EOF
# Install dependencies for nix installer (coreutils for GNU cp, sudo for installer)
RUN apk add --no-cache \
bash \
coreutils \
curl \
shadow \
sudo \
xz
# Create nix config
RUN cat >/tmp/extra-nix.conf <<EOF
extra-experimental-features = nix-command flakes
extra-substituters = https://nix-postgres-artifacts.s3.amazonaws.com
extra-trusted-public-keys = nix-postgres-artifacts:dGZlQOvKcNEjvT7QEAJbcV6b6uk7VF/hWMjhYleiaLI=
EOF
RUN curl -L https://releases.nixos.org/nix/nix-2.34.6/install | sh -s -- --daemon --no-channel-add --yes --nix-extra-conf-file /tmp/extra-nix.conf
ENV PATH="$PATH:/nix/var/nix/profiles/default/bin"
RUN nix --version
WORKDIR /nixpg
COPY . .
# Build PostgreSQL with extensions
RUN attr=psql_$VARIANT${PG_VERSION}_slim/bin && \
echo Building $attr && \
nix profile add path:.#$attr
# Build groonga and copy plugins
RUN nix profile add path:.#supabase-groonga && \
mkdir -p /tmp/groonga-plugins && \
cp -r /nix/var/nix/profiles/default/lib/groonga/plugins /tmp/groonga-plugins/
RUN nix store gc && nix store optimise
################################################
# Gosu #
################################################
FROM golang:1.26-alpine${ALPINE_VERSION} AS gosu
ARG GOSU_VERSION
RUN apk add --no-cache curl git
# Build gosu from source
RUN git clone --depth 1 --branch "${GOSU_VERSION}" https://github.com/tianon/gosu.git /gosu && \
cd /gosu && \
CGO_ENABLED=0 go build -ldflags="-s -w" -o /usr/local/bin/gosu . && \
chmod +x /usr/local/bin/gosu && \
/usr/local/bin/gosu --version
#########################################
# Production Image #
#########################################
FROM alpine:${ALPINE_VERSION} AS postgres
ARG ALPINE_VERSION
ARG PG_VERSION
# Install minimal runtime dependencies
RUN apk add --no-cache \
bash \
curl \
shadow \
su-exec \
tzdata \
musl-locales \
musl-locales-lang \
&& rm -rf /var/cache/apk/*
# Create postgres user/group
RUN addgroup -S postgres && \
adduser -S -G postgres -h /var/lib/postgresql -s /bin/bash postgres && \
addgroup -S wal-g && \
adduser -S -G wal-g -s /bin/bash wal-g && \
adduser postgres wal-g
# Copy Nix store and profiles from builder (profile already created by nix profile install)
COPY --from=pg /nix /nix
# Copy groonga plugins
COPY --from=pg /tmp/groonga-plugins/plugins /usr/lib/groonga/plugins
# Copy gosu
COPY --from=gosu /usr/local/bin/gosu /usr/local/bin/gosu
# Setup PostgreSQL directories
RUN mkdir -p /usr/lib/postgresql/bin \
/usr/lib/postgresql/share/postgresql \
/usr/share/postgresql \
/var/lib/postgresql/data \
/var/run/postgresql \
&& chown -R postgres:postgres /usr/lib/postgresql \
&& chown -R postgres:postgres /var/lib/postgresql \
&& chown -R postgres:postgres /usr/share/postgresql \
&& chown -R postgres:postgres /var/run/postgresql
# Create symbolic links for binaries
RUN for f in /nix/var/nix/profiles/default/bin/*; do \
ln -sf "$f" /usr/lib/postgresql/bin/ 2>/dev/null || true; \
ln -sf "$f" /usr/bin/ 2>/dev/null || true; \
done
# Create symbolic links for PostgreSQL shares
RUN ln -sf /nix/var/nix/profiles/default/share/postgresql/* /usr/lib/postgresql/share/postgresql/ 2>/dev/null || true && \
ln -sf /nix/var/nix/profiles/default/share/postgresql/* /usr/share/postgresql/ 2>/dev/null || true && \
ln -sf /usr/lib/postgresql/share/postgresql/timezonesets /usr/share/postgresql/timezonesets 2>/dev/null || true
# Set permissions
RUN chown -R postgres:postgres /usr/lib/postgresql && \
chown -R postgres:postgres /usr/share/postgresql
# Setup configs
COPY --chown=postgres:postgres ansible/files/postgresql_config/postgresql.conf.j2 /etc/postgresql/postgresql.conf
COPY --chown=postgres:postgres ansible/files/postgresql_config/pg_hba.conf.j2 /etc/postgresql/pg_hba.conf
COPY --chown=postgres:postgres ansible/files/postgresql_config/pg_ident.conf.j2 /etc/postgresql/pg_ident.conf
COPY --chown=postgres:postgres ansible/files/postgresql_config/conf.d /etc/postgresql/postgresql.conf.d
COPY --chown=postgres:postgres ansible/files/postgresql_config/postgresql-stdout-log.conf /etc/postgresql/logging.conf
COPY --chown=postgres:postgres ansible/files/postgresql_config/supautils.conf.j2 /etc/postgresql-custom/supautils.conf
COPY --chown=postgres:postgres ansible/files/postgresql_extension_custom_scripts /etc/postgresql-custom/extension-custom-scripts
COPY --chown=postgres:postgres ansible/files/pgsodium_getkey_urandom.sh.j2 /usr/lib/postgresql/bin/pgsodium_getkey.sh
COPY --chown=postgres:postgres ansible/files/postgresql_config/custom_walg.conf /etc/postgresql-custom/wal-g.conf
COPY --chown=postgres:postgres ansible/files/postgresql_config/custom_read_replica.conf /etc/postgresql-custom/read-replica.conf
COPY --chown=postgres:postgres ansible/files/walg_helper_scripts/wal_fetch.sh /home/postgres/wal_fetch.sh
COPY ansible/files/walg_helper_scripts/wal_change_ownership.sh /root/wal_change_ownership.sh
# Configure PostgreSQL settings
RUN sed -i \
-e "s|#unix_socket_directories = '/tmp'|unix_socket_directories = '/var/run/postgresql'|g" \
-e "s|#session_preload_libraries = ''|session_preload_libraries = 'supautils'|g" \
-e "s|#include = '/etc/postgresql-custom/supautils.conf'|include = '/etc/postgresql-custom/supautils.conf'|g" \
-e "s|#include = '/etc/postgresql-custom/wal-g.conf'|include = '/etc/postgresql-custom/wal-g.conf'|g" /etc/postgresql/postgresql.conf && \
echo "pgsodium.getkey_script= '/usr/lib/postgresql/bin/pgsodium_getkey.sh'" >> /etc/postgresql/postgresql.conf && \
echo "vault.getkey_script= '/usr/lib/postgresql/bin/pgsodium_getkey.sh'" >> /etc/postgresql/postgresql.conf && \
chown -R postgres:postgres /etc/postgresql-custom && \
ln -s /etc/postgresql/postgresql.conf.d /etc/postgresql-custom/conf.d
# pg17+ does not ship timescaledb or plv8; db_user_namespace was removed in pg17.
# Applied conditionally so the same Dockerfile works for pg15 (where these are valid).
RUN if [[ "${PG_VERSION#*-}" -ge 17 ]]; then \
sed -i 's/ timescaledb,//g; s/ plv8,//g' /etc/postgresql/postgresql.conf && \
sed -i 's/db_user_namespace = off/#db_user_namespace = off/g' /etc/postgresql/postgresql.conf && \
sed -i 's/ timescaledb,//g; s/ plv8,//g' /etc/postgresql-custom/supautils.conf; \
fi
# OrioleDB stuff if building it
RUN [[ "$VARIANT-" != orioledb ]] && exit 0; \
sed -i 's/\(shared_preload_libraries.*\)'\''\(.*\)$/\1, orioledb'\''\2/' "/etc/postgresql/postgresql.conf" && \
echo "default_table_access_method = 'orioledb'" >> "/etc/postgresql/postgresql.conf" && \
sed -i 's/ postgis,//g; s/ pgrouting,//g' "/etc/postgresql-custom/supautils.conf" && \
echo "CREATE EXTENSION orioledb;" > /docker-entrypoint-initdb.d/init-scripts/00-pre-init.sql && \
chown postgres:postgres /docker-entrypoint-initdb.d/init-scripts/00-pre-init.sql && \
true
# Include schema migrations
COPY migrations/db /docker-entrypoint-initdb.d/
COPY ansible/files/pgbouncer_config/pgbouncer_auth_schema.sql /docker-entrypoint-initdb.d/init-scripts/00-schema.sql
COPY ansible/files/stat_extension.sql /docker-entrypoint-initdb.d/migrations/00-extension.sql
ADD --chmod=0755 \
https://raw.githubusercontent.com/docker-library/postgres/6edb0a8c4def40c371514b34aef9037ec82d9110/${PG_VERSION}/alpine${ALPINE_VERSION}/docker-entrypoint.sh \
/usr/local/bin/docker-entrypoint.sh
# Setup pgsodium key script
RUN mkdir -p /usr/share/postgresql/extension/ && \
ln -s /usr/lib/postgresql/bin/pgsodium_getkey.sh /usr/share/postgresql/extension/pgsodium_getkey && \
chmod +x /usr/lib/postgresql/bin/pgsodium_getkey.sh
# Environment variables
ENV PATH="/nix/var/nix/profiles/default/bin:/usr/lib/postgresql/bin:${PATH}"
ENV PGDATA=/var/lib/postgresql/data
ENV POSTGRES_HOST=/var/run/postgresql
ENV POSTGRES_USER=supabase_admin
ENV POSTGRES_DB=postgres
ENV POSTGRES_INITDB_ARGS="--allow-group-access --locale-provider=icu --encoding=UTF-8 --icu-locale=en_US.UTF-8"
ENV LANG=en_US.UTF-8
ENV LANGUAGE=en_US:en
ENV LC_ALL=en_US.UTF-8
ENV GRN_PLUGINS_DIR=/usr/lib/groonga/plugins
# Point to minimal glibc locales included in slim Nix package for initdb locale support
ENV LOCALE_ARCHIVE=/nix/var/nix/profiles/default/lib/locale/locale-archive
# Marks the container unhealthy after 10 failed pg_isready probes, which blocks
# dependent services in Docker Compose (depends_on: condition: service_healthy).
# Kubernetes ignores Docker HEALTHCHECK entirely — use readinessProbe in the Pod spec.
HEALTHCHECK --interval=2s --timeout=2s --retries=10 CMD pg_isready -U postgres -h localhost
# SIGINT triggers PostgreSQL smart shutdown: waits for active sessions to finish
# before stopping. This avoids interrupting in-flight transactions but can delay
# pod termination if long-running queries are active.
# Consider SIGTERM (fast shutdown) to disconnect clients immediately, which
# respects Kubernetes terminationGracePeriodSeconds more predictably.
STOPSIGNAL SIGINT
EXPOSE 5432
# No USER directive: the entrypoint starts as root to fix volume ownership and
# set up permissions, then drops to the postgres user via gosu before exec'ing
# the postgres process. This follows the standard official PostgreSQL image pattern.
ENTRYPOINT ["docker-entrypoint.sh"]
CMD ["postgres", "-D", "/etc/postgresql"]
##################################################
## pgctld ##
##################################################
FROM golang:1.25-alpine${ALPINE_VERSION} AS pgctld
ARG PGCTLD_REV
RUN apk add --no-cache git
RUN git clone https://github.com/multigres/multigres.git /multigres && \
cd /multigres && \
git checkout ${PGCTLD_REV} && \
# Copy pico CSS assets before build (mirrors pgctld.nix preBuild step)
cp external/pico/pico.* go/common/web/templates/css/ 2>/dev/null || true && \
CGO_ENABLED=0 go build -ldflags="-s -w" -o /usr/local/bin/pgctld ./go/cmd/pgctld
##########################
## MULTIGRES ##
##########################
FROM postgres AS multigres
RUN if [ "$(postgres --version | awk -F. '{print $1}')" != 'postgres (PostgreSQL) 17' ]; then \
echo "Multigres only supports PG17, got -->$(postgres --version)<--" >&2; \
exit 1; \
fi
# Install pgbackrest (available in Alpine community repo)
RUN apk add --no-cache pgbackrest
# Copy pgctld binary; keep it separate so the wrapper script can reference it cleanly
COPY --from=pgctld /usr/local/bin/pgctld /usr/local/bin/pgctld-bin
# pgctld config template — /etc/pgctld is a mount point in k8s so use a custom dir
COPY docker/pgctld/postgresql.conf.tmpl /etc/pgctld-custom/postgresql.conf.tmpl
# Wrapper: injects --postgres-config-template on every pgctld call so unmodified
# k8s manifests and local provisioner commands work without extra flags
COPY --chmod=0755 docker/pgctld/pgctld /usr/local/bin/pgctld
# /etc/postgresql/postgresql.conf is not modified here: pgctld renders its own
# config from postgresql.conf.tmpl and passes it directly to PostgreSQL, so the
# supabase base config (including wal-g and data_directory settings) is never loaded.
# wal-g is not used in Multigres (pgbackrest handles backups); remove inherited files.
RUN rm -f \
/etc/postgresql-custom/wal-g.conf \
/home/postgres/wal_fetch.sh \
/root/wal_change_ownership.sh
# No HEALTHCHECK defined here: inherits the probe from the supabase base image.
# Kubernetes ignores Docker HEALTHCHECK entirely — use readinessProbe in the Pod spec.
# STOPSIGNAL inherited from supabase base image (SIGINT — smart shutdown).
USER postgres
# pgctld is the cluster lifecycle manager for Multigres: it handles initdb,
# config templating, replication setup, and coordinated restarts. Running it
# as PID 1 ensures it receives stop signals directly and can shut down
# PostgreSQL cleanly before the container exits.
ENTRYPOINT ["/usr/local/bin/pgctld"]
##########################################
## USELESS ##
##########################################
FROM scratch
RUN echo "You have tried to build this image without specifying --target" >&2 && \
echo "Either postgres or multigres is required" >&2 && \
echo "P.S.: You can specify PG version with --build-arg=PG_VERSION=15|17" >&2 && \
echo "P.P.S: You can build orioledb varian with --build-arg=VARIANT=orioledb" >&2 && \
exit 1
diff Dockerfile-supabase Dockerfile--- Dockerfile-supabase 2026-05-27 19:26:43.324333422 -0400
+++ Dockerfile 2026-05-27 19:31:23.248698133 -0400
@@ -1,21 +1,43 @@
-# syntax=docker/dockerfile:1.3
-# 1.3: minimum version for heredoc support in RUN (used for nix config below)
+# syntax=docker/dockerfile:1.4
+# 1.4: minimum version for heredoc support in RUN (used for nix config below)
# Supabase PostgreSQL image with Nix extensions — parameterised by major version.
#
# Build (defaults to PostgreSQL 17):
-# docker build -f Dockerfile-supabase -t supabase-postgres:17 .
+# docker build --target=postgres -t supabase-postgres:17 .
#
# Build for a different PostgreSQL version:
-# docker build -f Dockerfile-supabase --build-arg PG_VERSION=15 -t supabase-postgres:15 .
+# docker build --build-arg=PG_VERSION=15 --target=postgres -t supabase-postgres:15 .
+#
+# Build for OrioleDB:
+# docker build --build-arg=PG_VERSION=17 --build-arg=VARIANT=orioledb --target=postgres -t supabase-orioledb:17 .
+#
+# Build for Multigres:
+# docker build --build-arg=PG_VERSION=17 --target=multigres -t supabase-multigres:17 .
+# Put all ARGs up top for better discoverability
+# Use non-versioned `ARG THE_ARG` in the layer that uses it to get the ref to this one
+ARG ALPINE_VERSION=3.23
+ARG GOSU_VERSION=1.19
+ARG PGCTLD_REV=1e3bad798972600778ee27eb08ab7d34cc8be8e9 # Pinned to the commit that introduced --pg-initdb-sql-dirs (MUL-484)
ARG PG_VERSION=17
+ARG VARIANT
-####################
-# Stage 1: Nix builder
-####################
-FROM alpine:3.23 AS nix-builder
-
+###################################
+# Postgres + Exts #
+###################################
+FROM alpine:${ALPINE_VERSION} AS pg
ARG PG_VERSION
+ARG VARIANT
+
+RUN <<EOF
+case $PG_VERSION:$VARIANT in
+ 15: | 17:) ;;
+ 17:orioledb) VARIANT=$VARIANT-;;
+ 15:orioledb) echo "OrioleDB not supported on PG17" >&2 && exit 1;;
+ 17:*) echo "Unknown VARIANT ($VARIANT) only orioledb supported" >&2 && exit 1;;
+ *) echo "Unknown PG_VERSION ($PG_VERSION) only 15,17 supported" >&2 && exit 1;;
+esac
+EOF
# Install dependencies for nix installer (coreutils for GNU cp, sudo for installer)
RUN apk add --no-cache \
@@ -26,63 +48,50 @@
sudo \
xz
-# Create users (Alpine syntax)
-RUN addgroup -S postgres && \
- adduser -S -h /var/lib/postgresql -s /bin/bash -G postgres postgres && \
- addgroup -S wal-g && \
- adduser -S -s /bin/bash -G wal-g wal-g
-
# Create nix config
-RUN cat <<EOF > /tmp/extra-nix.conf
+RUN cat >/tmp/extra-nix.conf <<EOF
extra-experimental-features = nix-command flakes
extra-substituters = https://nix-postgres-artifacts.s3.amazonaws.com
extra-trusted-public-keys = nix-postgres-artifacts:dGZlQOvKcNEjvT7QEAJbcV6b6uk7VF/hWMjhYleiaLI=
EOF
RUN curl -L https://releases.nixos.org/nix/nix-2.34.6/install | sh -s -- --daemon --no-channel-add --yes --nix-extra-conf-file /tmp/extra-nix.conf
-ENV PATH="${PATH}:/nix/var/nix/profiles/default/bin"
+ENV PATH="$PATH:/nix/var/nix/profiles/default/bin"
RUN nix --version
WORKDIR /nixpg
COPY . .
# Build PostgreSQL with extensions
-RUN nix profile add path:.#psql_${PG_VERSION}_slim/bin
-
-RUN nix store gc
+RUN attr=psql_$VARIANT${PG_VERSION}_slim/bin && \
+ echo Building $attr && \
+ nix profile add path:.#$attr
# Build groonga and copy plugins
RUN nix profile add path:.#supabase-groonga && \
mkdir -p /tmp/groonga-plugins && \
cp -r /nix/var/nix/profiles/default/lib/groonga/plugins /tmp/groonga-plugins/
-RUN nix store gc
+RUN nix store gc && nix store optimise
-####################
-# Stage 2: Gosu builder
-####################
-FROM alpine:3.23 AS gosu-builder
-
-ARG TARGETARCH
-ARG GOSU_VERSION=1.19
-ARG GO_VERSION=1.26.1
+################################################
+# Gosu #
+################################################
+FROM golang:1.26-alpine${ALPINE_VERSION} AS gosu
+ARG GOSU_VERSION
RUN apk add --no-cache curl git
-
-# Install Go
-RUN curl -fsSL "https://go.dev/dl/go${GO_VERSION}.linux-${TARGETARCH}.tar.gz" | tar -C /usr/local -xz
-ENV PATH="/usr/local/go/bin:${PATH}"
-
# Build gosu from source
RUN git clone --depth 1 --branch "${GOSU_VERSION}" https://github.com/tianon/gosu.git /gosu && \
cd /gosu && \
CGO_ENABLED=0 go build -ldflags="-s -w" -o /usr/local/bin/gosu . && \
- chmod +x /usr/local/bin/gosu
-
-####################
-# Stage 3: Final production image
-####################
-FROM alpine:3.23 AS production
+ chmod +x /usr/local/bin/gosu && \
+ /usr/local/bin/gosu --version
+#########################################
+# Production Image #
+#########################################
+FROM alpine:${ALPINE_VERSION} AS postgres
+ARG ALPINE_VERSION
ARG PG_VERSION
# Install minimal runtime dependencies
@@ -104,13 +113,13 @@
adduser postgres wal-g
# Copy Nix store and profiles from builder (profile already created by nix profile install)
-COPY --from=nix-builder /nix /nix
+COPY --from=pg /nix /nix
# Copy groonga plugins
-COPY --from=nix-builder /tmp/groonga-plugins/plugins /usr/lib/groonga/plugins
+COPY --from=pg /tmp/groonga-plugins/plugins /usr/lib/groonga/plugins
# Copy gosu
-COPY --from=gosu-builder /usr/local/bin/gosu /usr/local/bin/gosu
+COPY --from=gosu /usr/local/bin/gosu /usr/local/bin/gosu
# Setup PostgreSQL directories
RUN mkdir -p /usr/lib/postgresql/bin \
@@ -165,20 +174,28 @@
# pg17+ does not ship timescaledb or plv8; db_user_namespace was removed in pg17.
# Applied conditionally so the same Dockerfile works for pg15 (where these are valid).
-RUN if [ "${PG_VERSION}" -ge 17 ]; then \
+RUN if [ "$PG_VERSION" -ge 17 ]; then \
sed -i 's/ timescaledb,//g; s/ plv8,//g' /etc/postgresql/postgresql.conf && \
sed -i 's/db_user_namespace = off/#db_user_namespace = off/g' /etc/postgresql/postgresql.conf && \
sed -i 's/ timescaledb,//g; s/ plv8,//g' /etc/postgresql-custom/supautils.conf; \
fi
+# OrioleDB stuff if building it
+RUN [[ "$VARIANT-" != orioledb ]] && exit 0; \
+ sed -i 's/\(shared_preload_libraries.*\)'\''\(.*\)$/\1, orioledb'\''\2/' "/etc/postgresql/postgresql.conf" && \
+ echo "default_table_access_method = 'orioledb'" >> "/etc/postgresql/postgresql.conf" && \
+ sed -i 's/ postgis,//g; s/ pgrouting,//g' "/etc/postgresql-custom/supautils.conf" && \
+ echo "CREATE EXTENSION orioledb;" > /docker-entrypoint-initdb.d/init-scripts/00-pre-init.sql && \
+ chown postgres:postgres /docker-entrypoint-initdb.d/init-scripts/00-pre-init.sql && \
+ true
+
# Include schema migrations
COPY migrations/db /docker-entrypoint-initdb.d/
COPY ansible/files/pgbouncer_config/pgbouncer_auth_schema.sql /docker-entrypoint-initdb.d/init-scripts/00-schema.sql
COPY ansible/files/stat_extension.sql /docker-entrypoint-initdb.d/migrations/00-extension.sql
-# Add entrypoint script from the official postgres Docker library (version-matched)
ADD --chmod=0755 \
- https://raw.githubusercontent.com/docker-library/postgres/6edb0a8c4def40c371514b34aef9037ec82d9110/${PG_VERSION}/alpine3.23/docker-entrypoint.sh \
+ https://raw.githubusercontent.com/docker-library/postgres/6edb0a8c4def40c371514b34aef9037ec82d9110/${PG_VERSION}/alpine${ALPINE_VERSION}/docker-entrypoint.sh \
/usr/local/bin/docker-entrypoint.sh
# Setup pgsodium key script
@@ -218,3 +235,73 @@
# the postgres process. This follows the standard official PostgreSQL image pattern.
ENTRYPOINT ["docker-entrypoint.sh"]
CMD ["postgres", "-D", "/etc/postgresql"]
+
+##################################################
+## pgctld ##
+##################################################
+FROM golang:1.25-alpine${ALPINE_VERSION} AS pgctld
+ARG PGCTLD_REV
+
+RUN apk add --no-cache git
+
+RUN git clone https://github.com/multigres/multigres.git /multigres && \
+ cd /multigres && \
+ git checkout ${PGCTLD_REV} && \
+ # Copy pico CSS assets before build (mirrors pgctld.nix preBuild step)
+ cp external/pico/pico.* go/common/web/templates/css/ 2>/dev/null || true && \
+ CGO_ENABLED=0 go build -ldflags="-s -w" -o /usr/local/bin/pgctld ./go/cmd/pgctld
+
+
+##########################
+## MULTIGRES ##
+##########################
+FROM postgres AS multigres
+
+RUN if [ "$(postgres --version | awk -F. '{print $1}')" != 'postgres (PostgreSQL) 17' ]; then \
+ echo "Multigres only supports PG17, got -->$(postgres --version)<--" >&2; \
+ exit 1; \
+ fi
+
+# Install pgbackrest (available in Alpine community repo)
+RUN apk add --no-cache pgbackrest
+
+# Copy pgctld binary; keep it separate so the wrapper script can reference it cleanly
+COPY --from=pgctld /usr/local/bin/pgctld /usr/local/bin/pgctld-bin
+
+# pgctld config template — /etc/pgctld is a mount point in k8s so use a custom dir
+COPY docker/pgctld/postgresql.conf.tmpl /etc/pgctld-custom/postgresql.conf.tmpl
+
+# Wrapper: injects --postgres-config-template on every pgctld call so unmodified
+# k8s manifests and local provisioner commands work without extra flags
+COPY --chmod=0755 docker/pgctld/pgctld /usr/local/bin/pgctld
+
+# /etc/postgresql/postgresql.conf is not modified here: pgctld renders its own
+# config from postgresql.conf.tmpl and passes it directly to PostgreSQL, so the
+# supabase base config (including wal-g and data_directory settings) is never loaded.
+
+# wal-g is not used in Multigres (pgbackrest handles backups); remove inherited files.
+RUN rm -f \
+ /etc/postgresql-custom/wal-g.conf \
+ /home/postgres/wal_fetch.sh \
+ /root/wal_change_ownership.sh
+
+# No HEALTHCHECK defined here: inherits the probe from the supabase base image.
+# Kubernetes ignores Docker HEALTHCHECK entirely — use readinessProbe in the Pod spec.
+# STOPSIGNAL inherited from supabase base image (SIGINT — smart shutdown).
+USER postgres
+
+# pgctld is the cluster lifecycle manager for Multigres: it handles initdb,
+# config templating, replication setup, and coordinated restarts. Running it
+# as PID 1 ensures it receives stop signals directly and can shut down
+# PostgreSQL cleanly before the container exits.
+ENTRYPOINT ["/usr/local/bin/pgctld"]
+
+##########################################
+## USELESS ##
+##########################################
+FROM scratch
+RUN echo "You have tried to build this image without specifying --target" >&2 && \
+ echo "Either postgres or multigres is required" >&2 && \
+ echo "P.S.: You can specify PG version with --build-arg=PG_VERSION=15|17" >&2 && \
+ echo "P.P.S: You can build orioledb varian with --build-arg=VARIANT=orioledb" >&2 && \
+ exit 1 |
Author
Ah. The main reason for separating this into its own Dockerfile was precisely to have a more streamlined workflow. If a single Dockerfile is created for all the different images, it means we have a monolith that is hard to maintain (in my opinion, of course.) My thinking was that if we have a good "base", then other projects should be able to build on top of that, even if they are not in the same repo, and having a separate Dockerfile for Multigres creates a good example for how this could be done. |
Author
Thanks for testing this. I'll update the version.
Thanks! I'll take a look at it.
My main concern was the duplication of code in all the files. Even if we end up with a single Dockerfile with multiple targets, it is a significant improvement. |
mkindahl
force-pushed
the
feat/dockerfile-refactor
branch
3 times, most recently
from
e5413bd to
f1207d8
Compare
…ered image
Dockerfile-supabase is a parameterised replacement for the old per-version
Dockerfile-15 / Dockerfile-17. Pass --build-arg PG_VERSION=15|17 to select
the PostgreSQL version (default: 17). Three stages: nix-builder (builds
psql_${PG_VERSION}_slim and supabase-groonga), gosu-builder (builds gosu
from source), and production (Alpine runtime with Nix store, config, and
entrypoint).
Dockerfile-multigres is rewritten as a thin layer on top of the supabase
image instead of a self-contained Nix build. Two stages: pgctld-builder
(compiles pgctld from the pinned multigres commit) and production (adds
pgbackrest, the pgctld wrapper, and config template). pgctld server runs
as PID 1; pgctld init + start are called via docker exec during testing.
pgctld init differences from docker-entrypoint.sh:
- initdb superuser is supabase_admin (POSTGRES_USER); postgres role does
not exist by default. Pre-init SQL creates it as a superuser so that
init-scripts (which run as postgres, matching migrate.sh convention) can
proceed.
- After demote-postgres migration, postgres becomes a non-superuser. A
multigres-specific migration reassigns pgcrypto/uuid-ossp extowner to
supabase_admin to match the reference expected output.
Ansible after-create hooks do not fire in Docker; prime-multigres.sql now
replicates the relevant ones inline: pgmq object ownership to postgres,
pg_tle pgtle_admin grant, and pg_repack default privileges in the repack
schema.
postgresql.conf.tmpl for pgctld uses session_preload_libraries for supautils
(matching Dockerfile-supabase) and includes supautils.conf so reserved-role
enforcement is active.
mkindahl
force-pushed
the
feat/dockerfile-refactor
branch
from
fa9529d to
c6eeada
Compare
ALTER EXTENSION ... OWNER TO is not valid PostgreSQL syntax. Use direct catalog update (matching pgmq after-create.sql pattern).
Contributor
Yeah I'd drop all the other Dockerfiles since they would now be unnecessary. I'd be concerned about the monolithic Dockerfile being hard to maintain but both orioledb and multigres were pretty tiny additions and not as bad as I would have thought. I suspect building the images with nix might not be terrible either (after other cleanups) I'd also expect it to be faster overall since nix's caching is better than docker's. |
Author
I updated the version and ensured that all tests pass (I hope, some are still running). The current changes does not include the single Dockerfile that you proposed: I just wanted to ensure that we have covered everything else before we discuss which approach to take. |
Contributor
I like the one dockerfile-for-everything approach because it really isn't too much extra effort. Multigres add 2 stages, pgctl build and final creation. Not too much to separate out into its own file especially considering that orioledb makes a lot of sense to keep in main Dockerfile, so multigres will odd-build-out. Keeping it all in one file also makes for simpler build in CI IMO. I think we can remove a bunch of the code by having 3 build steps (static or introspecting vars.yml), one each for pg15, pg17, orioledb. Multigres builds took ~30s on my machine on top its base. Thats likely much less time than getting a new CI VM, checking out code, fetching base image... So we'd have a build for each pg, then if version >15 we also build multigres. That should cut down on the matrix generation code and base/layered/suffix stuff too. benchNote I clean between each run because otherwise pgctld is cached and the orioledb flavored multigres finishes in 5s :D./bench.sh
+ docker system prune -af --volumes
+ docker pull alpine:3.23@sha256:5b10f432ef3da1b8d4c7eb6c487f2f5a8f096bc91145e68878dd4a5019afde11
+ docker pull golang:1.25-alpine3.23@sha256:8d22e29d960bc50cd025d93d5b7c7d220b1ee9aa7a239b3c8f55a57e987e8d45
+ docker pull golang:1.26-alpine3.23@sha256:91eda9776261207ea25fd06b5b7fed8d397dd2c0a283e77f2ab6e91bfa71079d
+ PG_VERSION=15
+ docker build --build-arg=PG_VERSION=15 --target=postgres -t supabase-15 .
+ tee pg-15.log.swp
real 1m41.767s
user 0m0.697s
sys 0m0.411s
+ docker system prune -af --volumes
+ docker pull alpine:3.23@sha256:5b10f432ef3da1b8d4c7eb6c487f2f5a8f096bc91145e68878dd4a5019afde11
+ docker pull golang:1.25-alpine3.23@sha256:8d22e29d960bc50cd025d93d5b7c7d220b1ee9aa7a239b3c8f55a57e987e8d45
+ docker pull golang:1.26-alpine3.23@sha256:91eda9776261207ea25fd06b5b7fed8d397dd2c0a283e77f2ab6e91bfa71079d
+ PG_VERSION=17
+ docker build --build-arg=PG_VERSION=17 --target=postgres -t supabase-17 .
+ tee pg-17.log.swp
real 1m36.863s
user 0m0.686s
sys 0m0.374s
+ docker build --build-arg=PG_VERSION=17 --target=multigres -t supabase-17-multigres .
+ tee pg-17-multigres.log.swp
real 0m20.832s
user 0m0.140s
sys 0m0.139s
+ docker system prune -af --volumes
+ docker pull alpine:3.23@sha256:5b10f432ef3da1b8d4c7eb6c487f2f5a8f096bc91145e68878dd4a5019afde11
+ docker pull golang:1.25-alpine3.23@sha256:8d22e29d960bc50cd025d93d5b7c7d220b1ee9aa7a239b3c8f55a57e987e8d45
+ docker pull golang:1.26-alpine3.23@sha256:91eda9776261207ea25fd06b5b7fed8d397dd2c0a283e77f2ab6e91bfa71079d
+ docker build --build-arg=PG_VERSION=17 --build-arg=VARIANT=orioledb --target=postgres -t supabase-17-orioledb .
+ tee pg-17-orioledb.log.swp
real 1m37.622s
user 0m0.715s
sys 0m0.414s
+ docker build --build-arg=PG_VERSION=17 --build-arg=VARIANT=orioledb --target=multigres -t supabase-17-orioledb-multigres .
+ tee pg-17-orioledb-multigres.log.swp
real 0m22.151s
user 0m0.210s
sys 0m0.129 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add
Dockerfile-supabaseand rewriteDockerfile-multigresas a layered imageSummary
The existing
Dockerfile-15andDockerfile-17are near-identical files that diverge only in a handful of version-specific lines. This PR introducesDockerfile-supabase, a single parameterised Dockerfile that replaces both, selectable via--build-arg PG_VERSION=15|17(default 17). It also rewritesDockerfile-multigresfrom a self-contained Nix-based build into a thin layered image built on top ofDockerfile-supabase, which significantly reduces duplication and makes the multigres image easier to keep in sync with the base.Dockerfile-15,Dockerfile-17, andDockerfile-orioledb-17are unchanged and remain in the repo.Dockerfile-supabasereplacesDockerfile-15andDockerfile-17in the release workflows;Dockerfile-orioledb-17continues to be built viarelease_matrix_layered.Changes
Dockerfile-supabase(new)Parameterised replacement for
Dockerfile-15andDockerfile-17. The three-stage structure (nix-builder,gosu-builder,production) is identical to the existing files. Version-specific differences (strippingtimescaledb/plv8, commenting outdb_user_namespace) are gated on[ "$PG_VERSION" -ge 17 ]at build time.Dockerfile-multigres(rewritten)Reduced from ~320 lines to ~70. A
pgctld-builderstage compilespgctldfrom Go source, pinned to the commit that introduced--pg-initdb-sql-dirs(MUL-484). The final stage doesFROM ${SUPABASE_IMAGE} AS production, addingpgbackrest, thepgctldbinary and wrapper script, a config template, and removing inheritedwal-gfiles. The base image is overridable via--build-arg SUPABASE_IMAGE.The old Dockerfile had separate
variant-17andvariant-orioledb-17final stages, selected via--target. These are replaced by a singleproductionstage — consistent with all other Dockerfiles. Thevariant-orioledb-17target is removed entirely; a multigres image layered on top of OrioleDB is not built in this PR (the standaloneDockerfile-orioledb-17image continues to be built viarelease_matrix_layered). All Dockerfiles now useproductionas the final stage name, so--target productionis passed unconditionally by the workflows and test scripts.docker/pgctld/pgctld(new, replacesdocker/pgctld/pgctld-wrapper)Wrapper script installed at
/usr/local/bin/pgctldthat injects--postgres-config-templateand--pg-initdb-sql-dirson everypgctldcall, so Kubernetes manifests work without extra flags.The old
pgctld-wrapperalso created a symlink/var/log/postgresql/postgresql.json -> /proc/1/fd/1to bridge PostgreSQL's JSON log file to the container's stdout (for kubelet + Vector, introduced in commit9313c070). This log-bridging is not carried over to the new wrapper. Whether pgctld itself now handles stdout logging needs to be confirmed before merging.ansible/vars.ymlvars.ymlis now the single source of truth for which images are built and how they are tagged. Two new sections are added:release_matrix_base— lists the base Dockerfiles to build (currentlyDockerfile-supabaseat pg15 and pg17)release_matrix_layered— lists the layered Dockerfiles to build on top of them (currentlyDockerfile-multigresandDockerfile-orioledb-17)Each entry carries a
release_keythat maps to a full version string inpostgres_release, and an optionaltag_suffix(e.g.-multigres). Image tags are derived entirely from these fields at workflow runtime — nothing is hardcoded in the workflow files themselves. To add a new version or change a tag, onlyvars.ymlneeds to be updated.Release workflows (
dockerhub-release-matrix.yml,manual-docker-release.yml)The release flow is split into two phases:
build_base_imagesbuilds and pushes the supabase base images first, thenbuild_layered_imagesbuilds images that depend on them. Strictly speaking, the OrioleDB image is not a layered image since it builds a patched version of Postgres inside the Dockerfile, but logically it is a layered image (if all patches were accepted into Postgres, it would be a pure layered image) and is therefore treated as one.The
preparejob now reads the explicit matrix fromvars.ymlrather than scanning the filesystem for Dockerfiles and inspecting their stage names. The singlebuild_release_imagejob is replaced by two sequential jobs:build_base_images— runs first, pushes base images to the registrybuild_layered_images— depends onbuild_base_images, passesSUPABASE_IMAGE=supabase/postgres:<base_tag>_<arch>as a build argThis ordering is required because
Dockerfile-multigresis now aFROMreference to the base image rather than a self-contained build. Tag computation inmerge_manifestis simplified from a ~15-line Nushell script per entry to a singleecho, since tags are precomputed inprepare.docker-image-test.ymlDockerfile-15andDockerfile-17replaced by twoDockerfile-supabaseentries withpg_version: "15"/"17"multigres-17entry gainsbase_dockerfile: Dockerfile-supabase, which triggers an inline base build before the layered build (each test runner has an isolated Docker daemon with no shared registry)--target productionis now hardcoded — all Dockerfiles use aproductionfinal stagemultigres-orioledb-17entry removed — that target no longer exists inDockerfile-multigresnix/packages/docker-image-test.nix,nix/packages/image-size-analyzer.nixBoth gain a
--pg-versionflag so they can pass--build-arg PG_VERSIONwhen building or analyzingDockerfile-supabase.Testing
The existing
docker-image-testsuite runs SQL regression tests against a live container for each matrix entry. The CI workflow now runs it against:Dockerfile-supabaseat pg15 and pg17 (replacing the formerDockerfile-15/Dockerfile-17entries)Dockerfile-multigres(multigres-17), withDockerfile-supabasebuilt inline as the baseDockerfile-orioledb-17tests are unchanged.Fixes MUL-483