feat: enable superset stats API to fetch extra columns that are not p…

[timothygachengo]

…art of the dimensions

Description

Type of change

• Bug fix (fix:)
• New feature (feat:)
• Breaking change (BREAKING CHANGE:)
• Documentation update (docs:)
• Refactor / chore (refactor: / chore:)

Affected module(s)

• api-gateway
• api-security
• commons
• registry
• superset-proxy

Checklist

• PR title follows Conventional Commits format (determines version bump)
• mvn checkstyle:check passes
• mvn test passes
• No hardcoded credentials, internal URLs, client names, or PII introduced
• Any new Maven dependency has an Apache-2.0-compatible license
• Database migrations added to src/main/resources/db/changelog/ if schema changed

[Copilot]

Pull request overview

This PR extends the superset-proxy stats endpoint to support returning additional dataset columns (via an includeColumns query parameter) without affecting the grouping/breakdown of the stats query, and makes the Superset base URL configurable in the dev profile.

Changes:

• Make viz.superset.url configurable via SUPERSET_URL in application-dev.yml (defaulting to localhost).
• Add includeColumns parsing/validation (against dataset metadata) and thread the resolved list through request building and response transformation.
• Implement “extra columns” retrieval by adding adhoc metrics to Superset queries and emitting the resolved values into each returned data item.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File	Description
superset-proxy/src/main/resources/application-dev.yml	Use SUPERSET_URL env var to configure Superset URL in dev.
superset-proxy/src/main/java/org/devgateway/viz/gateway/services/Utils.java	Add query/data helpers for extra columns (adhoc metric creation + emitting extra fields per data row).
superset-proxy/src/main/java/org/devgateway/viz/gateway/services/StatsService.java	Parse/validate includeColumns, pass extra columns into Superset request construction and response transformation.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

Suppressed comments (2)

superset-proxy/src/main/java/org/devgateway/viz/gateway/services/StatsService.java:46

• includeColumns can include duplicates, group-by dimensions, or existing metric names. That can lead to duplicate adhoc metrics and key collisions (extra-column metric label == group-by column or metric), producing ambiguous/incorrect rows. Consider de-duplicating and filtering out group-by columns and metric names before building the Superset request.

        Map<String, String> labelMap = buildLabelMap(datasetResult);
        List<String> extraColumns = resolveIncludeColumns(queryParams, datasetResult);
        JsonNode requestBody = buildSupersetDataRequest(datasetResult, datasetId, queryParams, groupsPath, extraColumns);

superset-proxy/src/main/java/org/devgateway/viz/gateway/services/StatsService.java:97

• Use parameterized logging instead of string concatenation so logging stays consistent and avoids unnecessary string building when the log level is disabled.

                logger.warn("includeColumns requested unknown column: " + trimmed);

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

Suppressed comments (3)

Previously missed (2) — in code that hasn't changed since the last review.

superset-proxy/src/main/java/org/devgateway/viz/gateway/services/Utils.java:224

• extractAllColumns assumes result.get("columns") is present and iterable; if Superset returns a dataset result without a "columns" array, this will throw a NullPointerException when includeColumns is used. Guard against missing/non-array nodes (similar to buildLabelMap).

    public static Set<String> extractAllColumns(JsonNode result) {
        Set<String> columnNames = new LinkedHashSet<>();
        for (JsonNode column : result.get("columns")) {
            String name = column.path("column_name").asText(null);
            if (name != null && !name.isEmpty()) {

superset-proxy/src/main/java/org/devgateway/viz/gateway/services/Utils.java:96

• createQuery is sorted explicitly for cache-key stability, but the new adhoc metric payload uses a HashMap. Jackson serialization of HashMap keys is not deterministic, which can make the request body ordering unstable and reduce cache hit rate. Use an order-stable map (e.g., TreeMap/LinkedHashMap) for the metric object.

    public static Map<String, Object> createExtraColumnMetric(String column) {
        Map<String, Object> metric = new HashMap<>();
        metric.put("expressionType", "SQL");
        metric.put("sqlExpression", "MAX(" + column + ")");
        metric.put("label", column);
        return metric;

.github/workflows/release-template.yml:88

• Using pre_release_branches: ".*" allows this workflow_call to generate pre-release tags from any branch when invoked via workflow_dispatch (or from any caller). This increases the risk of publishing/advertising release candidates from unintended branches; consider restricting to main (or to a configurable allowlist) like before.

          release_branches:  __no-stable-release-branch__
          pre_release_branches: .*
          append_to_pre_release_tag: rc