Skip to content

Introduce new app config - #4022

Open
dzemanov wants to merge 5 commits into
redhat-developer:mainfrom
dzemanov:config-thresholds
Open

Introduce new app config#4022
dzemanov wants to merge 5 commits into
redhat-developer:mainfrom
dzemanov:config-thresholds

Conversation

@dzemanov

@dzemanov dzemanov commented Jul 28, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Hey, I just made a Pull Request!

Thresholds moved from provider level to metric level. Configuration is restructured to enable thresholds to be defined directly for each metric with fallback options at the provider level. The new top-level key metricProviders now houses all metric-specific configurations. Threshold annotation overrides now require full metric ID instead of provider ID.

Example configuration:

scorecard:
  metricProviders:
    github:
      openPRs:
        schedule:
        thresholds:
      actionsCount: 
        metrics:
          actionsStarted7d:
            thresholds:
          actionsSuccessful7d:
            thresholds:
     filecheck:
       fileExistence:
         schedule:
         thresholds:
         options:
           files:
             license: 'LICENSE'
             codeowners: 'CODEOWNERS'

Fixes

Fixes https://redhat.atlassian.net/browse/RHDHBUGS-3340

How to test

  1. Test out that new filecheck configuration works and schedule configuration works
scorecard:
  metricProviders:
    filecheck:
      fileExistence:
        options:
          files:
            license: 'LICENSE'
            codeowners: 'CODEOWNERS'
        schedule:
          frequency: { minutes: 5 }
          timeout: { minutes: 10 }
          initialDelay: { seconds: 10 }
  1. Test out custom thresholds from metric level only override license
scorecard:
  metricProviders:
    filecheck:
      fileExistence:
        options:
          files:
            license: 'LICENSE'
            codeowners: 'CODEOWNERS'
        metrics:
          license:
            thresholds:
              rules:
                - key: present
                  expression: '==true'
                  icon: scorecardSuccessStatusIcon
                  color: 'success.main'
                - key: absent
                  expression: '==false'
                  icon: scorecardErrorStatusIcon
                  color: 'error.main'
  1. Test out custom thresholds from provider level override all filecheck fileExistence metrics
scorecard:
  metricProviders:
    filecheck:
      fileExistence:
        options:
          files:
            license: 'LICENSE'
            codeowners: 'CODEOWNERS'
        thresholds:
          rules:
            - key: present
              expression: '==true'
              icon: scorecardSuccessStatusIcon
              color: 'success.main'
            - key: absent
              expression: '==false'
              icon: scorecardErrorStatusIcon
              color: 'error.main'
  1. Test out custom metric thresholds override custom provider thresholds
scorecard:
  metricProviders:
    filecheck:
      fileExistence:
        options:
          files:
            license: 'LICENSE'
            codeowners: 'CODEOWNERS'
        thresholds:
          rules:
            - key: present
              expression: '==true'
              icon: scorecardSuccessStatusIcon
              color: 'success.main'
            - key: absent
              expression: '==false'
              icon: scorecardErrorStatusIcon
              color: 'error.main'
        metrics:
          license:
            thresholds:
              rules:
                - key: ok
                  expression: '==true'
                  icon: scorecardSuccessStatusIcon
                  color: 'success.main'
                - key: nok
                  expression: '==false'
                  icon: scorecardErrorStatusIcon
                  color: 'error.main'
  1. Test out entity annotations override thresholds:
    scorecard.io/github.openPRs.thresholds.rules.warning: '10-15'
    scorecard.io/github.openPRs.thresholds.rules.error: '>15'

✔️ Checklist

  • A changeset describing the change and affected packages. (more info)
  • Added or Updated documentation
  • Tests for new functionality and regression tests for bug fixes
  • Screenshots attached (for UI changes)

@dzemanov
Introduce new app config
a4fc84e 
Assisted-By: Cursor Desktop
Signed-off-by: Dominika Zemanovicova <dzemanov@redhat.com>
@rhdh-gh-app

rhdh-gh-app Bot commented Jul 28, 2026 •
edited
Loading

Copy link
Copy Markdown

Important

This PR includes changes that affect public-facing API. Please ensure you are adding/updating documentation for new features or behavior.

Changed Packages

Package Name Package Path Changeset Bump Current Version
@red-hat-developer-hub/backstage-plugin-scorecard-backend-module-dependabot workspaces/scorecard/plugins/scorecard-backend-module-dependabot major v0.3.1
@red-hat-developer-hub/backstage-plugin-scorecard-backend-module-filecheck workspaces/scorecard/plugins/scorecard-backend-module-filecheck major v0.2.1
@red-hat-developer-hub/backstage-plugin-scorecard-backend-module-github workspaces/scorecard/plugins/scorecard-backend-module-github major v2.8.1
@red-hat-developer-hub/backstage-plugin-scorecard-backend-module-jira workspaces/scorecard/plugins/scorecard-backend-module-jira major v2.8.1
@red-hat-developer-hub/backstage-plugin-scorecard-backend-module-openssf workspaces/scorecard/plugins/scorecard-backend-module-openssf major v0.3.1
@red-hat-developer-hub/backstage-plugin-scorecard-backend-module-sonarqube workspaces/scorecard/plugins/scorecard-backend-module-sonarqube major v0.2.1
@red-hat-developer-hub/backstage-plugin-scorecard-backend workspaces/scorecard/plugins/scorecard-backend major v2.8.1
@red-hat-developer-hub/backstage-plugin-scorecard-common workspaces/scorecard/plugins/scorecard-common major v2.8.1

@fullsend-ai-review

fullsend-ai-review Bot commented Jul 28, 2026 •
edited
Loading

Copy link
Copy Markdown

🤖 Review · ⚠️ Cancelled · Started 8:25 AM UTC · Ended 8:35 AM UTC
Commit: a4fc84e · View workflow run →

@codecov

codecov Bot commented Jul 28, 2026 •
edited
Loading

Copy link
Copy Markdown

❌ 2 Tests Failed:

Tests completed Failed Passed Skipped
1164 2 1162 0
View the top 2 failed test(s) by shortest run time 
Scorecard MCP tools integration filters scorecard.list-metrics for conditional metric permissions
Stack Traces | 0.033s run time 
Error: expect(received).toEqual(expected) // deep equality

- Expected  - 3
+ Received  + 1

- Array [
-   "filecheck.license",
- ]
+ Array []
    at toEqual (.../backend/src/mcp-tools.integration.test.ts:126:25)
    at processTicksAndRejections (node:internal/process/task_queues:103:5)
    at withMcpClient (.../src/__fixtures__/mcpTestUtils.ts:181:12)
    at Object.<anonymous> (.../backend/src/mcp-tools.integration.test.ts:114:5)
Scorecard MCP tools integration calls scorecard.list-metrics and returns configured metrics
Stack Traces | 0.04s run time 
Error: expect(received).toEqual(expected) // deep equality

Expected: ArrayContaining ["filecheck.license", "filecheck.codeowners"]
Received: []
    at toEqual (.../backend/src/mcp-tools.integration.test.ts:107:25)
    at processTicksAndRejections (node:internal/process/task_queues:103:5)
    at withMcpClient (.../src/__fixtures__/mcpTestUtils.ts:181:12)
    at Object.<anonymous> (.../backend/src/mcp-tools.integration.test.ts:96:5)

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@rhdh-qodo-merge

rhdh-qodo-merge Bot commented Jul 28, 2026

Copy link
Copy Markdown

PR Summary by Qodo

Move Scorecard thresholds to metric-level config with datasource/provider fallbacks

✨ Enhancement ⚙️ Configuration changes 📝 Documentation 🧪 Tests 🕐 40+ Minutes

Grey Divider

AI Description 

• Restructure scorecard plugin config under metricProviders, adding metric-level threshold
 overrides.
• Add datasource/provider schedule configuration with deterministic fallback behavior.
• Update threshold resolution and entity annotations to key off full metric IDs.
Diagram 

graph TD
  A["app-config.yaml (scorecard.plugins)"] --> B["metricProviderConfigKeys"] --> C["ThresholdResolver"] --> D["mergeEntityAndMetricThresholds"]
  A --> B --> E["PullMetricsByProviderTask"]
  F["MetricProvidersRegistry"] --> G["validateMetricProviderIds"]
  H["Docs & examples"] --> A
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Backward-compatible config shim (support old keys temporarily)
  • ➕ Avoids a hard breaking change for existing deployments
  • ➕ Allows deprecation messaging and phased migration
  • ➖ Adds ongoing complexity in config resolution code and typings
  • ➖ Requires clear precedence rules between old/new schemas
2. Provider-level thresholds only + per-metric override map
  • ➕ Keeps existing provider-level structure intact
  • ➕ Adds per-metric customization without introducing provider/metric nesting
  • ➖ Still needs new schema surface area; can become awkward for batch providers
  • ➖ Harder to align schedules/thresholds consistently across levels
3. Centralized config normalization step at startup
  • ➕ Keeps runtime logic simpler by operating on a normalized in-memory model
  • ➕ Single place to validate and report config errors
  • ➖ More upfront refactor and more involved migration logic
  • ➖ May complicate debugging if normalization diverges from raw config

Recommendation: If a major-version breaking change is acceptable, the PR’s approach is coherent: it makes thresholds truly per-metric and introduces a clean fallback chain (metric > provider > datasource) using shared path helpers. If ecosystem impact is a concern, consider a short-lived backward-compatible shim that reads old provider-level paths and logs deprecation warnings, but keep the new model as the single source of truth.

Files changed (35) +1493 / -805

Enhancement (6) +255 / -61
MetricProvidersRegistry.tsValidate provider and metric IDs; enforce unique provider IDs +25/-14

Validate provider and metric IDs; enforce unique provider IDs

• Introduces explicit providerId validation and a unique providerId registry for scheduler/config key safety. Validates each metric ID, changes conflict errors to report duplicate metric IDs precisely, and clarifies datasource index semantics.

workspaces/scorecard/plugins/scorecard-backend/src/providers/MetricProvidersRegistry.ts

PullMetricsByProviderTask.tsResolve provider schedule via shared config-key helpers +7/-14

Resolve provider schedule via shared config-key helpers

• Replaces per-provider schedule path construction with 'resolveScheduleFromConfig()' and falls back to a default schedule when absent. Updates threshold resolution calls to rely solely on the metric object (providerId no longer required).

workspaces/scorecard/plugins/scorecard-backend/src/scheduler/tasks/PullMetricsByProviderTask.ts

ThresholdResolver.tsLoad configured thresholds per metric with multi-level config fallback +27/-21

Load configured thresholds per metric with multi-level config fallback

• Changes the internal cache to map 'metricId -> ThresholdConfig' (instead of providerId). Uses 'resolveThresholdsConfigPath()' to pick the most specific configured thresholds and merges entity overrides via the metric-id keyed merge helper.

workspaces/scorecard/plugins/scorecard-backend/src/threshold/ThresholdResolver.ts

mergeEntityAndMetricThresholds.tsMerge entity annotation overrides using full metric ID prefix +12/-12

Merge entity annotation overrides using full metric ID prefix

• Switches the annotation prefix from providerId-based to metricId-based ('scorecard.io/<metricId>.thresholds.rules.*'). Updates error messages and keeps rule replacement/coverage validation semantics intact.

workspaces/scorecard/plugins/scorecard-backend/src/utils/mergeEntityAndMetricThresholds.ts

metricProviderConfigKeys.tsIntroduce centralized helpers for schedule/threshold config paths +136/-0

Introduce centralized helpers for schedule/threshold config paths

• Adds a small utility module to compute local provider/metric keys and build config paths. Implements 'resolveScheduleFromConfig' and 'resolveThresholdsConfigPath' to encapsulate precedence rules.

workspaces/scorecard/plugins/scorecard-backend/src/utils/metricProviderConfigKeys.ts

validateMetricProviderIds.tsAdd providerId and metricId format validators +48/-0

Add providerId and metricId format validators

• Introduces 'validateProviderId' (allows '<datasource>' or '<datasource>.<providerName>') and 'validateMetricId' (requires '<datasource>.<metricName>'). Used by registry to enforce consistent IDs.

workspaces/scorecard/plugins/scorecard-backend/src/validation/validateMetricProviderIds.ts

Refactor (3) +3 / -15
jiraOpenIssues.tsUpdate Jira provider config path constant +1/-1

Update Jira provider config path constant

• Changes the Jira Open Issues config base path to 'scorecard.plugins.jira.metricProviders.openIssues' to match the new schema.

workspaces/scorecard/plugins/scorecard-backend-module-jira/src/constants/jiraOpenIssues.ts

CatalogMetricService.tsStop using providerId when resolving entity thresholds +0/-2

Stop using providerId when resolving entity thresholds

• Removes unnecessary provider lookup and resolves entity thresholds based on the metric alone. Aligns service behavior with metric-id keyed threshold configuration.

workspaces/scorecard/plugins/scorecard-backend/src/service/CatalogMetricService.ts

router.tsRemove provider dependency from metric threshold resolution endpoints +2/-12

Remove provider dependency from metric threshold resolution endpoints

• Eliminates registry provider lookups when resolving thresholds for metric/aggregation responses. Uses metric-only threshold resolution consistently.

workspaces/scorecard/plugins/scorecard-backend/src/service/router.ts

Tests (9) +747 / -351
testUtils.tsUpdate Jira test fixtures for new config schema +5/-3

Update Jira test fixtures for new config schema

• Rehomes fixture config from 'jira.openIssues' to 'jira.metricProviders.openIssues', preserving options/thresholds in the expected structure.

workspaces/scorecard/plugins/scorecard-backend-module-jira/fixtures/testUtils.ts

MetricProvidersRegistry.test.tsAdd tests for provider/metric ID validation and duplicate detection +87/-56

Add tests for provider/metric ID validation and duplicate detection

• Updates registry tests to distinguish duplicate metric IDs vs duplicate provider IDs. Adds mocking to assert 'validateProviderId' and 'validateMetricId' are invoked during registration.

workspaces/scorecard/plugins/scorecard-backend/src/providers/MetricProvidersRegistry.test.ts

PullMetricsByProviderTask.test.tsUpdate scheduler task tests for datasource/provider schedule resolution +64/-48

Update scheduler task tests for datasource/provider schedule resolution

• Moves schedule config from top-level to datasource-level ('scorecard.plugins.<ds>.schedule') and validates default schedule fallback behavior. Updates threshold resolver expectations to resolve per entity+metric (no providerId argument).

workspaces/scorecard/plugins/scorecard-backend/src/scheduler/tasks/PullMetricsByProviderTask.test.ts

CatalogMetricService.test.tsAdjust service tests for metric-keyed threshold merging +1/-12

Adjust service tests for metric-keyed threshold merging

• Updates test expectations to merge entity and metric thresholds without passing providerId. Removes redundant provider lookup assertions now that metric lookup is sufficient.

workspaces/scorecard/plugins/scorecard-backend/src/service/CatalogMetricService.test.ts

router.test.tsUpdate router tests for metric-only threshold resolution +1/-5

Update router tests for metric-only threshold resolution

• Adjusts tests to call 'resolveMetricThresholds(metric)' without providing a providerId, reflecting the new resolver API.

workspaces/scorecard/plugins/scorecard-backend/src/service/router.test.ts

ThresholdResolver.test.tsExpand tests for datasource/provider/metric threshold precedence +235/-115

Expand tests for datasource/provider/metric threshold precedence

• Rewrites tests to reflect per-metric configured thresholds and adds coverage for datasource-level fallback, provider-level override, and metric-level override precedence. Adds a batch-provider test for per-metric overrides when providerId equals datasource.

workspaces/scorecard/plugins/scorecard-backend/src/threshold/ThresholdResolver.test.ts

mergeEntityAndMetricThresholds.test.tsRename/retarget merge tests to metric-id keyed annotations +28/-112

Rename/retarget merge tests to metric-id keyed annotations

• Updates the test suite to validate merge behavior keyed to metric IDs, including correct ignore behavior for non-matching prefixes and updated error messages referencing metrics (not providers).

workspaces/scorecard/plugins/scorecard-backend/src/utils/mergeEntityAndMetricThresholds.test.ts

metricProviderConfigKeys.test.tsAdd unit tests for config key/path helpers and precedence rules +259/-0

Add unit tests for config key/path helpers and precedence rules

• Introduces tests for local key derivation, schedule resolution (provider over datasource), and threshold path precedence (metric > provider > datasource). Covers both single-metric and batch-provider cases.

workspaces/scorecard/plugins/scorecard-backend/src/utils/metricProviderConfigKeys.test.ts

validateMetricProviderIds.test.tsAdd unit tests for providerId/metricId validation rules +67/-0

Add unit tests for providerId/metricId validation rules

• Covers allowed forms ('<datasource>' for provider IDs, '<datasource>.<name>' for both) and rejects invalid segment counts, empty names, and wrong prefixes.

workspaces/scorecard/plugins/scorecard-backend/src/validation/validateMetricProviderIds.test.ts

Documentation (10) +385 / -300
metric-level-thresholds-config.mdAdd breaking-change changeset for metric-level thresholds config +44/-0

Add breaking-change changeset for metric-level thresholds config

• Introduces a major-version changeset for scorecard backend and datasource modules. Documents the new 'metricProviders' nesting, threshold precedence (metric > provider > datasource), schedule keys, and the new full-metric-id annotation format.

workspaces/scorecard/.changeset/metric-level-thresholds-config.md

AGENTS.mdUpdate guidance docs for metric-id based threshold resolution +77/-85

Update guidance docs for metric-id based threshold resolution

• Refreshes internal documentation to reflect thresholds keyed by metric ID and the new config paths. Updates tables/sections to remove provider-level-only assumptions and clarifies provider vs metric ID formats.

workspaces/scorecard/AGENTS.md

README.mdUpdate Dependabot README config examples for 'metricProviders' +10/-9

Update Dependabot README config examples for 'metricProviders'

• Moves threshold override examples under 'scorecard.plugins.dependabot.metricProviders.<provider>.thresholds'. Keeps guidance about replacing provider names for other severities.

workspaces/scorecard/plugins/scorecard-backend-module-dependabot/README.md

README.mdClarify filecheck provider/metric IDs and schedule docs +7/-7

Clarify filecheck provider/metric IDs and schedule docs

• Extends the metrics table to include provider IDs (notably batch provider id 'filecheck'). Adds a link to central scheduling documentation for configuration details.

workspaces/scorecard/plugins/scorecard-backend-module-filecheck/README.md

README.mdUpdate GitHub README for provider/metric IDs and new config paths +22/-19

Update GitHub README for provider/metric IDs and new config paths

• Adds explicit Metric Provider ID documentation and migrates threshold/schedule snippets under 'metricProviders.openPRs'. Updates wording and links to central docs for schedule/threshold configuration.

workspaces/scorecard/plugins/scorecard-backend-module-github/README.md

README.mdUpdate Jira README examples for 'metricProviders' nesting +27/-24

Update Jira README examples for 'metricProviders' nesting

• Migrates options, schedule, and thresholds examples under 'scorecard.plugins.jira.metricProviders.openIssues'. Adds reference to central scheduling docs.

workspaces/scorecard/plugins/scorecard-backend-module-jira/README.md

README.mdUpdate OpenSSF README to include provider IDs and new config paths +31/-30

Update OpenSSF README to include provider IDs and new config paths

• Updates the metrics listing to show provider IDs and migrates default threshold configuration under 'metricProviders'. Clarifies wording around provider keys.

workspaces/scorecard/plugins/scorecard-backend-module-openssf/README.md

README.mdUpdate SonarQube README for provider IDs and 'metricProviders' config +85/-80

Update SonarQube README for provider IDs and 'metricProviders' config

• Adds provider ID column in metric tables and migrates all threshold examples to 'scorecard.plugins.sonarqube.metricProviders.<provider>.thresholds'. Updates guidance text to reference provider keys consistently.

workspaces/scorecard/plugins/scorecard-backend-module-sonarqube/README.md

thresholds.mdRewrite thresholds documentation for metric-level overrides and annotations +81/-45

Rewrite thresholds documentation for metric-level overrides and annotations

• Updates docs from provider-default to metric-default thresholds, introduces app-config precedence (metric > provider > datasource), and changes entity annotation format to 'scorecard.io/<metricId>.thresholds...'. Updates examples and priority/merging behavior descriptions accordingly.

workspaces/scorecard/plugins/scorecard-backend/docs/thresholds.md

threshold.tsClarify ThresholdConfig documentation comment +1/-1

Clarify ThresholdConfig documentation comment

• Updates the type docstring to explicitly describe ThresholdConfig as categorization rules for metric values.

workspaces/scorecard/plugins/scorecard-common/src/types/threshold.ts

Other (7) +103 / -78
app-config.local.EXAMPLE.yamlMigrate local example config to 'metricProviders' nesting +24/-22

Migrate local example config to 'metricProviders' nesting

• Moves Jira/GitHub provider configuration under 'plugins.<datasource>.metricProviders.<provider>'. Updates examples for schedule and threshold overrides to match new schema.

workspaces/scorecard/app-config.local.EXAMPLE.yaml

app-config.production.yamlUpdate production config comments for provider schedule nesting +6/-5

Update production config comments for provider schedule nesting

• Adjusts commented example to use 'metricProviders.<provider>.schedule' instead of the previous direct provider key under the datasource.

workspaces/scorecard/app-config.production.yaml

app-config.yamlUpdate main app config to provider schedule under 'metricProviders' +12/-10

Update main app config to provider schedule under 'metricProviders'

• Migrates Jira and GitHub schedule configuration to 'plugins.<datasource>.metricProviders.<provider>.schedule' in the repository’s canonical app-config example.

workspaces/scorecard/app-config.yaml

config.d.tsAdd filecheck datasource schedule and thresholds typing +6/-1

Add filecheck datasource schedule and thresholds typing

• Updates type definitions to support 'schedule' and 'thresholds' at the filecheck datasource level, and renames file map key from '[metricId]' to '[metricName]' to match local-key semantics.

workspaces/scorecard/plugins/scorecard-backend-module-filecheck/config.d.ts

config.d.tsRestructure GitHub config typing under 'metricProviders' +8/-8

Restructure GitHub config typing under 'metricProviders'

• Replaces the previous 'openPRs'-direct config block with 'metricProviders.openPRs' containing 'schedule' and 'thresholds' typed as shared 'ThresholdConfig'.

workspaces/scorecard/plugins/scorecard-backend-module-github/config.d.ts

config.d.tsRestructure Jira config typing under 'metricProviders' +11/-12

Restructure Jira config typing under 'metricProviders'

• Moves 'openIssues' config under 'metricProviders.openIssues' and switches thresholds typing to shared 'ThresholdConfig', alongside schedule typing.

workspaces/scorecard/plugins/scorecard-backend-module-jira/config.d.ts

config.d.tsRedesign backend config typing for datasource/provider/metric fallbacks +36/-20

Redesign backend config typing for datasource/provider/metric fallbacks

• Replaces the previous per-metric-per-datasource map with a structured schema: datasource-level 'schedule'/'thresholds', 'metricProviders' for provider-level overrides, and optional per-metric overrides under 'metrics'. Documents override semantics in JSDoc.

workspaces/scorecard/plugins/scorecard-backend/config.d.ts

@dzemanov
add commin
477650e 
Signed-off-by: Dominika Zemanovicova <dzemanov@redhat.com>
@fullsend-ai-review

fullsend-ai-review Bot commented Jul 28, 2026 •
edited
Loading

Copy link
Copy Markdown

🤖 Review · ❌ Terminated · Started 8:36 AM UTC · Ended 9:16 AM UTC
Commit: 477650e · View workflow run →

@rhdh-qodo-merge

rhdh-qodo-merge Bot commented Jul 28, 2026 •
edited
Loading

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

Context used
✅ Compliance rules (platform): 11 rules
✅ Cross-repo context
  Explored: repo: redhat-developer/rhdh (sha: 6bcb0141)
  Explored: repo: redhat-developer/rhdh-operator (sha: 095451d9)
  Explored: repo: redhat-developer/rhdh-local (sha: 2ae9e8c8)
  Not relevant to this PR: redhat-developer/rhdh-chart

Grey Divider


Remediation recommended

1. Schedule docs outdated ✓ Resolved 🐞 Bug ⚙ Maintainability
Description 
The scheduler now reads schedule config from scorecard.plugins.<datasource>.schedule and
scorecard.plugins.<datasource>.metricProviders.<provider>.schedule, but docs/providers.md still
instructs users to configure schedules under
scorecard.plugins.{datasourceId}.{metricName}.schedule, which will be ignored and fall back to the
default schedule.
Code

workspaces/scorecard/plugins/scorecard-backend/src/scheduler/tasks/PullMetricsByProviderTask.ts[R88-93]

+    const schedule =
+      resolveScheduleFromConfig(
+        this.config,
+        this.provider.getProviderDatasourceId(),
+        this.providerId,
+      ) ?? PullMetricsByProviderTask.DEFAULT_SCHEDULE;
Relevance

⭐⭐⭐ High

Outdated config docs are typically corrected; schema/example path fixes accepted recently (e.g., PR
#3417) and providers.md doc fixes in #3526.

PR-#3417
PR-#3526

ⓘ Recommendations generated based on similar findings in past PRs

Evidence 
The scheduler task resolves schedules using resolveScheduleFromConfig(...), which only checks the
datasource-level and metricProviders.<provider>.schedule paths. docs/providers.md still tells
users to configure schedules under scorecard.plugins.{datasourceId}.{metricName}, so schedules
configured per the docs will not be found and the default schedule will be used.

workspaces/scorecard/plugins/scorecard-backend/src/scheduler/tasks/PullMetricsByProviderTask.ts[87-96]
workspaces/scorecard/plugins/scorecard-backend/src/utils/metricProviderConfigKeys.ts[82-117]
workspaces/scorecard/plugins/scorecard-backend/docs/providers.md[162-180]

Agent prompt 
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The scheduling documentation in `workspaces/scorecard/plugins/scorecard-backend/docs/providers.md` still documents the old schedule config path (`scorecard.plugins.{datasourceId}.{metricName}.schedule`). The runtime now resolves schedules via datasource-level and provider-level paths, so users following the docs will have their schedule config ignored.

### Issue Context
- Runtime schedule resolution prefers provider schedule under `scorecard.plugins.<datasource>.metricProviders.<providerName>.schedule`, then falls back to `scorecard.plugins.<datasource>.schedule`, else uses a hardcoded default.
- The existing docs show an outdated path and YAML example.

### Fix Focus Areas
- workspaces/scorecard/plugins/scorecard-backend/docs/providers.md[162-180]
- workspaces/scorecard/plugins/scorecard-backend/src/utils/metricProviderConfigKeys.ts[82-117]
- workspaces/scorecard/plugins/scorecard-backend/src/scheduler/tasks/PullMetricsByProviderTask.ts[87-95]

### Suggested fix
1. Update `docs/providers.md` "Metric Collection Scheduling" section to document:
  - Datasource-level schedule: `scorecard.plugins.<datasource>.schedule`
  - Provider-level override: `scorecard.plugins.<datasource>.metricProviders.<providerName>.schedule`
  - Precedence: provider > datasource > default
2. Update the YAML example accordingly (and remove references to `scorecard.plugins.{datasourceId}.{metricName}.schedule`).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Informational

2. Broken schedule doc link 🐞 Bug ⚙ Maintainability
Description 
The GitHub backend module README links to providers.md#metric-collection-schedulin (missing
trailing 'g'), so the link does not jump to the intended "Metric Collection Scheduling" section.
Code

workspaces/scorecard/plugins/scorecard-backend-module-github/README.md[116]

+The schedule configuration follows Backstage's `SchedulerServiceTaskScheduleDefinitionConfig` [schema](https://github.com/backstage/backstage/blob/master/packages/backend-plugin-api/src/services/definitions/SchedulerService.ts#L157). See [schedule configuration](../scorecard-backend/docs/providers.md#metric-collection-schedulin) for custom schedule configuration.
Relevance

⭐⭐⭐ High

Team regularly fixes broken doc links/anchors; similar anchor-fragment fixes accepted in PR #3417
and link fix in #3526.

PR-#3417
PR-#3526

ⓘ Recommendations generated based on similar findings in past PRs

Evidence 
The GitHub module README contains a misspelled fragment identifier, while the target document has
the correctly spelled section header "Metric Collection Scheduling".

workspaces/scorecard/plugins/scorecard-backend-module-github/README.md[99-116]
workspaces/scorecard/plugins/scorecard-backend/docs/providers.md[162-163]

Agent prompt 
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
A README link in the GitHub backend module uses an incorrect Markdown anchor (`#metric-collection-schedulin`), which breaks navigation to the scheduling documentation.

### Issue Context
The target section header in `providers.md` is `## Metric Collection Scheduling`, which generates an anchor `#metric-collection-scheduling`.

### Fix Focus Areas
- workspaces/scorecard/plugins/scorecard-backend-module-github/README.md[116-116]

### Suggested fix
Change the link fragment from `#metric-collection-schedulin` to `#metric-collection-scheduling`.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Grey Divider

To customize comments, go to the Qodo configuration screen, or learn more in the docs.

Qodo Logo

@rhdh-qodo-merge rhdh-qodo-merge Bot added documentation Improvements or additions to documentation enhancement New feature or request Tests labels Jul 28, 2026
@fullsend-ai-review

fullsend-ai-review Bot commented Jul 28, 2026

Copy link
Copy Markdown

🤖 Finished Review · ❌ Failure · Started 8:36 AM UTC · Completed 9:16 AM UTC
Commit: 477650e · View workflow run →

@dzemanov
Remove unused code
8ea3189 
Signed-off-by: Dominika Zemanovicova <dzemanov@redhat.com>
@fullsend-ai-review

fullsend-ai-review Bot commented Jul 28, 2026 •
edited
Loading

Copy link
Copy Markdown

🤖 Review · ❌ Terminated · Started 9:54 AM UTC · Ended 10:36 AM UTC
Commit: 8ea3189 · View workflow run →

@fullsend-ai-review

fullsend-ai-review Bot commented Jul 28, 2026

Copy link
Copy Markdown

🤖 Finished Review · ❌ Failure · Started 9:54 AM UTC · Completed 10:36 AM UTC
Commit: 8ea3189 · View workflow run →

dzemanov added 2 commits July 28, 2026 17:27
@dzemanov
Rename plugins key to metricProviders and update filecheck provider id
2183cbb 
Assisted-By: Cursor Desktop
Signed-off-by: Dominika Zemanovicova <dzemanov@redhat.com>
@dzemanov
Remove unused provider properties
e5079b2 
Signed-off-by: Dominika Zemanovicova <dzemanov@redhat.com>
@fullsend-ai-review

fullsend-ai-review Bot commented Jul 28, 2026 •
edited
Loading

Copy link
Copy Markdown

🤖 Review · ❌ Terminated · Started 3:42 PM UTC · Ended 4:24 PM UTC
Commit: e5079b2 · View workflow run →

@sonarqubecloud

sonarqubecloud Bot commented Jul 28, 2026

Copy link
Copy Markdown

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

@fullsend-ai-review

fullsend-ai-review Bot commented Jul 28, 2026

Copy link
Copy Markdown

🤖 Finished Review · ❌ Failure · Started 3:42 PM UTC · Completed 4:24 PM UTC
Commit: e5079b2 · View workflow run →

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@christoph-jerolimov christoph-jerolimov Awaiting requested review from christoph-jerolimov christoph-jerolimov is a code owner
@its-mitesh-kumar its-mitesh-kumar Awaiting requested review from its-mitesh-kumar its-mitesh-kumar is a code owner
@Eswaraiahsapram Eswaraiahsapram Awaiting requested review from Eswaraiahsapram Eswaraiahsapram is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

documentation Improvements or additions to documentation enhancement New feature or request Tests workspace/scorecard

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@dzemanov