Skip to content

fix(scoring): apply 1.3x multiplier only to findings from executable files#122

Open
tcconnally wants to merge 1 commit into
NVIDIA:mainfrom
tcconnally:fix/risk-score-per-file-weight
Open

fix(scoring): apply 1.3x multiplier only to findings from executable files#122
tcconnally wants to merge 1 commit into
NVIDIA:mainfrom
tcconnally:fix/risk-score-per-file-weight

Conversation

@tcconnally

Copy link
Copy Markdown

Summary

Addresses the scoring concern in #103: the 1.3x executable-script multiplier was applied globally to ALL findings when any executable file existed. This meant documentation, teaching content, deny-lists, and anti-examples in .md files received the same score inflation as actual executable code — making thoroughly documented security skills appear MORE dangerous.

Fix

_compute_risk_score() now accepts optional component_metadata and builds a file-path lookup. The 1.3x multiplier is applied only to findings whose file is marked as executable in the component metadata. Findings from markdown, text, json, yaml, and toml files receive base severity weight (no multiplier).

Before

score = sum(severity_weights)
if has_executable_scripts:
    score = int(score * 1.3)  # inflates ALL findings

After

for each finding:
    weight = severity_weight
    if has_executable_scripts AND finding's file is executable:
        weight = int(weight * 1.3)  # only executable-file findings
    score += weight

Testing

  • 622 tests pass (all existing + new test_report_doc_findings_no_multiplier)
  • Backward compatible: when component_metadata is None, behavior is unchanged

…files

Previously the risk score multiplied ALL findings by 1.3x if any
executable script existed in the skill, punishing documentation
and teaching content that mentions attack patterns for educational
or deny-list purposes.

This change:
- Builds a file-executable lookup from component_metadata
- Applies 1.3x only to findings from executable files (.py, .sh, .js, etc.)
- Leaves non-executable-file findings at base severity weight
- No change when component_metadata is unavailable (backward compatible)

Added test: doc_findings_no_multiplier verifies markdown findings
are not inflated by the executable multiplier.

Addresses the scoring concern in NVIDIA#103

Signed-off-by: Perseus Computing <51974392+tcconnally@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant