Skip to content

fix: reduce public content credential fixture false positives#1929

Open
HanShaoshuai-k wants to merge 1 commit into
mainfrom
fix/public-content-credential-fixtures
Open

fix: reduce public content credential fixture false positives#1929
HanShaoshuai-k wants to merge 1 commit into
mainfrom
fix/public-content-credential-fixtures

Conversation

@HanShaoshuai-k

@HanShaoshuai-k HanShaoshuai-k commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

Summary

Reduce false positives in public-content credential detection while preserving detection of real credentials.

Changes

  • Require value evidence instead of blocking on credential-like field names alone.
  • Recognize provider tokens, encoded credentials, high-entropy values, and typed assignments.
  • Allow metadata fields, placeholders, synthetic prefixes, masked values, and security terminology.
  • Preserve review-comment file paths for context-aware scanning.
  • Add regression tests for reported false positives and real credential cases.

Test Plan

  • Qualitygate tests and race tests

Summary by CodeRabbit

  • Bug Fixes

    • Improved public comment credential auditing by analyzing both comment text and its path/context, ensuring findings are attributed to the correct location.
    • Reduced false positives by refining credential-shape, evidence, and parsing heuristics across more assignment syntaxes and encoded/provider formats.
    • Improved detection accuracy for credential assignments embedded in structured inputs (for example, diagrams).
  • Tests

    • Expanded and tightened scanning/auditing coverage with dynamic fixture values and additional edge cases to prevent regressions.

@coderabbitai

coderabbitai Bot commented Jul 17, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 00758d6f-2286-4de1-92c2-5a704e20d873

📥 Commits

Reviewing files that changed from the base of the PR and between 037667f and f6a0c8d.

📒 Files selected for processing (10)
  • internal/qualitygate/cmd/comment-audit/main.go
  • internal/qualitygate/cmd/comment-audit/main_test.go
  • internal/qualitygate/publiccontent/collect_test.go
  • internal/qualitygate/publiccontent/comment_audit.go
  • internal/qualitygate/publiccontent/comment_audit_test.go
  • internal/qualitygate/publiccontent/credential.go
  • internal/qualitygate/publiccontent/rules.go
  • internal/qualitygate/publiccontent/scan.go
  • internal/qualitygate/publiccontent/scan_test.go
  • internal/qualitygate/rules/run_test.go
🚧 Files skipped from review as they are similar to previous changes (9)
  • internal/qualitygate/publiccontent/comment_audit.go
  • internal/qualitygate/publiccontent/comment_audit_test.go
  • internal/qualitygate/cmd/comment-audit/main.go
  • internal/qualitygate/rules/run_test.go
  • internal/qualitygate/publiccontent/credential.go
  • internal/qualitygate/publiccontent/rules.go
  • internal/qualitygate/publiccontent/collect_test.go
  • internal/qualitygate/publiccontent/scan.go
  • internal/qualitygate/publiccontent/scan_test.go

📝 Walkthrough

Walkthrough

The public-content scanner now uses provider-specific credential evidence, refined assignment parsing, and broader benign-value handling. Comment auditing carries review-comment paths into scanning and diagnostics, with expanded tests for detection and fixture behavior.

Changes

Public content auditing

Layer / File(s) Summary
Credential evidence and assignment rules
internal/qualitygate/publiccontent/credential.go, internal/qualitygate/publiccontent/rules.go, internal/qualitygate/publiccontent/scan.go
Provider identifiers, encoding and entropy checks, assignment parsing, typed RHS handling, and benign source-code classification were updated.
Scanner control flow and validation
internal/qualitygate/publiccontent/scan_test.go
Tests cover provider detection, fixture handling, typed assignments, placeholder values, source-code expressions, and generic credential decision matrices.
Comment path propagation
internal/qualitygate/cmd/comment-audit/main.go, internal/qualitygate/publiccontent/comment_audit.go, internal/qualitygate/cmd/comment-audit/main_test.go, internal/qualitygate/publiccontent/comment_audit_test.go
Review-comment paths are extracted from events, passed to ScanCommentAtPath, and preserved in diagnostics.
Collection and fixture validation
internal/qualitygate/publiccontent/collect_test.go, internal/qualitygate/rules/run_test.go, tests/e2e/new-public-workflow.test.sh
Fixtures use generated provider-like values, excerpt checks avoid fixed secrets, and expected findings and allow cases are updated.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant EventFile
  participant CommentAudit
  participant PublicContent
  participant Diagnostics
  EventFile->>CommentAudit: read event body and review-comment path
  CommentAudit->>PublicContent: ScanCommentAtPath(kind, path, body)
  PublicContent->>Diagnostics: return diagnostics with the resolved path
  CommentAudit->>Diagnostics: print diagnostics and exit with report.ExitCode
Loading

Possibly related PRs

Suggested labels: feature

Suggested reviewers: liangshuo-1

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title is concise and accurately reflects the main change: reducing false positives in public content credential detection.
Description check ✅ Passed The description matches the required sections and covers the summary, changes, and test plan.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/public-content-credential-fixtures

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added the size/L Large or sensitive change across domains or core paths label Jul 17, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@internal/qualitygate/publiccontent/scan.go`:
- Around line 62-64: Update sourceCodeTypedCredentialRHS and both its
evidence-extraction and benign-expression classification call sites to accept
and use the current match offset location[0], rather than searching from the
start of the line. Propagate that offset through all affected paths, including
the additional occurrences noted in the review. Add an adjacent regression test
with two identical typed assignments on one line where only the second RHS is
provider-shaped, and verify the second assignment is evaluated correctly.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 31834fb3-2579-4c41-95ba-f650b46bc221

📥 Commits

Reviewing files that changed from the base of the PR and between 4e0a6a9 and 8465147.

📒 Files selected for processing (10)
  • internal/qualitygate/cmd/comment-audit/main.go
  • internal/qualitygate/cmd/comment-audit/main_test.go
  • internal/qualitygate/publiccontent/collect_test.go
  • internal/qualitygate/publiccontent/comment_audit.go
  • internal/qualitygate/publiccontent/comment_audit_test.go
  • internal/qualitygate/publiccontent/credential.go
  • internal/qualitygate/publiccontent/rules.go
  • internal/qualitygate/publiccontent/scan.go
  • internal/qualitygate/publiccontent/scan_test.go
  • internal/qualitygate/rules/run_test.go

Comment thread internal/qualitygate/publiccontent/scan.go
@github-actions

github-actions Bot commented Jul 17, 2026

Copy link
Copy Markdown

🚀 PR Preview Install Guide

🧰 CLI update

npm i -g https://pkg.pr.new/larksuite/cli/@larksuite/cli@f6a0c8d5aa446cceec0bc486797c48c05094ad08

🧩 Skill update

npx skills add larksuite/cli#fix/public-content-credential-fixtures -y -g

@codecov

codecov Bot commented Jul 17, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 88.12500% with 19 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.91%. Comparing base (27df16d) to head (f6a0c8d).

Files with missing lines Patch % Lines
internal/qualitygate/publiccontent/scan.go 86.56% 5 Missing and 4 partials ⚠️
internal/qualitygate/cmd/comment-audit/main.go 53.33% 6 Missing and 1 partial ⚠️
internal/qualitygate/publiccontent/credential.go 96.07% 1 Missing and 1 partial ⚠️
internal/qualitygate/publiccontent/rules.go 95.45% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1929      +/-   ##
==========================================
- Coverage   74.96%   74.91%   -0.06%     
==========================================
  Files         892      893       +1     
  Lines       94077    94162      +85     
==========================================
+ Hits        70529    70544      +15     
- Misses      18137    18200      +63     
- Partials     5411     5418       +7     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@HanShaoshuai-k
HanShaoshuai-k force-pushed the fix/public-content-credential-fixtures branch from 8465147 to 0ee2e94 Compare July 17, 2026 06:51

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@internal/qualitygate/publiccontent/scan_test.go`:
- Around line 417-418: Replace aggregate count-only assertions with individual
classification assertions in internal/qualitygate/publiccontent/scan_test.go at
lines 417-418, 394-395, 597-598, 1150-1163, 1183-1184, and 1656-1657: verify
each benign-key, provider, readable, placeholder, expression, and source-literal
fixture directly as allowed or rejected, then retain combined deduplication
checks where applicable.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: c8e13390-295e-4117-9338-57fe18486bd3

📥 Commits

Reviewing files that changed from the base of the PR and between 8465147 and 0ee2e94.

📒 Files selected for processing (10)
  • internal/qualitygate/cmd/comment-audit/main.go
  • internal/qualitygate/cmd/comment-audit/main_test.go
  • internal/qualitygate/publiccontent/collect_test.go
  • internal/qualitygate/publiccontent/comment_audit.go
  • internal/qualitygate/publiccontent/comment_audit_test.go
  • internal/qualitygate/publiccontent/credential.go
  • internal/qualitygate/publiccontent/rules.go
  • internal/qualitygate/publiccontent/scan.go
  • internal/qualitygate/publiccontent/scan_test.go
  • internal/qualitygate/rules/run_test.go
🚧 Files skipped from review as they are similar to previous changes (7)
  • internal/qualitygate/publiccontent/comment_audit_test.go
  • internal/qualitygate/cmd/comment-audit/main.go
  • internal/qualitygate/publiccontent/credential.go
  • internal/qualitygate/cmd/comment-audit/main_test.go
  • internal/qualitygate/publiccontent/rules.go
  • internal/qualitygate/publiccontent/scan.go
  • internal/qualitygate/publiccontent/collect_test.go

Comment thread internal/qualitygate/publiccontent/scan_test.go Outdated
@HanShaoshuai-k
HanShaoshuai-k force-pushed the fix/public-content-credential-fixtures branch from 0ee2e94 to 037667f Compare July 17, 2026 07:03
@github-actions

github-actions Bot commented Jul 17, 2026

Copy link
Copy Markdown

PR Quality Summary

CI did not complete successfully. Use the failed check links below to decide whether this PR needs a code change or a rerun.

Failed checks

@HanShaoshuai-k
HanShaoshuai-k force-pushed the fix/public-content-credential-fixtures branch from 037667f to f6a0c8d Compare July 17, 2026 08:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/L Large or sensitive change across domains or core paths

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant