Skip to content

feat: add DSH plugin scanning and runtime protection - #116

Merged
Mr-Lucky merged 40 commits into
GoPlusSecurity:mainfrom
EchoOfZion:codex/dsh-mvp
Aug 19, 2026
Merged

feat: add DSH plugin scanning and runtime protection#116
Mr-Lucky merged 40 commits into
GoPlusSecurity:mainfrom
EchoOfZion:codex/dsh-mvp

Conversation

@EchoOfZion

@EchoOfZion EchoOfZion commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

Summary

Add complete AgentGuard support for DeepSeek Harness (DSH): installation-time plugin scanning plus an observation-first, opt-in DSH-native runtime guard.

The scanner evaluates local directories or pinned HTTPS GitHub repositories without installing or executing the target. The runtime integration reuses AgentGuard's shared policy evaluator and connects to DSH's native tool lifecycle, approval service, and audit model. The packaged composition remains non-disruptive in observe; operators explicitly select protect when they want enforcement.

DSH interfaces

CLI:

agentguard dsh-scan ./path/to/plugin
agentguard dsh-scan https://github.com/owner/plugin --ref v1.2.3 --format json
agentguard dsh-scan-batch manifest.json
agentguard dsh-compare before.json after.json

Native DSH tools:

  • agentguard_dsh_scan
  • agentguard_dsh_scan_batch
  • agentguard_dsh_compare
  • agentguard_dsh_runtime_summary

Native installation:

dsh plugin --profile web add @goplus/agentguard

Installation-time scanning

  • Detects DSH bundles, profiles, client extensions, active instructions, tools, providers, UI/session/storage integrations, and Cordis composition.
  • Parses Cordis YAML with bounded AST traversal while keeping !!js values inert.
  • Reports conservative full-repository risk separately from installed runtime-surface risk.
  • Classifies evidence as runtime, instruction, generated, documentation, test, example, or data context.
  • Produces explainable capability profiles, impact layers, review priority, installation posture, diagnostics, hashes, and Git revision metadata.
  • Aggregates repeated findings by rule and file without reducing severity.
  • Resolves GitHub HEAD/ref first, fetches the exact commit without hooks or submodules, verifies local HEAD, and removes the temporary checkout.

Incomplete-scan fail-closed behavior (phase1-rc3)

  • Records structured scanCoverage: discovered, scanned, skipped, completeness, and stable skip reasons (fileLimit, oversized, unreadable).
  • Reuses one immutable file snapshot across DSH detection, capability profiling, classification, hashing, and security rules.
  • Any file-count truncation, oversized scannable file, or ordinary read failure emits DSH_SCAN_INCOMPLETE.
  • An incomplete scan forces both repository and runtime-surface posture to at least HIGH, review priority HIGH, and expert-review-required; it can never return safe-to-try.
  • Coverage is visible in JSON, Markdown, HTML, native single-scan summaries, and batch summaries.
  • Boundary tests cover file limits, oversized source, injected read failure, normal complete scans, and the existing unsafe-symlink fail-closed path.

The pinned real-world benchmark now exercises the bug directly: dsh-deep-whale contains a 2.7 MiB runtime asset. It previously appeared LOW at runtime after that file was skipped; it now reports DSH_SCAN_INCOMPLETE, HIGH, and expert review.

The scanner never installs the target, runs lifecycle scripts, or evaluates package/configuration code.

DSH runtime guard

  • Connects to native tools/pre-execute and tools/post-execute waterfalls.
  • Normalizes DSH shell, file, network, browser, deployment, skill-install, and MCP calls into AgentGuard's shared RuntimeAction model.
  • Uses the same policy resolver and OSS evaluator as other AgentGuard hosts.
  • Maps allow/warn/require_approval/block monotonically to DSH allow/ask/deny, preserving stronger downstream policies.
  • Delegates one-shot approval entirely to DSH's native approval service; no duplicate AgentGuard approval queue is created.
  • Fails closed on unexpected pre-execute evaluation errors by default, with an explicit compatibility override.
  • Supports exact operator-configured tool-owner attribution and monotonic per-owner minimum decisions. Unmapped tools remain explicitly unknown.
  • Writes bounded local audit events and exposes an aggregate-only runtime summary that omits raw tool input and evidence.
  • Excludes AgentGuard's own tools from recursive protection.

Runtime posture visibility

  • Startup logs explicitly say off, observe (audit only; enforcement inactive), or protect (pre-execute enforcement active), including the configured post-response mode.
  • agentguard_dsh_runtime_summary now returns configuredMode, preExecuteProtectionActive, and configuredPostResponseMode.
  • The summary distinguishes current configuration from historical audit counts, avoiding the impression that observe is active protection.

The npm bundle composes observe by default. A profile explicitly opts into enforcement:

- id: agentguard-dsh-plugin
  config:
    runtime:
      mode: protect
      failureMode: deny
      postResponseMode: block-malicious

Acceptance result

The installed complete candidate passed all 11 guided DSH UAT cases: four-tool availability, single and batch scans, report comparison, safe allow, native allow-once approval, rejection with zero execution, hard block with zero execution, redacted runtime summary, malicious-response containment, and Web service stability.

Latest validation for phase1-rc3:

  • npm run build — passed
  • npm test531 passed, 0 failed
  • npm run test:dsh-e2e — passed
  • npm run test:dsh-protect — passed
  • npm run test:dsh-approval — passed
  • npm run test:dsh-post-enforcement — passed
  • npm run test:dsh-lifecycle — passed
  • npm run test:dsh-package — passed
  • npm run benchmark:dsh5 exact-commit public-plugin cases stable
  • git diff --check — passed
  • Guided installed DSH UAT — 11/11 passed
  • Focused installed phase1-rc3 UAT — 6/6 passed on 2026-08-19; verified complete-scan compatibility, oversized-file fail-closed behavior, batch propagation, protect/observe visibility, restoration to protect, and HTTP 200

Intentional boundaries

  • Static reports are installation decision aids, not safety certificates.
  • The scanner does not yet prove npm artifact/source equivalence or marketplace reputation.
  • DSH lifecycle events do not currently expose a reliable native source-plugin identity. Exact operator bindings are supported; unmapped calls stay unknown.
  • Approval-class post results cannot be safely held and resumed with the current DSH protocol, so they remain audit-only. Explicit block-class results can be suppressed.
  • Static detection remains primarily pattern based; AST/data-flow analysis is a future enhancement.
  • Publication, automatic target installation, team policy, badges, and Cloud history are outside this PR.

Documentation

  • docs/dsh.md — architecture, scanner/report contract, operations, and limitations
  • docs/dsh-runtime.md — runtime modes, lifecycle mapping, audit, attribution, and containment boundary
  • docs/dsh-complete-candidate.md — complete candidate contract and acceptance gates
  • docs/dsh-user-acceptance-test.zh-CN.md — safe 11-case guided acceptance plan
  • docs/dsh-phase1-rc3-acceptance-test.zh-CN.md — focused incomplete-scan and runtime-mode visibility acceptance plan
  • docs/dsh-phase1-rc3-acceptance-result.zh-CN.md — completed 6/6 installed acceptance evidence
  • docs/dsh-phase1-rc.md — frozen static scanner baselines
  • benchmarks/dsh/README.md — exact-commit benchmark policy
  • benchmarks/dsh/manual-reviews.md — source-level benchmark qualifications

@github-actions

github-actions Bot commented Aug 15, 2026

Copy link
Copy Markdown

AgentGuard PR Review

This patch introduces a few concrete regressions and safety gaps in the new DSH scanner path.

  1. high — src/dsh/scan.ts / src/cli.ts

    • What can go wrong: dsh-scan always exits 0 for both high and medium risk reports, and only returns 2 for critical. This makes it easy for CI or scripted install gates to treat clearly dangerous plugins as passing scans, especially since the docs advertise the command as an installation-time trust layer.
    • Fix: Add an explicit non-zero exit code for high risk (or a --fail-on <level> option), and make the default behavior fail for at least high/critical when used in automation.
  2. medium — src/dsh/scan.ts (installation recommendation is based on filtered evidence, but riskLevel is computed after mutating riskTags)

    • What can go wrong: harmlessMismatch adds DSH_THEME_ELEVATED_CAPABILITY into riskTags and then recalculates risk, which can escalate UI/theme plugins to high even when the only finding is a benign-looking label mismatch. This can create false positives and break the intended “safe to try”/“isolated profile” guidance.
    • Fix: Keep mismatch as a separate finding or tag that does not affect base risk classification unless there is a real elevated capability finding.
  3. high — src/dsh/parse-cordis-patch.ts

    • What can go wrong: parseDocument(raw, { schema: 'failsafe', strict: false }) is not a valid/portable yaml v2 API usage for disabling !!js execution in the way the docs claim. If this parses with a non-failsafe schema or falls back unexpectedly, malicious Cordis YAML tags could still be interpreted incorrectly or parsing could silently degrade.
    • Fix: Use the library’s explicit safe schema API for v2 and add a regression test proving !!js tags remain inert as plain data; fail closed on unsupported tags.
  4. medium — src/dsh/source.ts

    • What can go wrong: GitHub cloning only validates the input with a regex and then passes the original URL to git clone. Because the source is untrusted, a crafted repository URL variant that matches the regex but behaves unexpectedly in Git could still be processed. More importantly, clone errors are wrapped but not differentiated, making it hard to enforce safe policy in callers.
    • Fix: Normalize and strictly parse the GitHub URL into owner/repo components, reconstruct the clone URL from those components, and surface structured clone failure reasons.
  5. medium — src/dsh/scan.ts / src/dsh/classify-plugin.ts

    • What can go wrong: The “harmless-looking theme/UI” mismatch heuristic treats labels as benign based on name/description only, while the plugin kind can be inferred from content. A malicious package can evade the mismatch warning simply by omitting those words, and a harmless package with a generic name can get over-penalized.
    • Fix: Base mismatch detection on stronger manifest signals (dsh.client, Cordis rows, package metadata) and do not use it as a risk escalator; keep it as an auxiliary warning.

@EchoOfZion

Copy link
Copy Markdown
Contributor Author

Addressed all four review items in 5413eca.

  1. Test-like path under-reporting — fixed. Removed development-path filtering from both capability inference and evidence/risk calculation. Dangerous code under tests, fixtures, *.spec.*, or *.test.* now affects capabilities, findings, risk, and the installation recommendation. The regression test now requires a test-path shell call to produce high risk.

  2. Cordis YAML API — verified and hardened. yaml@2.9.0 does accept SchemaOptions.schema: string, but the old failsafe schema parsed ordinary booleans as strings. The parser now uses the documented core schema with an explicit tag:yaml.org,2002:js scalar resolver that returns inert text. This preserves !!js without evaluation while retaining normal boolean/number semantics. Added a representative cordis.yml test with disabled: false, numeric config, and !!js.

  3. GitHub revision pinning — fixed. The resolver now canonicalizes the validated owner/repository URL, resolves remote HEAD first, fetches that exact SHA at depth 1 into an initialized temporary repository, checks it out detached, and verifies local HEAD before scanning. A live GitHub scan confirmed the resolved and reported revisions match.

  4. Nested output path — fixed. --output now creates missing parent directories explicitly. Added a CLI regression test that writes JSON into a nested, previously absent directory.

Validation after the fixes:

  • npm run build
  • npm test: 428 passed, 0 failed
  • Focused DSH tests: 13 passed, 0 failed
  • Live GitHub exact-SHA DSH scan
  • npm pack --dry-run
  • git diff --check

@EchoOfZion

Copy link
Copy Markdown
Contributor Author

Additional hardening from the updated review is in 9efa5a1:

  • Cordis resource bounds: removed whole-document toJS() materialization. The parser validates the YAML AST iteratively, caps it at 20,000 nodes and 64 levels, and reads only the map/sequence/scalar fields needed for Cordis rows. Malformed YAML remains rejected under strict: true. Added a deep-nesting rejection test.
  • Install instructions metadata: documented in the public report type and schema guide that project.hasInstallInstructions is informational README metadata only. It is not consumed by risk calculation or installation recommendations.
  • Git dependency diagnostics: GitHub scans now check git --version first and return a specific missing-PATH error. Local scans remain available without Git metadata when Git is unavailable. The exact-SHA fetch and checkout verification from 5413eca remains in place.
  • Theme mismatch summary: added and prioritized DSH_THEME_ELEVATED_CAPABILITY in the summary reason list, with a regression assertion.

Latest validation:

  • npm run build
  • npm test: 429 passed, 0 failed
  • Focused DSH tests: 14 passed, 0 failed
  • git diff --check

@EchoOfZion

Copy link
Copy Markdown
Contributor Author

The latest review items are addressed in a15e4c6:

  • Theme mismatch precision: network-only UI/theme behavior now remains a normal medium network finding. The derived mismatch is limited to shell execution, file writes, runtime mutation, or the environment-plus-network exfiltration combination. Added a network-only regression test.
  • Derived mismatch evidence: the DSH_THEME_ELEVATED_CAPABILITY finding now carries an explicit message and snippet containing the package identity and exact unexpected capability names. The summary prioritizes the same reason.
  • Cordis structural validation: top-level rows, inserted rows, and nested config.patches must have the expected sequence/mapping shapes. Invalid structures produce diagnostics and do not return a partial row set. Added a malformed-row regression test.
  • GitHub URL contract: the normalizer and docs now explicitly support https://github.com/owner/repo, optional .git, and one trailing slash, while rejecting repository subpaths. Added unit coverage for all forms.
  • README metadata naming: renamed the report field to hasReadmeInstallInstructions, broadened common README variants, and documented that it is informational only and never affects risk or recommendations.

Final validation for this revision:

  • npm run build
  • npm test: 432 passed, 0 failed
  • Focused DSH tests: 17 passed, 0 failed
  • git diff --check

@EchoOfZion

Copy link
Copy Markdown
Contributor Author

Final deterministic improvements are in ee9e156:

  • Malformed or oversized package.json now produces diagnostics.packageParseError without aborting the scan. Missing manifests remain a valid no-manifest case. Added regression coverage.
  • DSH_PATCH_OVERRIDE is no longer accepted or rejected by parsing a free-form scanner snippet. Findings and risk tags are built directly from parsed Cordis rows with operation === "replace" and a security-relevant structured row ID.

Two review statements refer to code that is no longer present: the current parser does not call document.toJS(), and GitHub scans do not call git clone; they resolve HEAD and fetch the exact SHA into a detached checkout. Risk remains evidence/rule-driven rather than being raised directly from the coarser capability profile, because doing the latter would make benign descriptive strings security findings. Remote scanning is already explicit in the dsh-scan <repo-or-path> argument and documented as a static fetch; adding an interactive prompt would break CI and JSON automation.

Validation at ee9e156:

  • npm run build
  • npm test: 433 passed, 0 failed
  • Focused DSH tests: 18 passed, 0 failed
  • git diff --check

@EchoOfZion

Copy link
Copy Markdown
Contributor Author

Phase 1 RC stabilization is complete in a6e1edc.

  • Risk semantics are frozen at 83db977 (phase1-rc1); the latest commit adds regression infrastructure and documentation, not sample-specific retuning.
  • Full suite: 466 passed, 0 failed.
  • Native DSH E2E: profile composition, loopback boot, and installed agentguard_dsh_scan execution passed.
  • Exact-commit real-world benchmark: 5/5 stable across LOW-through-CRITICAL runtime postures.
  • Added deterministic snapshots, field-level diff failure, snapshot update policy, and offline validation of benchmark assets.
  • Added source-level reviews for dsh-vision-router, MisakaNet, dsh-open-in-vscode, and superdesign-skill, including documented evidence qualifications and known false-positive examples without reproducing credential values.
  • Current AgentGuard PR Review check passes and there are no unresolved inline comments.

The PR body now contains the complete Phase 1 boundary, architecture, calibration history, security properties, acceptance gates, benchmark results, and known limitations. Runtime enforcement remains explicitly deferred to Phase 2.

@EchoOfZion

Copy link
Copy Markdown
Contributor Author

Phase 1 RC delivery follow-up is ready in 840a5f1:

  • adds scanner provenance to JSON, Markdown, HTML, and the DSH tool result (version, phase1-rc1, frozen rules baseline 83db977a566d8a853568a2d2903b142106d80196)
  • keeps schema-v1 backward compatibility and labels provenance as unavailable when rendering older reports
  • documents install/update/remove, profile verification, report interpretation, and ERR_CONNECTION_REFUSED troubleshooting
  • adds a clean temporary-profile lifecycle smoke test covering initialize → link install → Cordis composition → scan → uninstall

Validation after the change:

  • npm test: 466/466 passed
  • npm run test:dsh-lifecycle: passed
  • npm run test:dsh-e2e: passed (HTTP 200 and scanner tool execution)
  • npm run benchmark:dsh: all 5 pinned real-world cases stable

No Phase 1 detection rule or risk threshold changed in this follow-up.

@EchoOfZion

Copy link
Copy Markdown
Contributor Author

Release-artifact validation follow-up is ready in 55224ee.

The new npm run test:dsh-package gate builds the exact npm tarball and validates it in a clean temporary DSH profile: required runtime/type/Cordis/docs assets → tarball install → Cordis composition → scanner execution → package update → uninstall. It never publishes the package.

The first run caught that files: ["dist"] shipped all compiled tests. The package now explicitly excludes dist/tests:

  • entries: 457 → 353
  • packed size: 492,102 → 389,842 bytes
  • unpacked size: 2,451,003 → 1,636,026 bytes

Final validation:

  • npm test: 466/466 passed
  • clean link-profile lifecycle: passed
  • exact npm tarball lifecycle: passed
  • DSH runtime E2E: HTTP 200 and scanner execution passed
  • 5 pinned real-world benchmarks: stable
  • git diff --check: passed

No scanner rule, severity, recommendation, or Phase 1 boundary changed.

@Mr-Lucky

Copy link
Copy Markdown
Contributor

New findings:

  1. Remote repository resource exhaustion — High
    [High] Add repository-level resource limits before git fetch

This path accepts an attacker-controlled GitHub repository and performs a shallow fetch and checkout, but there is no limit on repository size, Git object count, disk usage, or total downloaded bytes. The later file-count and file-size limits only apply after checkout has completed, so a large repository can exhaust network, disk, memory, or CPU before scanning starts.

Please add acquisition-level limits, or use a bounded partial/sparse checkout, and abort safely when the repository exceeds the configured budget.

  1. Symlink escape — High
    [High] Do not follow symlinks outside the scan root

glob() can return symlinked files, and the subsequent stat() / readFile() calls follow those symlinks. A malicious GitHub repository could include a symlink such as leak.js pointing outside the checkout, causing the scanner to read local files that are not part of the repository. The contents may then appear in findings or reports.

Please use lstat() and reject symlinks, or resolve the real path and verify that it remains within the scan root before reading.

  1. Markdown report prompt injection — Medium/High
    [Medium/High] Escape all artifact-controlled values in Markdown output

The Markdown renderer only escapes | and newlines, while the package name, description, repository metadata, and detection signals can be controlled by the scanned repository. These values are returned through the DSH tool as model-visible text, so a malicious package could inject Markdown/HTML or prompt-injection instructions into the scan result.

The HTML renderer escapes these fields, but the Markdown and JSON/tool-output paths still expose them as trusted-looking text. Please escape all untrusted fields and clearly mark artifact content as untrusted data, or return structured fields instead of embedding them into a natural-language report.

  1. Cordis parse failures can cause false negatives — Medium/High
    [Medium/High] Fail closed when Cordis parsing fails

When Cordis parsing fails or the file exceeds the parser limits, parseCordisConfigs() returns no rows and records only a diagnostic. Later, scanDshPlugin() removes regex-based DSH_PATCH_OVERRIDE findings and calculates risk from successfully parsed rows. As a result, a malformed or unsupported Cordis patch could hide a security-relevant core-row replacement while still producing a low-risk recommendation.

Please make parse errors and unsupported/truncated Cordis structures raise the review status or minimum risk level. The scanner should not return safe-to-try when security-relevant configuration was not successfully understood.

  1. Insufficient dsh.client schema validation — Medium
    [Medium] Validate dsh.client instead of using truthiness

hasClientExtension is currently derived from Boolean(client). This means any truthy object, including { "dsh": { "client": {} } }, is treated as a valid DSH client extension. That can skew plugin classification, capability inference, and risk recommendations.

Please validate the expected dsh.client schema and only set hasClientExtension when the required fields are present and valid. Invalid metadata should produce an explicit diagnostic.
Overall review comment
Overall, the feature direction is reasonable, but I do not recommend merging until the scanner has bounded remote acquisition, rejects symlink escapes, and fails closed on unsupported or malformed security-relevant metadata.

These issues affect the scanner itself rather than the scanned plugin, and could lead to resource exhaustion, local file disclosure, prompt injection, or false-negative security recommendations.

@EchoOfZion

Copy link
Copy Markdown
Contributor Author

Addressed the five new scanner-self-protection findings in 367227c; c1231fc records the resulting phase1-rc2 provenance.

  1. Remote acquisition bounds

    • GitHub scans now use a depth-one, blob-less fetch.
    • Fetch and checkout are monitored against a 256 MiB on-disk budget and are killed on overflow.
    • Git object count is checked before and after checkout with a 100,000-object limit.
    • Existing timeouts, exact-HEAD resolution, detached checkout verification, disabled hooks, and no-submodule behavior remain.
  2. Symlink containment

    • Every scanner read now resolves the real path and verifies it remains inside the real scan root.
    • A symlink escaping the root aborts the scan; an in-repository symlink remains supported (the pinned superdesign-skill case uses one).
    • Package, Cordis, README, capability, rule-scan, and artifact-hash reads all use the same containment helper.
  3. Markdown / model-output trust boundary

    • Artifact-controlled metadata is removed from the Markdown title and rendered as JSON-escaped quoted data under an explicit untrusted-data boundary. Finding paths are JSON-quoted and Markdown-escaped.
    • The native DSH tool renderer no longer returns the target-controlled detailed report as model-visible prose. It renders a scanner-generated decision summary containing only enum decisions plus a warning that detailed content is untrusted. Raw Markdown/JSON remains output data for explicit consumers.
  4. Cordis fail closed

    • Malformed, oversized, unsupported, or unreadable Cordis/package metadata produces the new high-severity DSH_SCAN_INCOMPLETE finding.
    • Both risk views are at least HIGH, review priority is HIGH, and both recommendations are expert-review-required; safe-to-try is impossible when security metadata is incomplete.
  5. dsh.client validation

    • A client extension now requires a non-empty platform string and, when present, a string-array inject field. Invalid truthy objects are not classified as clients and enter the fail-closed path with an explicit diagnostic.

Validation:

  • full suite: 470 passed, 0 failed
  • clean link-profile lifecycle: passed
  • exact npm tarball install/update/remove lifecycle: passed
  • native DSH HTTP E2E: passed
  • real GitHub URL scan using the bounded acquisition path: passed
  • 5/5 exact-commit real-world benchmark cases: stable (no outcome changes for successfully parsed artifacts)
  • git diff --check: passed

The security baseline is now phase1-rc2 at 367227cc2b8bc064af369bf41e4490f6c4d3ea8b.

@EchoOfZion EchoOfZion changed the title feat: add DSH plugin security scanner feat: add DSH plugin scanning and runtime protection Aug 18, 2026
@Mr-Lucky
Mr-Lucky merged commit d4a49b8 into GoPlusSecurity:main Aug 19, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants