Skip to content

ai-integrations AGENTS.md: add processor-behavior verification instructions for doc tasks #3936

Description

@fullsend-ai-retro

What happened

The code agent produced PR #3859 documenting AIResource entity registration and schema fields. Two major accuracy issues required a fix cycle:

  1. Schema table marked fields as "Required" that the processors (AIResourceExtensionsProcessor, AIResourceOciProcessor) don't actually enforce — they only validate spec.scope enum values and oci:// URI format (human review).
  2. Claimed UrlReaderProcessor uses backstage.io/source-location for ingestion — this is inaccurate; the annotation is source metadata, not what drives entity ingestion.

The code agent's workspace AGENTS.md mentions checking openspec/changes/ specs but has no instruction to verify behavioral claims against actual processor source code. The fix agent (run) addressed all findings correctly in one iteration, confirming the processor source code is readable and checkable.

What could go better

The code agent wrote documentation about processor behavior without reading the processor source files to verify its claims. The workspaces/ai-integrations/AGENTS.md file directs agents to check openspec/changes/ for specs but doesn't instruct them to cross-reference documentation claims against the actual plugin source code under plugins/catalog-backend-module-*/src/.

This is a high-confidence assessment: the fix agent demonstrated that reading the processor source and correcting the claims was straightforward. The workspace AGENTS.md is the natural place for this guidance because it's already the entry point agents read before working in this workspace.

This complements but does not duplicate #3745 (changeset/doc-update instructions) — that issue covers mechanical PR conventions, not behavioral accuracy verification.

Proposed change

Add a "Documentation Tasks" section to workspaces/ai-integrations/AGENTS.md with instructions like:

## Documentation Tasks

- When documenting processor behavior (validation rules, required fields,
  ingestion behavior, error handling), read the relevant processor source
  files under `plugins/catalog-backend-module-*/src/` to verify claims.
  Do not infer behavior from field names or conventions alone.
- Cross-reference schema field documentation against the actual processor
  code to confirm which fields are validated/enforced vs. merely consumed.
- When writing examples, ensure field values (e.g., `spec.type`,
  `spec.scope`) are consistent with the asset description in the example.

This is repo-specific guidance that complements the platform-level improvements tracked in fullsend-ai/fullsend #3497 and #2199.

Validation criteria

The next 3 documentation PRs in the ai-integrations workspace produced by the code agent should not require human review feedback about inaccurate processor behavior claims. Specifically: documentation about what fields are required/enforced, what processors do, and what side effects they have should match the actual source code.


Generated by retro agent from #3859

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions