Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions .github/skills/github-mcp-server/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -164,8 +164,9 @@ This section maps individual tools to their respective toolsets to help with mig

When invoking `list_pull_requests` from workflow prompts/templates:
- Default to a small page size (`perPage: 10` unless a smaller/larger value is justified).
- Request `minimal_output: true` when the installed MCP server exposes that input (minimal output trims non-essential nested fields such as large `head`/`base` payloads).
- Confirm parameter support in the method schema from `mcp list-tools` or the tool docs for your server version; if `minimal_output` is unavailable, rely on small `perPage` values.
- On GitHub MCP server ≥ 1.6.0 with the `fields_param` feature (enabled automatically in Insiders mode), pass `fields: [number, title, state, html_url]` (or whichever top-level fields you need) to reduce response size. The same `fields` parameter is available on `list_issues`, `search_issues`, `search_pull_requests`, `list_commits`, `list_releases`, `search_code`, and `get_file_contents`.
- On older servers, request `minimal_output: true` when the installed MCP server exposes that input (minimal output trims non-essential nested fields such as large `head`/`base` payloads).
- Confirm parameter support in the method schema from `mcp list-tools` or the tool docs for your server version.

### Actions Toolset
- `list_workflows` - List GitHub Actions workflows
Expand All @@ -178,7 +179,7 @@ When invoking `list_pull_requests` from workflow prompts/templates:
- `get_code_scanning_alert` - Get details of a specific alert
- `create_code_scanning_alert` - Create a code scanning alert

When invoking `list_code_scanning_alerts` from workflow prompts/templates, always include `state: open` and `severity: critical,high`.
When invoking `list_code_scanning_alerts` from workflow prompts/templates, always include `state: open` and `severity: critical,high` to bound the response size and avoid oversized payloads.

### Discussions Toolset
- `list_discussions` - List discussions in a repository
Expand Down
4 changes: 4 additions & 0 deletions actions/setup/md/github_mcp_tools_prompt.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,8 @@
The GitHub MCP server is read-only. Use GitHub MCP tools for all GitHub reads: listing and searching issues, pull requests, discussions, labels, milestones; reading workflow runs, jobs, and artifacts; accessing repository contents, code, and metadata. Do not use shell `gh` commands for GitHub API reads — `gh` is not authenticated.

**Identity**: Do not call `get_me` — it returns 403 under the integration token. Read your identity (actor, repository, run ID) from the `<github-context>` block provided at the start of the prompt.

**Code scanning queries**: When calling `list_code_scanning_alerts`, always include `state: open` and `severity: critical,high` to bound the response size and avoid oversized payloads.

**Field selection (GitHub MCP server ≥ 1.6.0)**: When calling list/search tools that accept a `fields` parameter (`list_pull_requests`, `list_issues`, `search_issues`, `search_pull_requests`, `list_commits`, `list_releases`, `search_code`, `get_file_contents`), pass only the fields you need to reduce response size. Example: `list_pull_requests` with `fields: [number, title, state, html_url]`.
</github-mcp-tools>
4 changes: 4 additions & 0 deletions actions/setup/md/github_mcp_tools_with_safeoutputs_prompt.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,8 @@
The GitHub MCP server is read-only. Use GitHub MCP tools for all GitHub reads: listing and searching issues, pull requests, discussions, labels, milestones; reading workflow runs, jobs, and artifacts; accessing repository contents, code, and metadata. Do not use shell `gh` commands for GitHub API reads — `gh` is not authenticated. Use safeoutputs tools for GitHub writes and completion signaling.

**Identity**: Do not call `get_me` — it returns 403 under the integration token. Read your identity (actor, repository, run ID) from the `<github-context>` block provided at the start of the prompt.

**Code scanning queries**: When calling `list_code_scanning_alerts`, always include `state: open` and `severity: critical,high` to bound the response size and avoid oversized payloads.

**Field selection (GitHub MCP server ≥ 1.6.0)**: When calling list/search tools that accept a `fields` parameter (`list_pull_requests`, `list_issues`, `search_issues`, `search_pull_requests`, `list_commits`, `list_releases`, `search_code`, `get_file_contents`), pass only the fields you need to reduce response size. Example: `list_pull_requests` with `fields: [number, title, state, html_url]`.
</github-mcp-tools>
108 changes: 108 additions & 0 deletions pkg/workflow/prompt_validation_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,114 @@ import (
"github.com/stretchr/testify/require"
)

// TestGitHubMCPToolsPromptBoundsCodeScanningAlertQueries verifies that the
// github_mcp_tools_prompt.md file includes the required guardrail instructing
// agents to always bound list_code_scanning_alerts calls with state: open and
// severity: critical,high. Unbounded queries produce oversized responses that
// break downstream workflow runs.
func TestGitHubMCPToolsPromptBoundsCodeScanningAlertQueries(t *testing.T) {
promptPath := filepath.Join("..", "..", "actions", "setup", "md", "github_mcp_tools_prompt.md")
data, err := os.ReadFile(promptPath)
require.NoError(t, err, "should be able to read github_mcp_tools_prompt.md")

content := string(data)

assert.Contains(t, content, "list_code_scanning_alerts",
"prompt should mention list_code_scanning_alerts to set agent expectations")
assert.Contains(t, content, "state: open",
"prompt should require state: open to bound list_code_scanning_alerts queries")
assert.Contains(t, content, "severity: critical,high",
"prompt should require severity: critical,high to bound list_code_scanning_alerts queries")
Comment on lines +31 to +32
}

// TestGitHubMCPToolsWithSafeOutputsPromptBoundsCodeScanningAlertQueries verifies that
// the github_mcp_tools_with_safeoutputs_prompt.md file includes the same guardrail.
func TestGitHubMCPToolsWithSafeOutputsPromptBoundsCodeScanningAlertQueries(t *testing.T) {
promptPath := filepath.Join("..", "..", "actions", "setup", "md", "github_mcp_tools_with_safeoutputs_prompt.md")
data, err := os.ReadFile(promptPath)
require.NoError(t, err, "should be able to read github_mcp_tools_with_safeoutputs_prompt.md")

content := string(data)

assert.Contains(t, content, "list_code_scanning_alerts",
"prompt should mention list_code_scanning_alerts to set agent expectations")
assert.Contains(t, content, "state: open",
"prompt should require state: open to bound list_code_scanning_alerts queries")
assert.Contains(t, content, "severity: critical,high",
"prompt should require severity: critical,high to bound list_code_scanning_alerts queries")
}

// TestGitHubMCPToolsPromptHasFieldSelectionGuidance verifies that both prompt files
// contain guidance about the fields parameter introduced in GitHub MCP server 1.6.0.
// Field selection allows agents to request only the fields they need, significantly
// reducing response size for list/search tools.
func TestGitHubMCPToolsPromptHasFieldSelectionGuidance(t *testing.T) {
promptFiles := []string{
filepath.Join("..", "..", "actions", "setup", "md", "github_mcp_tools_prompt.md"),
filepath.Join("..", "..", "actions", "setup", "md", "github_mcp_tools_with_safeoutputs_prompt.md"),
}

for _, promptPath := range promptFiles {
t.Run(filepath.Base(promptPath), func(t *testing.T) {
data, err := os.ReadFile(promptPath)
require.NoError(t, err, "should be able to read %s", promptPath)

content := string(data)

assert.Contains(t, content, "fields",
"prompt should mention fields parameter for response size reduction")
assert.Contains(t, content, "list_pull_requests",
"prompt should reference list_pull_requests as a tool that supports fields")
assert.Contains(t, content, "1.6.0",
"prompt should reference the GitHub MCP server version that introduced fields")
})
}
}

// TestGitHubMCPToolsPromptIncludedForCodeSecurityToolset verifies that when a
// workflow uses the code_security GitHub toolset the generated lock file references
// one of the github_mcp_tools prompt files (which carry the list_code_scanning_alerts
// guardrail). This is a regression test for unbounded alert queries.
func TestGitHubMCPToolsPromptIncludedForCodeSecurityToolset(t *testing.T) {
tmpDir, err := os.MkdirTemp("", "gh-aw-code-scanning-prompt-test-*")
require.NoError(t, err)
defer os.RemoveAll(tmpDir)

testFile := filepath.Join(tmpDir, "test-workflow.md")
testContent := `---
on: push
engine: claude
permissions:
security-events: read
tools:
github:
toolsets: [code_security]
---

# Test Workflow with Code Security Toolset

List open critical code scanning alerts.
`
require.NoError(t, os.WriteFile(testFile, []byte(testContent), 0644))

compiler := NewCompiler()
require.NoError(t, compiler.CompileWorkflow(testFile))

lockFile := strings.TrimSuffix(testFile, ".md") + ".lock.yml"
lockContent, err := os.ReadFile(lockFile)
require.NoError(t, err)

lockStr := string(lockContent)

// The generated workflow must reference one of the github_mcp_tools prompt files
// (plain or with_safeoutputs variant) so the list_code_scanning_alerts guardrail
// is injected at runtime. Both variants carry the same guardrail.
hasGitHubMCPPrompt := strings.Contains(lockStr, githubMCPToolsPromptFile) ||
strings.Contains(lockStr, githubMCPToolsWithSafeOutputsPromptFile)
assert.True(t, hasGitHubMCPPrompt,
"lock file should reference a github_mcp_tools prompt file when code_security toolset is active")
}

// TestGeneratedWorkflowsValidatePromptStep tests that all generated workflows
// include the prompt validation step
func TestGeneratedWorkflowsValidatePromptStep(t *testing.T) {
Expand Down