Skip to content

docs: update llms.txt with documentation index and add Playwright test#46287

Closed
pelikhan with Copilot wants to merge 3 commits into
mainfrom
copilot/geo-optimizer-add-robots-llms
Closed

docs: update llms.txt with documentation index and add Playwright test#46287
pelikhan with Copilot wants to merge 3 commits into
mainfrom
copilot/geo-optimizer-add-robots-llms

Conversation

Copilot AI commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

GEO audit scored the docs site 0/18 on llms.txt — the existing llms.txt.ts served agent prompt links identical to agents.txt, rather than a proper documentation index.

Changes

  • docs/src/pages/llms.txt.ts — Rewrites the endpoint to generate a standards-compliant llms.txt documentation index:

    • Uses getCollection('docs') to fetch all docs pages at build time
    • Groups pages into ordered sections: Introduction → Setup → Design Patterns → Guides → Reference → Experimental → Specifications → Troubleshooting
    • Each entry uses frontmatter title + description, linking to https://github.github.com/gh-aw/<id>/
    • Appends the agent prompts section (.github/aw/*.md) at the end, preserving prior content for AI agents
    • Extracts DocPageData interface to avoid repeated type assertions
  • docs/tests/llms-txt.spec.ts — New Playwright test verifying /gh-aw/llms.txt returns 200, correct Content-Type, expected section headings, and well-formed - [Title](url) link lines

The existing robots.txt (in public/) and its test are unchanged.

# GitHub Agentic Workflows

> Write agentic AI workflows in natural language markdown, run as GitHub Actions.

## Introduction

- [How They Work](https://github.github.com/gh-aw/introduction/how-they-work/): ...
- [About Workflows](https://github.github.com/gh-aw/introduction/overview/): ...

## Setup
...
## Agent Prompts
- [github-agentic-workflows](https://raw.githubusercontent.com/...): ...

Copilot AI and others added 2 commits July 17, 2026 17:04
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
…GEO optimization

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Add robots.txt and llms.txt to docs site for AI bot crawling docs: update llms.txt with documentation index and add Playwright test Jul 17, 2026
Copilot AI requested a review from pelikhan July 17, 2026 17:15
@github-actions

Copy link
Copy Markdown
Contributor

Great work on the llms.txt documentation index! 🚀

The PR includes a standards-compliant rewrite that properly groups docs pages into ordered sections, and adds a new Playwright test verifying the endpoint — exactly the kind of quality infrastructure improvement this project benefits from. The workflow lock file changes look like automated bumps.

This looks ready for review! ✅

Generated by ✅ Contribution Check · 84.1 AIC · ⌖ 13.8 AIC · ⊞ 6.2K ·

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[geo-optimizer] Add robots.txt and llms.txt to docs site to unlock AI bot crawl permissions

2 participants