feat: add llms markdown in doc pages#891
Conversation
4d75c03 to
5f883db
Compare
…Markdown button Signed-off-by: Shreya Sharma <shreyasharma.1510001@gmail.com>
Signed-off-by: Shreya Sharma <shreyasharma.1510001@gmail.com>
9a729fd to
7ed125a
Compare
There was a problem hiding this comment.
Pull request overview
Adds first-class “LLM-friendly” outputs for the docs site by generating per-page .md twins plus llms.txt / llms-full.txt, and exposes the per-page Markdown via UI actions under each doc title.
Changes:
- Add and configure
docusaurus-plugin-llmsto generatellms.txt,llms-full.txt, and per-page Markdown outputs at build time. - Add a
MarkdownPageActionsUI control to “Copy as Markdown” / “View as Markdown” for eligible doc pages. - Wire the new control into the DocItem theme and update lockfile for new dependencies.
Reviewed changes
Copilot reviewed 4 out of 5 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| yarn.lock | Locks new transitive dependencies required by docusaurus-plugin-llms. |
| package.json | Adds docusaurus-plugin-llms dependency. |
| docusaurus.config.js | Configures the plugin to generate llms*.txt and Markdown per doc page. |
| src/theme/DocItem/index.js | Computes .md twin URL and renders Markdown actions under the doc title. |
| src/components/MarkdownPageActions.js | Implements “Copy as Markdown” / “View as Markdown” UI and clipboard logic. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…leak Signed-off-by: Shreya Sharma <shreyasharma.1510001@gmail.com>
…tripping Signed-off-by: Shreya Sharma <shreyasharma.1510001@gmail.com>
manasmanohar
left a comment
There was a problem hiding this comment.
Please address the comments below. Specifics left inline; this one isn't tied to a diff line:
Collides with the llms.txt we already ship.
We already have curated static/llms.txt and static/llms-full.txt (content team's — About Keploy, awards, positioning, curated links). The plugin generates the same two filenames, and its postBuild runs after static/ is copied, so it overwrites them: build/llms.txt is the generated version and the curated content is gone. Net effect — the content team's file is silently replaced, and editing static/llms.txt later would have no effect. The static file also drifts today since it's hand-maintained.
Option: merge both — feed the curated text into the plugin via rootContent/fullRootContent (+ title/description) so it stays on top and the auto-generated index below stays fresh. Needs the curated source moved out of static/ (so it isn't overwritten) and the static copies removed. Worth confirming the exact split with the content team.
| [ | ||
| "docusaurus-plugin-llms", | ||
| { | ||
| docsDir: "versioned_docs/version-4.0.0", |
There was a problem hiding this comment.
docsDir is hardcoded to versioned_docs/version-4.0.0 — will need updating when we cut the next docs version, since the buttons show on whatever the latest version is (gated on isLatestVersion).
Signed-off-by: Shreya Sharma <shreyasharma.1510001@gmail.com>
8595693 to
986babe
Compare
…button width shift Signed-off-by: Shreya Sharma <shreyasharma.1510001@gmail.com>
Signed-off-by: Shreya Sharma <shreyasharma.1510001@gmail.com>
This reverts commit da566b4. Signed-off-by: Shreya Sharma <shreyasharma.1510001@gmail.com>
f2fac16 to
ae501bb
Compare
Signed-off-by: Shreya Sharma <shreyasharma.1510001@gmail.com>
| docsDir: "versioned_docs/version-4.0.0", | ||
| ignoreFiles: ["**/shared/**"], | ||
| generateLLMsTxt: true, | ||
| generateLLMsFullTxt: true, | ||
| generateMarkdownFiles: true, | ||
| preserveDirectoryStructure: false, | ||
| excludeImports: true, |
| // Folder-style permalinks (section landing pages, ending in "/") are | ||
| // written by docusaurus-plugin-llms as index.md inside that folder, | ||
| // not as <folder>.md -- so the two cases need different URL shapes. | ||
| const markdownUrl = showMarkdownActions | ||
| ? permalink.endsWith("/") | ||
| ? `${permalink}index.md` | ||
| : `${permalink}.md` | ||
| : null; |
| // .md twins are only generated for the current (4.0.0) version's docs | ||
| // (docusaurus-plugin-llms docsDir points at versioned_docs/version-4.0.0), | ||
| // and only for real doc pages, not root/category-index pages. |
What has changed?
Every docs page is now also available as plain Markdown, and
llms.txt/llms-full.txtare auto-generated at build time — so AI agents (and humans) can grab clean per-page content instead of parsing HTML.docusaurus-plugin-llms, configured to generatellms.txt,llms-full.txt, and a.mdtwin for every doc page (same URL,.mdextension)..mdfile is actually discoverable, not just guessable. Styled with Tailwind utility classes to match this repo's existing convention.docusaurus-plugin-llms(viapatch-package, auto-reapplied on install) to fix two real content bugs found during review:import WhatIsKeploy from '../concepts/what-is-keploy.md'); the plugin only resolved these when the filename had an underscore, so ~7 pages were leaking that raw import/JSX line into their.mdoutput instead of the real text.<Tabs>,<ProductTier>,<HowTo>, etc.) into their.mdoutput. Fixed by stripping self-closing component tags entirely and unwrapping paired ones while preserving their real inner content — carefully excluding<YOUR_INGRESS_HOST>-style fill-in-your-own-value placeholders, which look similar but aren't components.setTimeoutresetting its label was never cleared, so it could fire after the component unmounted (React warning) or race itself on repeated clicks.Fixes https://github.com/keploy/enterprise/issues/2272
How Has This Been Tested?
Ran
npm run buildthennpm run serve— confirmed the build succeeds,llms.txt/llms-full.txtgenerate correctly, each page's.mdtwin loads at its matching URL, and the Copy/View buttons work in both light and dark mode.Preview.Video.mov