DOCS-2966: Skip the bare calicousers.slack.com URL in the link check - #2935
Merged
Conversation
The Netlify deploy preview for the tigera site fails on the link check with one dead link: https://calicousers.slack.com is dead (403) ==>Origin: http://localhost:4242/calico/latest/reference/involved The skip list already carried this workspace, but as the exact strings https://calicousers.slack.com/ and a channel URL. Entries given as strings are matched exactly, so the bare host on the get involved page was still checked, and Slack returns 403 to datacenter crawlers. Replace both strings with an anchored pattern for the host, which covers the bare host, the trailing slash form and channel links. The 403 is intermittent, so the failure does not reproduce reliably on a local run. This failure is not caused by the 3.22.7 release. Only latest is crawled, and Calico Enterprise latest maps to 3.23, so no page in version-3.22-2 is checked at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Updates the Playwright/Crawlee docs crawler link-check skip list to reliably ignore Calico Users Slack workspace URLs that return intermittent 403s to datacenter crawlers, preventing Netlify preview link-check failures.
Changes:
- Replace two exact-string skip entries for
calicousers.slack.comwith a single regex-based skip entry. - Add inline comments explaining why Slack URLs are skipped and which page links the bare host.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| // Slack workspace URLs return 403 to datacenter crawlers; the links are fine in a browser. | ||
| // Matched as a pattern so the bare host, the trailing-slash form and channel links are all | ||
| // covered. The bare host is linked from /calico/latest/reference/involved. | ||
| /^https:\/\/calicousers\.slack\.com/, |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Netlify deploy preview for the tigera site fails on the 3.22.7 publishing PR #2917. The build itself succeeds. The failure is in the link check that runs after it, and it reports one dead link:
https://calicousers.slack.com is dead (403)
==>Origin: http://localhost:4242/calico/latest/reference/involved
The skip list in tests/crawler.test.js already covered this Slack workspace, but as two exact strings, https://calicousers.slack.com/ with a trailing slash and one channel URL. Entries given as strings are matched exactly, so the bare host linked from the get involved page was still checked. Slack returns 403 to datacenter crawlers, and the link is correct in a browser.
This replaces both strings with an anchored pattern for the host, which covers the bare host, the trailing slash form and channel links in one entry. The pattern is anchored so it does not match a lookalike host.
The 403 is intermittent, so the failure does not reproduce reliably. A local run of make netlify passed against the same content while the Netlify build was failing, which is why the pattern is the right fix rather than adding one more exact string.
This failure has nothing to do with the 3.22.7 release. Only latest is crawled, seeded from sitemap.xml, and Calico Enterprise latest maps to 3.23, so no page under version-3.22-2 is checked at all. The same dead link fails every open pull request's tigera preview, so this unblocks more than the publishing PR.
Based on the publication branch so the fix reaches #2917 and its preview can rebuild.
Netlify does not build a deploy preview for a pull request based on a publish branch, so there is no preview link here, and the pull request checks do not run the link check. The verification is the tigera preview on #2917 once this merges.