Extract Sitemap

This repo is to extract sitemap from the main index file.

• Fetch the index URL https://www.example.com/sitemap_index.xml
• Parse it (XML or HTML fallback) to get the list of child URLs
• Fetch each child URL
• Pass each child's response body through the same code path as upload XML
• Merge results into one single xml file

Setup

• Clone the repo git clone https://github.com/BrowserStackCE/yoast-sitemap-extractor.git and run cd yoast-sitemap-extractor.
• Set the INDEX_URL in the index file.
• Run npm i to install the dependencies.

Running the extractor

• To run the extractor, please use the below command

    npm run start

• The sitemap xml should be extracted into the file merged_sitemap.xml

• Once the file is ready, you can upload the xml to the Website scanner.