Does it find the sitemap for me?

No, you supply the sitemap or sitemap index URL directly. It does not read robots.txt to discover sitemap locations, so check there first if you do not know the URL.

What happens with malformed or broken sitemaps?

If XML parsing fails, a regex fallback pulls anything between loc tags, so you still get URLs, but lastmod, priority and changefreq are not captured in fallback mode. Errors fetching an individual child sitemap are shown in the status line and the run continues; failed sitemaps are not listed in the final export, so sanity check the Sitemaps Processed count.

Where does the request delay actually apply?

Between sitemap file fetches only. The listed URLs are never requested, so a 50,000 URL sitemap is a single request; the delay only matters for indexes with many child sitemaps.

It decompresses when the URL ends in .gz or when the response body starts with the gzip magic bytes, so compressed sitemaps served without the .gz extension still work.

What if the server blocks the tool?

The default user agent is "Mozilla/5.0 (compatible; SitemapExtractor/1.0)", which some hosts and CDNs reject. Change the user agent field to a full browser string if you see 403s. Each request also times out after 30 seconds.

Back to Tools

Sitemap URL Extractor

Use cases

SEO audits - get complete URL inventory Site migrations - document all URLs before changes Competitor analysis - understand site structure Content inventory - catalog all pages Indexation monitoring - compare sitemap vs indexed

Dual-strategy XML parsing: ElementTree with namespace handling, regex fallback (<loc>.*?</loc>).

Recursive queue-based processing of sitemap indexes.

Gzip decompression for .xml.gz files.

Configurable request delay (0-5 seconds slider).

Duplicate prevention via processed_sitemaps set tracking.

Streamlit App

Platform

Browser-based (no installation required)

Input

Sitemap URL (index or individual .xml/.xml.gz)

Custom user agent string

Request delay: 0-5 seconds

Output

URL list with sitemap metadata (CSV/Excel)

Launch App View Source

Features

Recursive sitemap index processing via queue
Dual parsing: ElementTree + regex fallback
Gzip decompression (.xml.gz support)
Duplicate sitemap prevention
Configurable delay slider (0-5 seconds)

How to use

1 Enter sitemap URL in the sidebar
2 Set custom user agent if needed
3 Configure request delay (recommended for large sites)
4 Review extracted URLs with metadata
5 Download as CSV, TXT, or Excel (timestamped filenames)

Frequently asked questions

Does it find the sitemap for me?: No, you supply the sitemap or sitemap index URL directly. It does not read robots.txt to discover sitemap locations, so check there first if you do not know the URL.
What happens with malformed or broken sitemaps?: If XML parsing fails, a regex fallback pulls anything between loc tags, so you still get URLs, but lastmod, priority and changefreq are not captured in fallback mode. Errors fetching an individual child sitemap are shown in the status line and the run continues; failed sitemaps are not listed in the final export, so sanity check the Sitemaps Processed count.
Where does the request delay actually apply?: Between sitemap file fetches only. The listed URLs are never requested, so a 50,000 URL sitemap is a single request; the delay only matters for indexes with many child sitemaps.
How is gzip handled?: It decompresses when the URL ends in .gz or when the response body starts with the gzip magic bytes, so compressed sitemaps served without the .gz extension still work.
What if the server blocks the tool?: The default user agent is "Mozilla/5.0 (compatible; SitemapExtractor/1.0)", which some hosts and CDNs reject. Change the user agent field to a full browser string if you see 403s. Each request also times out after 30 seconds.

Want me to run this for you?

I run this tool as a managed service, or build something custom around your data. You get the insights without touching the code.

Book a Call

Redirect ValidatorAppTechnical SEO

Validate implemented redirects against your mapping specification.

Hreflang CheckerAppTechnical SEO

Extract and validate hreflang tags from any website.

Hreflang GeneratorAppTechnical SEO

Generate hreflang tags for multi-language websites.

OnCrawl API SuiteAppTechnical SEO

Comprehensive OnCrawl API client for data extraction, crawl management, and analysis.

Q&A Schema ExtractorAppTechnical SEO

Extract Question/Answer pairs from JSON-LD schema markup.

SERP Snippet PreviewBrowserContent

Preview how your page will appear in Google search results.

Need something built for your business?

This tool started as bespoke client work. I build custom scripts, data pipelines, and full apps for SEO and product data problems that off-the-shelf tools don't solve.