Why do some pages come back as 'Content too short for analysis'?

Pages are skipped if the extracted text is under 100 words. Trafilatura also excludes tables and comment sections from extraction, so a page that looks long in the browser can still fall under the threshold if its substance is in tables or if extraction only found boilerplate.

Can I point it at a sitemap index?

No, use an individual sitemap. The tool reads the loc entries from the single XML file you give it and does not recurse into child sitemaps, so the entries of a sitemap index (which are sitemap URLs, not pages) would be queued as pages and fail extraction. URLs ending in .png, .jpg, .jpeg, .gif, .svg, .pdf and .webp are filtered out automatically; other non-HTML URLs are attempted and simply error.

How is reading time calculated?

Word count divided by 200 words per minute, rounded to one decimal. The word count itself is textstat's lexicon count with punctuation removed, the same figure shown in the word_count column.

How does it avoid getting blocked on larger runs?

Requests are sequential (never parallel) with your configured delay of 0.5 to 10 seconds between pages, and each request sends a different randomised browser user agent via fake-useragent. Note the default Maximum URLs is 100; raise it (up to 10,000) or a big sitemap will be silently cut to the first 100 with just a warning.

Can I get the extracted text as well as the scores?

Yes, but only if you tick 'Include content in export' in the sidebar before running; it is off by default. The extracted text is then added as a content column in the CSV, which is useful for spot-checking what the scores were actually calculated on.

Back to Tools

Reading Score Analyser

Use cases

Content readability audits Finding hard-to-read pages Content simplification planning Competitor readability comparison

Uses Trafilatura for content extraction and textstat for 8 readability metrics: Flesch Reading Ease, Flesch-Kincaid Grade, Gunning Fog, SMOG, ARI, Coleman-Liau, Linsear Write, Dale-Chall.

Also calculates word/sentence counts, avg sentence length, difficult words, reading time.

Fake-useragent for request randomisation.

Streamlit App

Platform

Browser-based (no installation required)

Input

XML sitemap URL (auto-extracts URLs)

CSV with URL column

Or paste URLs directly (one per line)

Output

CSV with readability scores and metrics

Launch App View Source

Features

Trafilatura content extraction
8 textstat readability metrics
Word/sentence counts and reading time
Fake-useragent request randomisation
Request delay slider (0.5-10 seconds)
URL limit up to 10,000 per session

How to use

1 Enter a sitemap URL, upload a CSV, or paste URLs
2 Configure request delay (0.5-10s) and timeout (5-60s)
3 Set the URL limit (default 100, up to 10,000)
4 Run the analysis; image and PDF URLs from sitemaps are filtered out
5 Review the readability distribution and hardest-to-read pages
6 Download the CSV, optionally including the extracted content

Frequently asked questions

Why do some pages come back as 'Content too short for analysis'?: Pages are skipped if the extracted text is under 100 words. Trafilatura also excludes tables and comment sections from extraction, so a page that looks long in the browser can still fall under the threshold if its substance is in tables or if extraction only found boilerplate.
Can I point it at a sitemap index?: No, use an individual sitemap. The tool reads the loc entries from the single XML file you give it and does not recurse into child sitemaps, so the entries of a sitemap index (which are sitemap URLs, not pages) would be queued as pages and fail extraction. URLs ending in .png, .jpg, .jpeg, .gif, .svg, .pdf and .webp are filtered out automatically; other non-HTML URLs are attempted and simply error.
How is reading time calculated?: Word count divided by 200 words per minute, rounded to one decimal. The word count itself is textstat's lexicon count with punctuation removed, the same figure shown in the word_count column.
How does it avoid getting blocked on larger runs?: Requests are sequential (never parallel) with your configured delay of 0.5 to 10 seconds between pages, and each request sends a different randomised browser user agent via fake-useragent. Note the default Maximum URLs is 100; raise it (up to 10,000) or a big sitemap will be silently cut to the first 100 with just a warning.
Can I get the extracted text as well as the scores?: Yes, but only if you tick 'Include content in export' in the sidebar before running; it is off by default. The extracted text is then added as a content column in the CSV, which is useful for spot-checking what the scores were actually calculated on.

Want me to run this for you?

I run this tool as a managed service, or build something custom around your data. You get the insights without touching the code.

Tell Me What You Need

Page Intent ClassifierAppContent

Use OpenAI to classify page intent and expected user actions.

Review Sentiment ExtractorAppContent

Use OpenAI to extract positive and negative sentiments from product reviews.

Keyword-to-Page MapperAppContent

Semantically match keywords to existing pages using ML embeddings.

Category Title SuggesterAppContent

Analyse category pages and suggest high-performing keywords for titles.

Meta Description GraderAppContent

Score and compare meta descriptions using OpenAI GPT models on key SEO criteria.

Redirect List FormatterBrowserMigration

Convert URL lists into redirect rules for different platforms.

Need something built for your business?

This tool started as bespoke client work. I build custom scripts, data pipelines, and full apps for SEO and product data problems that off-the-shelf tools don't solve.