What must my Screaming Frog exports contain?

internal_html.csv needs Address, Indexability, H1-1 and Title 1 columns plus your two custom extraction columns; any non-empty cell in an extraction column marks that URL as a product or category page. inlinks.csv needs the standard From and To columns. Non-indexable URLs are discarded, and H1s are stripped to ASCII, so accented characters vanish from generated keywords. Encoding is auto-detected, including Screaming Frog's UTF-16 exports.

How is DataForSEO API usage charged?

Deduplicated keyword suggestions are sent to the DataForSEO Google Ads search volume endpoint in batches of 700 keywords per request, and DataForSEO charges per request rather than per keyword. The app tells you how many requests the run needs before fetching, and waits 5 seconds between batches to stay within the 12 requests per minute rate limit.

Why do keywords with search volume disappear from the results?

Both post-API filters are strictly greater-than: volume must exceed the minimum (default 100) and CPC must exceed the minimum CPC slider. Because the CPC default is 0, keywords with a 0.00 CPC are dropped even at default settings, so suggestions with volume but no advertiser data are silently removed whenever API credentials are used.

How does the minimum product match work?

It is a substring check: an n-gram counts as matching a product if it appears anywhere inside the product's H1 (default minimum 3 products). The optional fuzzy mode instead requires every word of the keyword to appear in the H1 in any order, which catches reworded titles but is much slower.

How does it avoid suggesting categories I already have?

Three checks: PolyFuzz TF-IDF similarity against existing category H1s (suggestions above the max similarity slider, default 96 percent, are dropped), a word-set comparison against the matched category's page title in any word order, and the same comparison with a naive plural formed by adding s. Irregular plurals and synonyms can still slip through, so expect a manual review pass.

Are the DataForSEO credentials required?

No. Without them you still get every n-gram suggestion with product match counts and similarity scores, but no Search Volume or CPC columns, which means many suggestions nobody actually searches for. The credentials are what filter the list down to keywords with demonstrated demand.

Back to Tools

Category Page Suggester

Use cases

Expanding category structure based on actual product inventory Aligning taxonomy with search demand using real keyword data Identifying gaps between what you sell and how users search Prioritising new categories by search volume and product coverage

Extracts 2-7 word n-grams from product H1s, matches against existing categories using PolyFuzz TF-IDF similarity scoring, and validates search demand via the DataForSEO Google Ads search volume endpoint.

Filters by minimum product matches, search volume, and similarity threshold.

Originally presented at Brighton SEO.

Streamlit App

Platform

Browser-based (no installation required)

Input

Screaming Frog inlinks and crawl exports

DataForSEO login and password (optional)

Output

CSV with category suggestions, volumes, and similarity scores

Launch App View Source

Features

N-gram extraction (2-7 words) from product H1s
PolyFuzz TF-IDF fuzzy matching to existing categories (0-100% similarity)
DataForSEO search volume integration (volume, CPC, competition) in batches of 700 keywords per request
Configurable filters: min volume, CPC, product matches, similarity threshold
Pluralisation handling and multi-stage deduplication

How to use

1 Crawl site with Screaming Frog, set up custom extraction for product vs category pages
2 Export inlinks.csv and internal_html.csv
3 Upload both files and map the product/category identification columns
4 Configure similarity threshold, min product matches, min volume/CPC
5 Run n-gram extraction and fuzzy matching
6 Download CSV of suggestions sorted by parent category and keyword

Frequently asked questions

What must my Screaming Frog exports contain?: internal_html.csv needs Address, Indexability, H1-1 and Title 1 columns plus your two custom extraction columns; any non-empty cell in an extraction column marks that URL as a product or category page. inlinks.csv needs the standard From and To columns. Non-indexable URLs are discarded, and H1s are stripped to ASCII, so accented characters vanish from generated keywords. Encoding is auto-detected, including Screaming Frog's UTF-16 exports.
How is DataForSEO API usage charged?: Deduplicated keyword suggestions are sent to the DataForSEO Google Ads search volume endpoint in batches of 700 keywords per request, and DataForSEO charges per request rather than per keyword. The app tells you how many requests the run needs before fetching, and waits 5 seconds between batches to stay within the 12 requests per minute rate limit.
Why do keywords with search volume disappear from the results?: Both post-API filters are strictly greater-than: volume must exceed the minimum (default 100) and CPC must exceed the minimum CPC slider. Because the CPC default is 0, keywords with a 0.00 CPC are dropped even at default settings, so suggestions with volume but no advertiser data are silently removed whenever API credentials are used.
How does the minimum product match work?: It is a substring check: an n-gram counts as matching a product if it appears anywhere inside the product's H1 (default minimum 3 products). The optional fuzzy mode instead requires every word of the keyword to appear in the H1 in any order, which catches reworded titles but is much slower.
How does it avoid suggesting categories I already have?: Three checks: PolyFuzz TF-IDF similarity against existing category H1s (suggestions above the max similarity slider, default 96 percent, are dropped), a word-set comparison against the matched category's page title in any word order, and the same comparison with a naive plural formed by adding s. Irregular plurals and synonyms can still slip through, so expect a manual review pass.
Are the DataForSEO credentials required?: No. Without them you still get every n-gram suggestion with product match counts and similarity scores, but no Search Volume or CPC columns, which means many suggestions nobody actually searches for. The credentials are what filter the list down to keywords with demonstrated demand.

Want me to run this for you?

I run this tool as a managed service, or build something custom around your data. You get the insights without touching the code.

Book a Call

eCommerce Image Centering ToolAppE-commerce

Center product images using OpenCV multi-method contour detection with threshold fallbacks.

High-Res Image FinderScriptE-commerce

Find higher resolution images using Google Vision API with Playwright fallback and human-like delays.

Internal Search MapperAppE-commerce

Map site search queries to existing pages and discover content gaps from user behaviour.

PDF Branding InjectorAppE-commerce

Add branded headers to PDFs using PyPDF2 and ReportLab with responsive text scaling.

Product Title OptimiserScriptE-commerce

LLM-powered title restructuring with numerical validation and 80% word overlap checks.

Schema Markup GeneratorBrowserTechnical SEO

Generate JSON-LD structured data for common schema types.

Need something built for your business?

This tool started as bespoke client work. I build custom scripts, data pipelines, and full apps for SEO and product data problems that off-the-shelf tools don't solve.