Where does the breadcrumb column come from? It is not in my Screaming Frog export.

Breadcrumbs are not part of the default internal_html.csv export. You need to configure a custom extraction (Configuration > Custom > Extraction) targeting your breadcrumb element before crawling, then export with that column included. The tool auto-selects any column with 'breadcrumb' in its name, and you can map Address and H1 columns manually.

What does the similarity threshold actually measure?

A difference, not an absolute score. For each product the tool computes TF-IDF similarity between the H1 and its existing breadcrumb, and separately finds the best-matching category breadcrumb on the site. A product is flagged when best-match similarity minus existing similarity is at or above the threshold (default 0.3), meaning a noticeably better-fitting category exists elsewhere.

Does it need an OpenAI API key?

No. Despite what the in-app help expander says, there is no LLM involved; matching is pure PolyFuzz TF-IDF running locally, so no API key or per-row cost.

What happens if my URL patterns do not match anything?

If no URLs contain the product pattern (default /product/) the run stops with an error. If no URLs contain the category pattern (default /category/) it warns and falls back to using every crawled page as a candidate category, which can produce odd best matches such as other product pages. Set both patterns to match your actual URL structure, for example /p/ or /collections/.

Why do some products appear twice in the results?

Similarity results are joined back onto products by H1 text, so products sharing an identical H1 can produce duplicated rows after the merge. Deduplicate on URL in the exported CSV, or fix the duplicate H1s, which are usually an SEO issue anyway.

How is breadcrumb depth calculated?

By counting '/' characters in the breadcrumb string. This only makes sense if your extracted breadcrumbs are slash-separated paths; if your extraction returns text separated by '>' or similar, the depth columns will read 0 and the depth difference is meaningless, though similarity scoring still works.

Back to Tools

Breadcrumb Relevancy Checker

Use cases

Finding products in wrong categories Improving site taxonomy E-commerce category optimisation Internal linking improvement

Uses PolyFuzz TF-IDF fuzzy matching to compare product H1s/titles against breadcrumb category paths.

Filters rows with missing values and calculates similarity scores.

Configurable product URL pattern (default: /product/), category URL pattern (default: /category/), and flagging threshold (0.0-1.0, default 0.3) applied to the difference between the best-match similarity and the existing breadcrumb similarity.

Streamlit App

Platform

Browser-based (no installation required)

Input

Crawl CSV with URLs, titles, breadcrumbs

Rows with missing values filtered automatically

Output

CSV: product URLs, existing breadcrumbs, best matching categories, similarity scores, breadcrumb depth differences, miscategorisation flags.

Launch App View Source

Features

PolyFuzz TF-IDF product-to-category matching
Flags products where the best match beats the existing breadcrumb by a similarity difference threshold (0.0-1.0, default 0.3)
Configurable product/category URL patterns
Breadcrumb depth difference calculation (counts "/" in the breadcrumb text)
Detects products assigned to generic "/all" categories
Summary metrics: products analysed, miscategorisations, "all" assignments

How to use

1 Configure a Screaming Frog custom extraction for breadcrumbs before crawling
2 Upload the crawl CSV export
3 Map Address, H1 and Breadcrumb columns
4 Set product and category URL patterns (defaults /product/, /category/)
5 Adjust the similarity difference threshold for flagging
6 Review and download the full results CSV

Frequently asked questions

Where does the breadcrumb column come from? It is not in my Screaming Frog export.: Breadcrumbs are not part of the default internal_html.csv export. You need to configure a custom extraction (Configuration > Custom > Extraction) targeting your breadcrumb element before crawling, then export with that column included. The tool auto-selects any column with 'breadcrumb' in its name, and you can map Address and H1 columns manually.
What does the similarity threshold actually measure?: A difference, not an absolute score. For each product the tool computes TF-IDF similarity between the H1 and its existing breadcrumb, and separately finds the best-matching category breadcrumb on the site. A product is flagged when best-match similarity minus existing similarity is at or above the threshold (default 0.3), meaning a noticeably better-fitting category exists elsewhere.
Does it need an OpenAI API key?: No. Despite what the in-app help expander says, there is no LLM involved; matching is pure PolyFuzz TF-IDF running locally, so no API key or per-row cost.
What happens if my URL patterns do not match anything?: If no URLs contain the product pattern (default /product/) the run stops with an error. If no URLs contain the category pattern (default /category/) it warns and falls back to using every crawled page as a candidate category, which can produce odd best matches such as other product pages. Set both patterns to match your actual URL structure, for example /p/ or /collections/.
Why do some products appear twice in the results?: Similarity results are joined back onto products by H1 text, so products sharing an identical H1 can produce duplicated rows after the merge. Deduplicate on URL in the exported CSV, or fix the duplicate H1s, which are usually an SEO issue anyway.
How is breadcrumb depth calculated?: By counting '/' characters in the breadcrumb string. This only makes sense if your extracted breadcrumbs are slash-separated paths; if your extraction returns text separated by '>' or similar, the depth columns will read 0 and the depth difference is meaningless, though similarity scoring still works.

Want me to run this for you?

I run this tool as a managed service, or build something custom around your data. You get the insights without touching the code.

Tell Me What You Need

Non-White Background DetectorAppE-commerce

Detect product images that don't have white backgrounds.

Product Title Gap AnalyserAppE-commerce

Compare product titles with competitors using MPN matching to find missing keywords.

Product Spec ExtractorAppE-commerce

Scrape product specifications from e-commerce pages into structured CSV.

Product Q&A ExtractorAppE-commerce

Extract product reviews, ratings, and Q&A from e-commerce pages.

Internal Search to Landing Page MapperScriptE-commerce

Match internal site search terms to landing pages using fuzzy TF-IDF matching.

Break-even ROAS CalculatorBrowserPPC

Calculate the minimum ROAS needed to break even.

Need something built for your business?

This tool started as bespoke client work. I build custom scripts, data pipelines, and full apps for SEO and product data problems that off-the-shelf tools don't solve.