Your Audit Knows Where Pages Are Filed, It Cannot Tell You What They Mean.
How do you know which pages on your site are topically stranded? Most audits answer that with folder structure. They group `/blog/`, `/services/`, `/guides/`, check that each has internal links, and call it done. That tells you how your URLs are filed. It tells you nothing about whether pages that should reinforce each other actually connect.
A page can sit in the right folder, carry internal links, and still be isolated from the cluster of content it belongs to. Folder paths cannot see that. Meaning can.
The limit of structural audits
Crawlers like Screaming Frog map your site as a graph of links and directories. That is useful for finding broken links and crawl depth. It is blind to topic. Two pages about the same buyer question can live in different folders with no link between them, and a structural audit will report both as healthy.
For AI search this gap costs you. Engines build their sense of what you are authoritative on from how your content clusters around a topic, not from your URL tree. Pages that should form a tight cluster but sit disconnected read as thin coverage instead of depth.
Embeddings turn pages into coordinates
An embedding is a list of numbers that represents the meaning of a piece of text, produced by a model. Pages about similar topics get similar vectors. Once every URL has one, you can measure how close any two pages are by the angle between their vectors, a score called cosine similarity that runs from 0 for unrelated to 1 for near-identical.
That score is the part folder structure cannot give you. Two pages with high similarity and no link between them are a missed internal link. A page with low similarity to everything else is an orphan in meaning, even if it has plenty of links.
The workflow is short.
Pull the main content from each URL, not the boilerplate.
Generate an embedding for each page with an off-the-shelf model.
Score every page pair for cosine similarity.
Flag high-similarity pairs that have no internal link, and pages that score low against your whole site.
Reading the results
The same similarity number points to opposite actions depending on how high it runs, which is the whole reason it needs a person reading it rather than a threshold.
| What you see | What it usually means | The move |
|---|---|---|
| High similarity, no internal link | Two pages already cover related ground and never got connected | Add the link, with anchor text that names the shared topic |
| Low similarity to everything | The page is off-strategy, or the lonely start of a cluster you never built | Drop it, or commit to the supporting content that gives it company |
| Similarity above 0.9 | Near-duplicates competing for the same intent, not a linking gap | Consolidate, do not cross-link two pages fighting each other |
High-similarity, no-link pairs are your fastest wins. These are pages waiting for a link that tells both readers and engines they belong together. A page that scores low against everything is the harder call, because it could be dead weight or it could be a cluster you owe content to. Either way it is a decision, not a default.
The edge case is the one that bites. Two pages above 0.9 look identical to the math whether they are a linking opportunity or a consolidation problem, and only context tells you which. Cross-link two near-duplicates and you have reinforced the exact confusion you were trying to fix.
Where this fits
You do not need a data team to start. The embedding models are cheap to call, the similarity math is a few lines of Python, and the output is a ranked list of linking opportunities your structural tools will never surface.
The harder part is judgment. Knowing when a high score means "link these" and when it means "merge these," and which orphan deserves a cluster versus deletion. That reading is a skill worth building on your own team rather than renting forever. It is exactly the kind of work our corporate SEO training is built to hand over, so your people can run this analysis every quarter without waiting on anyone. If you would rather see it run on your site first, our free AI Visibility Audit is where that starts.
See where your brand stands in AI answers today, benchmarked against your competitors, no pitch required.
Who Google AI Mode actually cites when Indonesians ask about banking
We ran 72 Indonesian banking and finance prompts through Google AI Mode, 216 answers in all, and logged every source it cited. Banks dominate, comparison sites and Instagram punch above their weight, and regulators barely show up.
read_post →
AI stopped reading your pages and started reading your entity graph
AI is starting to retrieve your brand as a graph of connected entities, and a lot of well-written sites are illegible in that format. Here is what GraphRAG changes and how to fix your entity legibility.
read_post →
Brand mentions or backlinks, what actually gets you cited in 2026?
Links still move Google, but a plain unlinked mention of your brand may do more for you in AI search. Here is how the two signals split in 2026, and how to build both.
read_post →