Technical SEO: a complete audit framework

Most technical SEO audits are organised the way a textbook is organised: a chapter on crawling, a chapter on speed, a chapter on structured data, a chapter on mobile. That ordering is comfortable to write and almost useless to act on, because it gives no signal about which of the ninety-four items in the report will change anything.

This audit is ordered differently. It runs from "this page does not exist as far as a search engine is concerned" down to "this is a preference, not a fault". Work top to bottom. Stop when the remaining items stop mattering — which, on most sites, is sooner than the tool suggests.

The three stages that people collapse into one

Before any of the checks make sense, separate three things that get used interchangeably and are not the same.

Crawling is a request. A crawler asks your server for a URL and your server responds. If it cannot reach the URL, nothing downstream happens.

Rendering is execution. The fetched HTML gets processed, and where JavaScript builds the content, that JavaScript has to run before the content exists. Rendering happens on Google's schedule, not yours, and it can lag behind the crawl.

Indexing is a decision. Having crawled and rendered a page, the search engine decides whether to store it, and under which canonical URL. Crawled and indexed are different states, and Search Console reports them separately for exactly this reason.

Nearly every confusing indexing problem comes from mixing these up. "Google can't see my page" is three different faults with three different fixes, and you cannot pick the fix until you know which stage failed. The same distinction underpins how ranking actually works from crawl to result — ranking is the fourth stage, and it only happens to pages that survived the first three.

Tier one: faults that remove the page from search entirely

These are binary. The page is either eligible or it is not. Nothing else on the audit matters if one of these is live.

Is the URL blocked in robots.txt?

Fetch yoursite.com/robots.txt in a browser and read it. Not a tool's summary of it — the actual file. Look for Disallow rules that are broader than intended. The classic disaster is a staging rule surviving a launch:

User-agent: * Disallow: /

Look also for accidental prefix matches. Disallow: /blog blocks /blog-archive too, because robots.txt matches prefixes, not path segments.

Is the page noindexed?

Check the raw HTML for a robots meta tag, and check the HTTP response headers for X-Robots-Tag. The header version is the one people miss, because it is invisible in the page source — it is set at the server or CDN and only appears if you inspect the response.

The confusion that costs the most time

Robots.txt and noindex do opposite things, and using the wrong one is one of the most common serious faults in real audits.

| | robots.txt Disallow | noindex | |---|---|---| | Stops crawling | Yes | No | | Stops indexing | No, not reliably | Yes | | Page can still appear in results | Yes, occasionally, as a bare URL | No | | Requires the crawler to fetch the page | No | Yes — it must read the tag |

The trap: if you want a page out of the index, you must let it be crawled so the noindex can be read. Blocking a page in robots.txt while it carries a noindex tag means the tag is never seen and the page may linger. Google's documentation is explicit that a disallowed URL can still be indexed if it is linked from elsewhere.

Do the canonicals point where you think?

A canonical tag tells search engines which URL is the master copy of a set of near-identical pages. When it is right, it consolidates duplicates. When it is wrong, it deletes pages from search.

Check for these, in rough order of how often they cause real damage:

• Every page canonicalising to the homepage. Usually a template or plugin default. It tells the search engine your entire site is one page.
• Canonicals pointing to a URL that redirects, 404s, or is itself noindexed.
• Protocol and host mismatches — http canonicals on an https site, www canonicals on a non-www site.
• Paginated pages all canonicalising to page one, hiding everything past the first twenty products.
• Canonicals injected by JavaScript that conflict with a different one in the raw HTML.

Does the server actually respond?

Run a status-code check across your URL list. What you want to find and fix:

• 5xx errors, especially intermittent ones under crawl load. A server that fails one request in twenty is worse than it sounds, because crawling is bursty.
• Soft 404s: a "product not found" page returning 200. The search engine indexes an empty page and eventually distrusts the pattern.
• Redirect chains longer than two hops, and redirect loops.
• Important pages reachable only from a sitemap and linked from nowhere on the site.

Can the content be seen without JavaScript?

Disable JavaScript in your browser and load a key page. If the main body copy vanishes, you are dependent on rendering — which usually works, but adds latency and one more failure point. If your content is behind a client-side fetch that requires a session, a cookie consent interaction, or a click, treat it as invisible until proven otherwise.

The check that settles it: use the URL Inspection tool in Search Console, request the live test, and read the rendered HTML it returns. That is the closest available view of what Google actually stored.

The triage rule

Here is the part most audits skip, and the reason most audits get filed and forgotten.

Crawling tools flag warnings by category, not by consequence. A missing alt attribute on a decorative divider and a homepage-wide canonical error appear in the same list, styled the same way, often sorted by count so the trivial one appears more urgent because there are four hundred of them.

Use this test on every flagged item:

Name the page that loses traffic if this stays broken, and name the mechanism by which it loses it.

If you can name both — "our twelve highest-value service pages are canonicalising to the homepage, so they are not indexed, so they cannot rank" — it is a real fault. Fix it.

If you can only name a rule being violated — "the meta description is 168 characters and the recommended maximum is 155" — it is a cosmetic warning. Note it, batch it, and do it when you are next in that template anyway.

Three follow-up questions that sharpen the triage:

1. How many pages does it affect, and are they pages anyone wants? A fault on 4,000 tag-archive pages nobody searches for is less urgent than a fault on six pages that carry your revenue.
2. Is it a symptom or the disease? Four hundred broken internal links usually come from one bad URL pattern in one template. Fix the template, not the four hundred.
3. Would fixing it change what a searcher sees or whether they can reach it? If neither, it is housekeeping.

A useful expectation: most sites have one or two genuine technical problems and a long tail of noise. If your audit produces a list of forty "critical" items, the list is miscalibrated, not your site. If you are trying to work out why traffic is flat and technical checks all come back clean, the problem is probably elsewhere entirely — a structured diagnostic for a site that will not rank walks through the non-technical causes.

Tier two: faults that degrade pages that do exist

These pages are indexed. They are just worth less than they should be.

Duplication and near-duplication

Faceted navigation, tracking parameters, session IDs, printer-friendly versions, and trailing-slash inconsistencies can turn one page into dozens of URLs. The search engine picks one to keep and may not pick yours.

The fix is usually a combination: canonical tags to declare the master, consistent internal linking to the canonical form, and parameter handling so that filter combinations that produce no unique value are not linked at all. On an online shop this is often the single largest technical issue in the whole audit — the specific traps in shop and catalogue sites go further into faceted URLs.

Internal linking and depth

Count clicks from the homepage to your most important pages. Anything at depth four or more on a small site is a signal that your navigation and the page's importance disagree.

Also check for orphan pages — indexed URLs that no internal link points to. They exist in the sitemap and nowhere else, which is a weak statement of importance.

Sitemaps that describe a site you do not have

An XML sitemap should list canonical, indexable, 200-status URLs. Nothing else. Common faults: sitemaps containing redirects, noindexed pages, or 404s; sitemaps that were generated once in 2019 and never regenerated; multiple conflicting sitemaps referenced from different places.

The sitemap is a hint, not an instruction. It will not force indexing. It does help discovery on large or poorly linked sites, and a sitemap full of errors quietly reduces how much the hint is trusted.

Mobile

Google indexes the mobile version of pages. That makes mobile parity a tier-two issue with a tier-one edge case: if your mobile template hides or omits content that exists on desktop, the hidden content may effectively not exist.

Check three things and stop:

• Does the mobile page contain the same main content and the same internal links as desktop?
• Is structured data present on the mobile version too?
• Is anything essential locked behind an interstitial that covers the page on load?

Core Web Vitals, in proportion

This is where audits lose their sense of scale, so be blunt about it.

Speed matters, and Google has confirmed page experience signals are used. But they are one input among many, and they are a tie-breaker rather than a substitute for relevance. A fast page about the wrong topic does not outrank a slow page about the right one.

The proportionate position: get out of the "poor" bucket, then stop optimising and go do something else. The metrics worth knowing are Largest Contentful Paint (how long until the main content appears), Interaction to Next Paint (how sluggish the page feels when tapped), and Cumulative Layout Shift (how much things jump around while loading).

In practice, on ordinary sites, most of the win comes from a short list: oversized uncompressed images, render-blocking scripts in the head, no caching headers, and third-party tags — chat widgets, heat-mapping, four analytics tools — that nobody has audited since they were added. Fixing layout shift is usually the highest-value item on the list, because it is the one real users actually notice and swear at.

What is rarely worth it: chasing a score from 88 to 96, or refactoring a working framework because a tool suggested a different one.

Tier three: refinements

Genuine improvements. None of them will rescue a site that fails tier one.

Structured data. Schema markup does not make you rank higher. It makes you eligible for particular result features — review stars, FAQ blocks, product details, event listings. Mark up what is genuinely on the page, validate it, and expect eligibility rather than guarantee. Marking up content that is not visible to users, or inventing reviews to get stars, falls foul of Google's spam policies and structured data guidelines, and the enforcement is not gentle. For a business with a physical premises or service area, local business markup plus consistent details across the web is the version that pays — covered in more depth in what actually moves the needle in local search.

Hreflang, if you serve multiple languages or countries. It is finicky, must be reciprocal, and is easy to get subtly wrong.

HTTP/2 or 3, image formats, font loading, preconnects. Real wins, small wins.

Log file analysis. Genuinely useful on large sites, where you can see how crawl attention is distributed. On a fifty-page site it is a hobby.

A worked triage (illustrative)

An illustrative example, not measured data.

Say a two-branch veterinary practice runs a 60-page site and a crawl returns 214 issues, 38 marked critical. Triaged by consequence:

| Finding | Tier | Action | |---|---|---| | Both branch pages canonical to homepage | 1 | Fix today — the two most commercially important pages are not independently indexed | | /services/ blocked in robots.txt from a redesign | 1 | Fix today | | 41 pages missing meta descriptions | 3 | Batch, low priority — affects click-through, not eligibility | | 180 images missing alt text | 2 for accessibility, 3 for search | Fix on the templates; genuinely worth doing for users | | LCP 3.4s on mobile, driven by a hero image | 2 | Compress and resize the hero — one change, most of the gain | | No FAQ schema | 3 | Optional | | 6 broken links to a retired supplier | 3 | Fix when convenient |

Two items out of 214 were the audit. The rest was work, and some of it was worth doing, but none of it was the reason the branch pages were invisible.

What a technical audit cannot do

A clean technical audit is a floor, not a ceiling. It removes reasons you cannot rank. It does not supply the reasons you should.

If your pages are indexable, fast enough, mobile-parity, and correctly canonicalised, and you still do not rank, the remaining variables are the ones technical work does not touch: whether the page genuinely answers the query better than the ones above it, and whether anything else on the web references you. Both take time — a realistic view of how long organic results take to arrive is worth reading before you commit a quarter to it.

We should be straight about our own position here. This site runs a marketplace where businesses pay for a listed position and every figure is public on the live spend-ranked leaderboard. That is a different mechanism from search entirely — it does not improve your technical health, it does not make Google index anything, and it is not a substitute for a site that works. If your pages are noindexed, paying for visibility somewhere else does not fix that; it just adds a second channel on top of a broken one. Fix the site first. The methodology and audit trail behind how positions are calculated is public if you want to judge whether the paid channel is worth anything to you afterwards.

Short FAQ

How often should I run a full technical audit? Once properly, then after any migration, redesign, CMS change, or template rewrite. Between those, monitor: index coverage and status codes in Search Console will surface real problems faster than a scheduled crawl.

Does word count matter technically? No. Google's documentation states there is no preferred word count and it is not a ranking factor. Length follows from what the query needs answering.

My crawler says I have thousands of duplicate title tags. Emergency? Look at which pages. If they are paginated archives and filter combinations, it is noise, and the canonical strategy is the real fix. If they are your product or service pages, it is a template problem worth fixing properly.

Should I hire someone for this? The tier-one checks in this article take a competent developer an afternoon, and they are where the value is. Ongoing retained technical work makes sense at genuine scale or during a migration. Below that, most sites are paying for reports. If you are weighing it up, the decision framework for hiring outside SEO help is the more useful next read, and the plain-language grounding in how search works is worth having first if any of this was unfamiliar.