How to Run a Technical SEO Audit (Tools and Checklist)

A step-by-step technical SEO audit: the access to get first, the site-wide files to check, a crawl set up to match Googlebot, how to confirm issues in Search Console, how to rank them and how to write tickets developers can act on. Tool limits and prices were checked on vendor sites, and every Google behavior against Google's own documentation, in October 2026.

Sev LeoLinkedInTwitter

Sev Leo is an SEO expert and IT graduate from Lapland University, specializing in technical SEO, search systems, and performance-driven web architecture.

The technical SEO audit process at a glance

Each step produces something the next one needs. Keeping crawler findings and Google’s own reports apart stops you filing tickets for issues Google never sees.

Google Search Central’s own walkthrough covers the same ground in video form:

Technical SEO audit tools and what they cost

You need three kinds of tool: Google’s own reports (free, and the final word on what Google sees), one crawler, and a speed test. A paid suite is optional. It mainly adds bigger crawls, scheduled re-crawls and history. Technical fixes are only one way to grow search traffic; our guide to AI tools to increase organic traffic maps the others.

Prices and limits come from each vendor’s own pages (Ahrefs Free, Semrush’s Site Audit help, SE Ranking pricing) in October 2026.

How to choose the crawler:

• Under 500 URLs and the HTML comes from the server: the free version of Screaming Frog is enough.
• Bigger site, or links and content added by JavaScript: you need JavaScript rendering. That means a paid Screaming Frog license or a cloud suite that renders JavaScript. Semrush’s help page lists JavaScript rendering only for its Guru and Business SEO Toolkit tiers, so check which current plan includes it before you buy.
• You already pay for Ahrefs, Semrush or SE Ranking: use its auditor. Scheduled re-crawls that flag new issues are the main thing a cloud tool adds over a desktop crawl.

What Skribra’s free audit checks, and what it doesn’t

Skribra’s free technical SEO audit is a quick first pass, not a crawler. It fetches the homepage of the domain you enter (a full URL is cut down to its domain), plus robots.txt, and checks whether the http:// version redirects to https://. It runs 30 checks on the HTML the server returns, including canonical, meta robots, X-Robots-Tag, title, hreflang, headings, alt text, security headers, robots.txt rules and structured data, and shows a PageSpeed Insights mobile run when Google returns one.

Two limits matter. It doesn’t run JavaScript, so tags or content a script adds in the browser won’t be seen. And it checks one page: template problems on product, category or blog pages need a crawler. Use it to catch a staging noindex, a wrong canonical or a broken HTTPS redirect in seconds, then move on to the crawl.

Step 1: Get access before you crawl

Settle the scope and the access first, so every finding can be proven. You’re testing two gates: crawlability (can a crawler fetch the URL and the files it needs?) and indexability (is the page allowed into the index?). Without access you can’t show which gate failed.

Write the scope down: which hosts, subdomains and folders are in, and which are out (staging, logged-in areas, internal tools). Then name who owns each step that needs permission: adding a DNS record, exporting Search Console data, pulling server logs and approving robots.txt or sitemap changes. Audits stall when the crawl is ready but the last approval isn’t.

Verify a Domain property in Search Console

Use a Domain property, verified with a DNS TXT record. It covers every protocol and subdomain, so http, https, www and other hosts report in one place, and it gives you the Crawl Stats report for the whole domain. Add a URL-prefix property only if you also need a narrower view of one host or folder. The Search Console setup guide walks through property types, verification and user permissions.

Access checklist

• Search Console with Full or Owner access, so you can see every report and request validation.
• Server logs, or someone who can export them, to see what Googlebot actually requested.
• CMS or admin access, or a developer contact, to inspect templates, canonicals and metadata at the source.
• CDN and firewall settings, so your crawler isn’t rate-limited or blocked partway through.
• Credentials or rules for staging and logged-in areas, so you don’t crawl a login wall by accident.
• DNS access, or its owner, for verification and emergency changes.
• A named approver for robots.txt and sitemap changes. Both files can remove whole sections from search, so changes need a second pair of eyes.

Step 2: Check robots.txt, sitemaps and the homepage

Site-wide files go first, because one bad line can block or hide thousands of pages. These checks take minutes.

robots.txt

Open https://yourdomain.com/robots.txt in a browser and note the HTTP status. Google’s robots.txt documentation explains how each response is handled:

• 4xx (except 429): Google treats the site as if there were no robots.txt and crawls everything.
• 5xx: Google stops crawling the whole site for the first 12 hours, then uses the last cached copy for up to 30 days. A robots.txt that errors is a site-wide crawl problem, not a small one.
• Size: Google reads only the first 500 KiB. Rules after that are ignored.
• Caching: Google usually caches the file for up to 24 hours, so a fix doesn’t take effect instantly.

Then read the rules. Look for a Disallow: / left over from staging, blocked folders that hold pages you want in search, and blocked CSS or JavaScript files that Google needs to render the page. If the file is a mess, rebuild a clean one with the robots.txt generator and compare.

Remember what robots.txt can’t do. It controls crawling, not indexing. Google says a blocked URL can still be indexed if other pages link to it, and Google can’t see a noindex on a page it isn’t allowed to fetch. To keep a page out of search, allow crawling and use noindex in a meta tag or an X-Robots-Tag header.

XML sitemaps

Find the sitemap (usually listed in robots.txt or at /sitemap.xml) and run it through the sitemap validator. It counts URLs, follows nested sitemap indexes and flags relative URLs, duplicates and missing lastmod dates. Google’s sitemap rules to check against:

• Each sitemap file can hold up to 50,000 URLs or 50MB uncompressed. Bigger sites split it and use a sitemap index.
• URLs must be absolute (https://example.com/page, not /page).
• Google ignores priority and changefreq. It uses lastmod only when it’s consistently accurate.

Keep the sitemap export for Step 3. Every URL in it should return 200, be indexable and be its own canonical. Any URL that doesn’t is a finding, and any indexable page missing from it is a question to ask.

Homepage quick check

Run the homepage through the free audit described above, or view its source and the response headers in your browser’s developer tools. You’re looking for the expensive mistakes: a noindex meta tag or X-Robots-Tag header, a canonical pointing at another domain or at staging, http that doesn’t redirect to https, or a non-200 status. If the homepage has one of these, check the templates next, because the same code usually runs on every page.

Step 3: Crawl the site the way Google does

A technical audit is only as good as the crawl behind it. Set up your main crawl to match how Google discovers and evaluates pages, or you’ll spend time on issues that never affect indexing.

Start with the user agent. Google uses the mobile version of a site’s content, crawled with the smartphone agent, for indexing and ranking. Crawl with a Googlebot Smartphone user agent and with your real robots.txt and login rules in place.

Next, decide whether you need JavaScript rendering. Google crawls, then renders, then indexes pages that rely on JavaScript. If your navigation, internal links, canonicals or main content are added by scripts, an HTML-only crawl will miss or misread them. View the page source (not the inspector) of a key template: if the content and links are there, an HTML crawl is fine. Sites on a headless CMS should check this first; here’s how rendering mode affects indexing on headless sites.

Keep robots.txt respected for the main crawl. If you want to see what robots.txt hides, run a second crawl that ignores it, label it diagnostic and keep its data separate.

Screaming Frog setup

Most of these settings need a paid license (see the free-version limits in the tool table above). The free version runs the default crawl, capped at 500 URLs.

1. Choose the rendering mode. Leave it on text-only HTML for server-rendered sites. Switch to JavaScript rendering when links or content depend on scripts. Screaming Frog then renders pages in headless Chromium and reads the rendered HTML.
2. Set the user agent to Googlebot Smartphone, to match mobile-first indexing.
3. Respect robots.txt for this crawl, and write that down in the audit notes.
4. Handle logins on purpose. Either give the crawler credentials so it reaches real content, or exclude logged-in areas so the data isn’t full of redirects to a login page.
5. Limit the scope before you start. Restrict the crawl to the hosts and folders in scope, and exclude traps like logout links, calendars and endless filter or sort parameters.
6. Add your XML sitemap to the crawl, so the report can show sitemap URLs that no internal link reaches (orphan pages) and linked URLs missing from the sitemap.
7. Save the configuration with the crawl, so the next crawl after fixes uses identical settings.

What to look for in the crawl

Sort findings by pattern, not by URL. Fifty broken links that come from one footer template are one issue.

A crawler finds template and status-code problems, not thin or off-intent copy. For content-level checks, see our comparison of on-page SEO tools vs a manual audit.

Step 4: Confirm what Google sees in Search Console

The crawler is where you find problems. Google’s tools are where you prove them. Before a finding becomes a ticket, check that it shows up in the reports Google uses to fetch, render and judge your pages.

Page indexing report

Start with Indexing > Pages. It lists every reason Google gives for not indexing URLs it knows about, with sample URLs for each. Match those reasons to your crawl: a template your crawler flagged for noindex should appear under “Excluded by ‘noindex’ tag”, and canonical problems appear under the duplicate statuses. The guide to pages that aren’t indexed explains what each status means and how to fix it, so this guide doesn’t repeat it.

URL Inspection workflow

Use the URL Inspection tool on a few sample URLs per issue, not on every URL.

1. Read the indexed result first. It shows the last version Google indexed, the user-declared canonical and the Google-selected canonical. The Google-selected canonical appears only here, not in the live test.
2. Run Test Live URL. This fetches the page now, so it shows whether a fix you just shipped works.
3. Open View tested page. You get the rendered HTML, a screenshot (live test only), the HTTP response headers, the JavaScript console output and the page resources that loaded or failed.
4. Compare the raw and rendered HTML. If the crawler flagged a missing title, canonical or link, check whether it’s missing after rendering too, or only in the initial HTML.
5. Save the evidence. The screenshot, the relevant HTML snippet, the header and any failing resource URL go straight into the ticket.

For structured data, test a sample URL in the Rich Results Test to see whether Google reads the markup and whether required properties are missing.

Crawl Stats

Find it under Settings > Crawl stats. It works only on root-level properties, meaning a Domain property or a URL-prefix property at the root of the site. Google’s help page says sites with fewer than about a thousand pages usually don’t need it, but it’s the closest thing to server logs if you can’t get them.

Check three things:

• Host status. Problems with robots.txt fetching, DNS or server connectivity in the last 90 days.
• Responses. A rising share of 404s or 5xx errors.
• Total requests. A sudden drop in crawling.

Then classify the problem. It’s blocked when robots.txt fetches fail or a new Disallow rule appeared. It’s overloaded when responses shift toward server errors and timeouts. Blocked means fixing rules; overloaded means fixing capacity or firewall settings. For large sites, the list of crawl budget mistakes shows how to spot each one in Crawl Stats.

Core Web Vitals: field data first, lab data to debug

Core Web Vitals are measured on real users. The thresholds for a good result are LCP at 2.5 seconds or less, INP at 200 milliseconds or less and CLS at 0.1 or less, measured at the 75th percentile of page loads.

Lighthouse can’t measure INP, so it reports Total Blocking Time as a stand-in. Google’s INP guide calls TBT a reasonable proxy but “not a substitute” for INP. A good lab TBT doesn’t prove you pass INP, so judge pass or fail on field data.

Low-traffic pages often have no field data. PageSpeed Insights then falls back to data for the whole site, or shows no real-user data at all. When the crawler says a page is slow but field data is fine, or the reverse, treat it as a question to settle before you write a ticket.

How to audit a site built on Wix, Squarespace, Webflow or Shopify

On a hosted website builder you check the same things, but the list of things you can fix is shorter. The platform controls the hosting, the rendering and much of the HTML. Find out what the site runs on with the CMS detector, then check what the platform lets you change.

What changes in the audit:

• Logs. Hosted builders generally don’t hand you raw server logs, so Crawl Stats becomes your log substitute.
• Templates. Almost every issue lives in a page type: product, collection, blog post, tag page. Report it once per template, with sample URLs.
• Platform defaults. Look for tag, filter and internal search pages that the builder publishes on its own, and for the platform’s staging or preview domain being crawlable.
• Owners. The fix may be a setting, an app or theme code rather than a developer task. Put the exact setting path in the ticket.

The indexing guide linked in Step 4 also covers the platform settings that most often keep pages out of Google on WordPress, Webflow and Shopify.

Step 5: Prioritize what you found

Rank each finding by three questions: does it stop pages that matter from being crawled, rendered or indexed? How many URLs or templates does it affect? How much work is the fix? Then sort into tiers.

Fix tier 1 first, even if it’s one line of code. A single template fix in tier 2 often clears more URLs than every tier 3 item put together. Tier 4 is worth doing when it’s cheap, but it shouldn’t hold up anything above it.

Step 6: Turn findings into developer tickets

A good ticket lets a developer reproduce the problem, fix it and prove the fix without asking you anything. Each one needs a pattern, proof, an owner and a test.

• Pattern: the template or component, the URL count and 3 to 5 sample URLs from the crawl export.
• Proof: the response header, rendered HTML snippet or URL Inspection screenshot that shows the problem.
• Owner: a developer, DevOps, or whoever controls the CMS setting. SEO stays the reviewer.
• Acceptance test: what passes after the fix, such as the live test in URL Inspection and a clean re-crawl of the template. For crawl problems, add Googlebot hits from server logs. Screaming Frog Log File Analyser reads up to 1,000 log events free, or costs £99 a year.

If the same finding keeps coming back after releases, learn how to turn audit findings into SEO checks in CI.

A template you can paste into Jira, Linear or GitHub:

Title: [Template] [issue] ([count] URLs)
Priority: Tier 1 / 2 / 3 / 4
What's wrong: one sentence in plain language
Where: template or component name, sample URLs (3 to 5)
Evidence: crawl export link, URL Inspection screenshot, header or HTML snippet
Expected: what the page should return or contain
Acceptance test: live test in URL Inspection passes for the sample URLs;
                 re-crawl of the template shows 0 instances
After release: click Validate fix in Search Console (if listed there)
Owner: name    Reviewer: SEO owner

Technical SEO audit checklist

Copy this into a spreadsheet, add a Status column (Pass, Fail or N/A) and an Evidence column with links, and reuse it for every audit.

After the fixes ship: re-test and re-audit

An audit isn’t finished when tickets are filed. It’s finished when the fixes are confirmed. For each shipped ticket, run the live test in URL Inspection on the sample URLs, re-crawl the affected template with the saved configuration, and click Validate fix for any issue listed in the Page indexing report. Google says that validation typically takes up to about two weeks, sometimes longer. Core Web Vitals fixes take time to show too, because field data in PageSpeed Insights covers the previous 28 days.

Run a new audit after any migration, redesign, CMS or theme change, or switch of hosting or CDN, and on a regular schedule that matches how often your site changes. For the wider picture beyond technical fixes, Skribra’s SEO guide covers content, links and measurement.