View source vs Inspect element: the SEO bug hiding between raw and rendered HTML

The product pages looked perfect in DevTools: unique titles, correct canonicals, a full set of internal links. Rankings slid anyway. Someone finally ran curl on a product URL and found a generic title, a canonical pointing at the homepage, and zero links in the body. All the good stuff was added by JavaScript after load.

"Inspect element" shows you the page after JavaScript. Crawlers start from the page before it.

TL;DR

• Raw HTML is what the server sends. Rendered HTML is the DOM after JavaScript runs.
• Search engines can render JavaScript, but rendering can be delayed, partial or skipped for some fetches, and many other crawlers (social previews, some SEO tools, AI crawlers) don't render at all.
• The risky case isn't "content only in rendered HTML". It's raw and rendered disagreeing on title, canonical, robots or links.
• Diff the two for the SEO-critical tags. Script below.
• My opinion: anything that decides indexing (robots, canonical, status) belongs in the raw HTML. No exceptions.

Two views of the same URL

# raw: what the server sent curl -sL -A "Mozilla/5.0" https://example.com/product/42 > raw.html grep -oiE '<title>[^<]*|rel="canonical"[^>]*|name="robots"[^>]*' raw.html 

For the rendered view, use a headless browser. "View source" in a browser also shows raw HTML; "Inspect" shows rendered.

The diff that matters

Comparing full HTML is noisy. Compare the handful of things that change how a page is indexed:

# pip install playwright beautifulsoup4 requests && playwright install chromium import sys, requests from bs4 import BeautifulSoup from playwright.sync_api import sync_playwright def extract(html): s = BeautifulSoup(html, "html.parser") canon = s.find("link", rel="canonical") robots = s.find("meta", attrs={"name": "robots"}) return { "title": (s.title.string or "").strip() if s.title else None, "canonical": canon.get("href") if canon else None, "robots": robots.get("content") if robots else None, "h1": [h.get_text(strip=True) for h in s.find_all("h1")][:3], "links": len({a.get("href") for a in s.find_all("a", href=True)}), "words": len(s.get_text(" ", strip=True).split()), } def raw(url): return requests.get(url, timeout=20, headers={"User-Agent": "Mozilla/5.0"}).text def rendered(url, page): page.goto(url, wait_until="networkidle", timeout=30000) return page.content() with sync_playwright() as p: page = p.chromium.launch().new_page() for url in sys.argv[1:]: a, b = extract(raw(url)), extract(rendered(url, page)) print(f"\n{url}") for k in a: flag = "" if a[k] == b[k] else " <-- differs" print(f" {k:9} raw={a[k]!r:.60} rendered={b[k]!r:.60}{flag}") 

python rawdiff.py https://example.com/ https://example.com/product/42 

Typical output on a client-rendered page:

 title raw='Shop' rendered='Blue Widget 42 – Shop' <-- differs canonical raw='https://example.com/' rendered='https://example.com/product/42' <-- differs links raw=3 rendered=118 <-- differs words raw=12 rendered=940 <-- differs 

Which differences are dangerous

Canonical differs: the worst one. If raw says "canonical is the homepage" and rendered says "canonical is this page", you've sent two conflicting signals. Whichever one wins, it's a coin flip you didn't need. Canonical problems in general are covered in How to check a canonical tag.

Robots differs: a raw noindex that JavaScript later removes. A page that's noindex in raw HTML may never get rendered at all, so the removal never counts. The reverse, JavaScript adding noindex, does take effect when rendered. How to check if a page is noindex: 5 ways has the full list of places a noindex can hide.

Title differs: usually fine for search engines that render, but social previews and many tools see the raw one. A generic raw title is also what gets indexed if rendering is delayed.

Links: raw 3, rendered 118: internal links that only exist after JavaScript are discovered later and less reliably. If they're <div onclick> instead of <a href>, they may not be discovered at all.

Words: raw 12: the content itself depends on rendering. Workable for big search engines, invisible to non-rendering crawlers.

Fixes, from cheapest to most work

1. Put indexing directives in the raw HTML: title, canonical, robots and hreflang rendered on the server, even if the rest of the page is client-side.
2. Use real <a href> links for navigation, not click handlers.
3. Pre-render or server-render key templates. Most modern frameworks do this with a config change for static or semi-static pages.
4. Never let JavaScript contradict the raw HTML. If JavaScript must set a tag, make the server send the same value.

Cross-check with the search engine's own view

The script tells you what could go wrong. Search Console's URL Inspection tool, with "Test live URL" and "View tested page", tells you what Google's renderer actually produced for one fetch. Use both: the script for every template on every deploy, URL Inspection for spot checks when a specific page behaves oddly.

Make it a regular check

Run the diff on one URL per template (homepage, category, product, article) after each deploy, and fail loudly if canonical or robots differ. Template changes are where these bugs come from. A monitor such as seotto.com that tracks title, canonical and robots tags over time catches the rest.

My take

Client-side rendering for content is a trade-off you can make. Client-side rendering for indexing directives is a bug. Search engines are good at JavaScript today, but "good" isn't "guaranteed on every fetch", and you only need one bad fetch of a canonical to have a confusing week.

Over to you

Do you check raw HTML as part of your SEO QA, or rely on rendered views and Search Console's URL inspection? I'd like to know which frameworks have surprised you with raw-vs-rendered mismatches.