What is server-side rendering for crawlers?
Server-side rendering for crawlers is the practice of making sure the HTML a crawler downloads already contains the page's full content, built on the server rather than in the browser. A crawler that never runs JavaScript still gets every heading, paragraph, link and meta tag.
The difference shows up in the first response. A client-side rendered product page might send this:
<div id="app"></div>
<script src="/bundle.js"></script>The same page with server-side rendering sends this:
<h1>Trail Runner 2 Shoe</h1>
<p>Lightweight trail shoe with a 4 mm drop. $129.</p>
<a href="/shoes/trail/">More trail shoes</a>
<script src="/bundle.js"></script>Both pages can look identical to a visitor. Only the second is readable by a crawler that does not execute JavaScript. "Server-side rendering" is also the name of one specific technique (building HTML on each request), but for SEO the point is the outcome: complete HTML in the response, whether it was built per request or ahead of time.
Why does server-side rendering matter for AI crawlers?
Server-side rendering matters for AI crawlers because most of them do not run JavaScript, so server-built HTML is the only content they see. Vercel's December 2024 analysis found that none of the major AI crawlers from OpenAI, Anthropic, Meta, ByteDance or Perplexity executed JavaScript [2]. ChatGPT's and Claude's crawlers did download JavaScript files, but did not run them [2].
That makes server-built HTML a precondition for being cited from those crawls. A page that relies on JavaScript rendering for its main content can be indexed and ranked by Google, which renders pages, while GPTBot, ClaudeBot and PerplexityBot receive an empty shell. No amount of on-page optimization helps a page a crawler cannot read. Vercel's own recommendation is to "prioritize server-side rendering for critical content" because ChatGPT and Claude do not execute JavaScript [2].
Does Google need server-side rendering?
Google does not strictly need server-side rendering, because Google renders JavaScript with a headless Chromium browser [1]. Google still recommends it: "Server-side or pre-rendering is still a great idea because it makes your website faster for users and crawlers, and not all bots can run JavaScript" [1].
Server-rendered HTML also removes the wait between crawling and rendering. Google queues pages for rendering, and a page "may stay on this queue for a few seconds, but it can take longer than that" [1]. Content in the first HTML response does not wait.
What is the difference between server-side rendering, dynamic rendering and prerendering?
Server-side rendering sends the same finished HTML to everyone, dynamic rendering sends finished HTML only to bots, and prerendering builds the HTML ahead of time instead of on each request.
| Approach | Who gets server-built HTML | Google's view |
|---|---|---|
| Server-side rendering (per request) | Every visitor and every crawler | Recommended [3] |
| Static rendering (pre-built at deploy) | Every visitor and every crawler | Recommended [3] |
| Hydration | Everyone gets server HTML, then JavaScript adds interactivity | Recommended [3] |
| Dynamic rendering | Only detected bots; users get the JavaScript app | "A workaround and not a recommended solution" [3] |
Google defines dynamic rendering as a server that "detects bots that may have problems with JavaScript-generated content and serves a server-rendered version without JavaScript to these bots while showing the client-side rendered version of the content to users" [3]. Google says it is a workaround "because it creates additional complexities and resource requirements" [3].
Dynamic rendering has a second weakness for AI search: it only helps the bots it detects. A rendering service that recognizes Googlebot but not a new AI crawler sends that crawler the empty shell. Sending the same HTML to everyone avoids the bot list altogether.
Is serving rendered HTML to bots cloaking?
Serving rendered HTML to bots is not cloaking as long as the content matches what users see. Google says "Googlebot generally doesn't consider dynamic rendering as cloaking" when it "produces similar content" [3].
The line is the content, not the method. Google says using dynamic rendering "to serve completely different content to users and crawlers can be considered cloaking", giving the example of a page about cats for users and a page about dogs for crawlers [3]. Adding extra text, links or keywords to the bot version only is the fastest way to cross that line. Universal server-side or static rendering cannot cross it, because there is only one version.
How do you check what crawlers receive?
To check what crawlers receive, fetch the page without running JavaScript and look for the main content in the response. If the content is there, every crawler can read it.
Three checks, from quickest to most thorough:
- View Page Source in a browser (not Inspect) and search for a sentence from the main content, a price and an internal link.
- Fetch it as a bot from a command line, and compare with a normal fetch:
curl -s -A "GPTBot" https://example.com/page | grep -c "a sentence from the page"
curl -s https://example.com/page | grep -c "a sentence from the page"Both counts should match and be above 0. A difference means the site serves bots and people different HTML. A 0 for both means the content needs JavaScript.
- Inspect in Search Console. The URL Inspection tool and Rich Results Test show the HTML Google rendered, plus resources and JavaScript errors [4].
Also check that the links in the server HTML are real <a href> links, since Google can generally only follow those [6], and that title, canonical and robots tags are in the first response rather than added by scripts [1].
How is searchinsider.io rendered?
Searchinsider.io is built with Astro, and every page is pre-built as static HTML at deploy time and served from Cloudflare. No article or glossary content depends on JavaScript running in the browser.
That has three practical effects for crawlers:
- Every crawler gets the full page. Googlebot, AI crawlers and any bot that ignores JavaScript receive the same complete HTML, including this entry.
- There is no bot detection to maintain. Because every visitor gets the same file, there is no dynamic rendering layer and no list of user agents to keep current.
- Nothing waits in a render queue. Content, internal links and meta tags are in the first response.
Static rendering suits content that changes when the site is deployed, such as articles and glossary entries. Pages that change per visitor, such as account pages or live stock levels, need per-request server rendering instead, or a static page with the changing part added on top.
How has Google's advice on rendering for crawlers changed?
Google moved from offering dynamic rendering as a way to serve crawlers to calling it a workaround and recommending that every visitor get server-built HTML. The AI crawler data from 2024 made the same advice matter beyond Google.
February 6, 2024: JavaScript guidance trimmed
Google reviewed its JavaScript SEO documentation on February 6, 2024, to "remove outdated or unnecessary information" [5]. Google's dynamic rendering page now says dynamic rendering "was a workaround and not a long-term solution" and recommends server-side rendering, static rendering or hydration instead [3].
December 17, 2024: AI crawlers shown not to render
Vercel published measurements showing the major AI crawlers did not execute JavaScript, while Googlebot and AppleBot did render it [2]. From that point, server-built HTML was the difference between being readable or not to most AI crawlers.
What does server-side rendering not fix?
Server-side rendering does not fix crawlers that are blocked, content hidden behind interaction, or pages that are thin to begin with. Specifically:
- Blocked bots. A robots.txt rule or a firewall setting that blocks an AI crawler stops it before rendering matters. See AI crawler and crawler allowlist.
- Content that loads on click or scroll. If a tab, accordion or "load more" fetches its text only after interaction, that text is still missing from the server HTML.
- Status codes. A server-rendered "not found" page that returns 200 is still a soft 404 [4].
- Quality. Rendering makes a page readable. It does not make it worth citing.
Should you switch to server-side rendering?
Switch to server-side or static rendering if your main content, internal links or meta tags only appear after JavaScript runs, and you want to be visible to AI crawlers as well as Google. Most modern JavaScript frameworks offer a server or static rendering mode, so the change is usually a configuration and hosting decision rather than a rebuild.
Start with the pages that earn traffic or should be cited: articles, product and category pages, and definitions. Run the raw HTML check above before and after the change. If the main content is already in View Page Source, rendering is not your problem and the effort is better spent elsewhere.