# JavaScript and crawlers: content in the first HTML

German title: JavaScript und Crawler: Inhalte im ersten HTML

> Googlebot executes JavaScript in a second step with headless Chromium and indexes the rendered HTML. Google recommends server-side or pre-rendering because not all bots can run JavaScript. For the crawlers of OpenAI, Perplexity and Anthropic, the documentation read contains no statement on rendering.

## Facts

- **Google:** renders with headless Chromium
- **Google recommendation:** server-side or pre-rendering
- **Other AI crawlers:** rendering not documented
- **noindex:** may cause rendering to be skipped

## What Google documents

- Three phases: crawling, rendering, indexing. Pages with status 200 enter the render queue unless noindex is set; rendering uses headless Chromium.
- Server-side or pre-rendering is recommended because it makes the site faster for users and crawlers and not all bots can run JavaScript.
- Implement links as  . Do not use fragments ( # ) to load different content; use the History API with real paths instead.
- Soft 404 in client-side rendered apps: a JavaScript redirect to a URL that returns a 404 status, or a noindex set via JavaScript on the error page.
- If Google encounters noindex, it may skip rendering and JavaScript execution. Removing noindex via JavaScript may therefore not work.
- Content in web components must appear in the rendered HTML; check with the URL Inspection tool.

## What is documented about other crawlers

The documentation of OpenAI, Perplexity and Anthropic names purpose and robots.txt behavior of their bots, but says nothing on whether they execute JavaScript. Bot names and behavior are in the report [Brand retrieval and RAG](/reports/aeo-brand-retrieval-rag).

## Developer check (recommendation)

- View the page source (not the browser developer tools, which show the rendered DOM).
- Check: are the key statement, headings and links in the delivered HTML?
- If yes, legibility does not depend on rendering. This is the most cautious assumption and applies regardless of provider.
- For Google, additionally use the URL Inspection tool.

## Frameworks and hydration

The sources read make no statement on Next.js (App Router), hydration or other frameworks. What matters is not the framework but whether the content is in the delivered HTML; the developer check can verify that.

## Not verified

Whether the crawlers of OpenAI, Perplexity and Anthropic execute JavaScript is not stated in the documentation read. The developer check is the author's recommendation.

## Sources

- Google Search Central: JavaScript SEO basics (primary source)
- OpenAI: Overview of OpenAI crawlers (primary source)
- Perplexity Docs: Crawlers (primary source)
- Anthropic Support: Does Anthropic crawl data from the web? (primary source)

Checked on: 24 October 2026.

## Questions

### Is the Next.js App Router enough?

What matters is whether the content is in the delivered HTML. The sources read make no statement on Next.js; the result can be checked in the page source.

### Do GPTBot and PerplexityBot render JavaScript?

The documentation read makes no statement on this.

### Does hydration help crawlers?

The sources read make no statement on this. Check whether the key statement, headings and links are already in the delivered HTML.

### Can I remove noindex via JavaScript?

Possibly not: Google may skip rendering when it sees noindex. A page meant to be indexed should not contain noindex in the original code.

### How do I avoid soft 404s in a single-page app?

Redirect via JavaScript to a URL that returns a 404 status, or set a noindex via JavaScript on the error page.

## Related

- [Semantic HTML: structure for readers and parsers](https://munichmedia.com/reports/aeo-dom-semantic-html.md)
- [Headless CMS: quality checks in the build](https://munichmedia.com/reports/aeo-headless-cms-pipeline.md)
- [Visibility in Perplexity, ChatGPT and Claude](https://munichmedia.com/reports/perplexity-chatgpt-sichtbarkeit.md)
- [Technical SEO: fundamentals](https://munichmedia.com/reports/technisches-seo-grundlagen.md)

Canonical HTML: https://munichmedia.com/reports/aeo-crawler-javascript