# Systems with and without web search

German title: Systeme mit und ohne Websuche

> A language model without retrieval answers from its parametric memory; a RAG system supplements it with retrieved documents (Lewis et al. 2020). Which providers retrieve when is described in the respective product documentation. How quickly training data changes is not verified here.

## Facts

- **Without retrieval:** Training knowledge
- **With retrieval:** Documents + model (RAG)

## Two paths

Answers with and without retrievalType Source of the answer What providers document
Without retrieval Training knowledge (parametric memory) Training crawlers: GPTBot, ClaudeBot; control via robots.txt
With retrieval (RAG) Retrieved documents plus model Search and user crawlers: OAI-SearchBot, ChatGPT-User, PerplexityBot, Perplexity-User, Claude-SearchBot, Claude-User

## Conclusion (recommendation)

Keep the same details on your own pages current and unambiguous so that they support the same statement on both paths. The effect per provider is not verified.

## Sources

- Lewis et al.: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (arXiv 2005.11401v4) (primary source)
- OpenAI: Overview of OpenAI crawlers (primary source)
- Perplexity Docs: Crawlers (primary source)
- Anthropic Support: Does Anthropic crawl data from the web? (primary source)

Checked on: 24 October 2026.

## Questions

### Is optimizing for Google alone enough?

That is a project decision. The providers document their own crawlers with their own rules.

### Does training change more slowly than the live index?

That is not verified.

### Which bots belong to training and which to search?

See the table in the report on Perplexity, ChatGPT and Claude.

## Related

- [Brand retrieval & RAG: what can be influenced](https://munichmedia.com/reports/aeo-brand-retrieval-rag.md)
- [AEO tracking: metrics, tools and limits](https://munichmedia.com/reports/aeo-tracking-kpis.md)
- [Entities and information gain: concept](https://munichmedia.com/reports/aeo-entity-information-gain.md)
- [Visibility in Perplexity, ChatGPT and Claude](https://munichmedia.com/reports/perplexity-chatgpt-sichtbarkeit.md)

Canonical HTML: https://munichmedia.com/reports/aeo-multi-model-oekosystem