How to Track ChatGPT's Crawler on Your Website

Google Analytics will never show you this. The crawler does not run JavaScript, so a tag on your page never fires. Your server sees every visit, so the server log is where to look. Five steps, and two prompts you can hand to your AI.

Which visitors to tell apart

Name in the log	What it is	What to do with it
`OAI-SearchBot`	OpenAI building the index behind ChatGPT search	No visits here and you cannot be cited. Check this one first.
`ChatGPT-User`	Someone asked ChatGPT a question and it opened your page right then	The strongest signal. Do not rewrite that page's headline without checking.
`GPTBot`	OpenAI collecting training data	Not search. Blocking it does not remove you from answers.

People leave a mark too. A visit with chatgpt.com as the referrer, or utm_source=chatgpt.com in the address, is a real person who clicked your link inside an answer.

Five steps, about an hour

1. Open yourdomain.com/robots.txt

Look for a Disallow line under OAI-SearchBot, or Disallow: / under User-agent: *. Either one keeps you out of ChatGPT search. Fix that before anything else.

2. Find out what your host lets you see

Your own server or a VPS: the access log is already there (nginx, Apache).
WordPress on shared hosting: “Raw Access” logs in cPanel.
Vercel or Netlify: request logs in the dashboard, kept briefly. Add the small logger from prompt 1 below for more.
Cloudflare in front of the site: its dashboard has an AI crawler report. What it shows depends on your plan.
Shopify, Wix, Squarespace: no raw logs, so you cannot see the crawler. Ask ChatGPT ten buyer questions logged out and read its sources instead.

3. Turn on logging

If your host does not already keep a log, paste prompt 1 below into your AI and follow what it gives you.

4. Prove it works before you trust it

Fake a visit: curl -A "OAI-SearchBot" https://yourdomain.com/ (on Windows PowerShell type curl.exe). Find that line in your log. If it is not there, you are not logging, and “zero visits” means nothing.

5. Read it once a week

Count visits by crawler and by page, and keep the numbers in a sheet. Prompt 2 does the counting. The crawler does not visit on a schedule, so give it a few weeks.

One caution: anyone can type “OAI-SearchBot” into a request. For a visit that matters, check its IP address against the lists OpenAI publishes: searchbot.json and chatgpt-user.json.

Two prompts for your AI

Copy each one whole into the AI you already use. Replace the parts in square brackets.

Prompt 1: set up the logging

I want to measure which AI crawlers visit my website. I am not a developer, so give me exact steps and complete code to paste, and say which file or screen each piece goes in.

My website: [YOUR DOMAIN]
How it is built and where it is hosted: [for example WordPress on GoDaddy, Next.js on Vercel, a Python app on my own server, or "I don't know"]

First, if you cannot tell from this whether my setup lets me record incoming requests, ask me up to three short questions. If it does not (Shopify and Wix do not), say so plainly and tell me the closest alternative. Do not invent a feature.

Then set up logging that records, for every request: date and time in UTC, page address, status code, user agent and referrer.

Rules:
1. Match these names in the user agent, ignoring capital letters: OAI-SearchBot, ChatGPT-User, GPTBot, PerplexityBot, ClaudeBot, Bingbot, Googlebot.
2. For those bots, record every request, including images, CSS and JavaScript files, and also the IP address. For everyone else, record page requests only, with no IP address.
3. Also mark any visit where the referrer contains chatgpt.com or the address contains utm_source=chatgpt.com. Those are people who clicked my link inside a ChatGPT answer.
4. If logging fails for any reason, the page must still load normally.
5. No cookies, no tracking scripts, no names, emails or form contents. Crawlers do not run scripts, so a script would see nothing.
6. Keep the records somewhere I own, for at least 90 days.

Finally give me: (a) one command that fakes a visit from OAI-SearchBot and tells me exactly where that visit should appear, (b) a weekly report with visits per crawler, visits per page, and the number of ChatGPT visitors, and (c) how to switch all of it off again.

Prompt 2: read the log

Below are lines from my website's server log. Count the visits from each crawler (OAI-SearchBot, ChatGPT-User, GPTBot, PerplexityBot, ClaudeBot, Bingbot, Googlebot). Then list: the ten pages OpenAI's crawlers visited most; every page ChatGPT-User opened; every crawler request that got an error status (400 or higher); and whether any crawler asked for JavaScript or CSS files. Finish with what I should check next, in plain words. If a line is unreadable, say so. Do not guess.

[PASTE LOG LINES HERE]

What your numbers mean

OAI-SearchBot, none in 30 days	Robots.txt is fine, so check your host's firewall or bot protection. Something may be turning it away.
Any crawler getting 403, 429 or 5xx	It is being refused or the site is failing. Fix that page first.
ChatGPT-User on a page	Someone is asking about it. Protect it: no headline rewrites without a check.

We ran this on our own site before writing it down. The full log from a month of it, and everything else we tested, is in the free ChatGPT visibility guide. If you would rather have us set this up and read it for you each week, talk to us.

How to track ChatGPT's crawler on your website.