How Do You Measure ChatGPT, Gemini and Perplexity Recommendations?

Measure AI recommendations with a fixed monthly prompt set. Record whether each engine names the business, the position, cited pages, factual errors and competitors. Keep the location and account state consistent. Then compare the manual results with ChatGPT referral traffic, Google Search Console, server logs and qualified enquiries. One answer proves little; repeated tests show direction.

A screenshot of one favourable ChatGPT answer cannot prove that a business ranks in AI search.

The answer can change with the prompt, location, account history, available web results and product version. Gemini, ChatGPT and Perplexity also use different retrieval systems.

A useful measurement process repeats the same test, stores the evidence and connects visibility to business outcomes.

Build a prompt set from buyer questions

Start with ten to fifteen questions a customer may ask before contacting a supplier.

Use four groups:

Group	Example
Category	“Which process safety consultants serve Gauteng?”
Problem	“Who can help a factory complete a HAZOP study in South Africa?”
Comparison	“Compare three website designers for a Pretoria service business.”
Evidence	“Which South African firms show case studies for fire-risk assessments?”

Do not put the business name in discovery prompts. A branded prompt tests whether an engine can find the business after you name it. It does not test recommendation visibility.

Keep vague prompts out of the scorecard. “Best company near me” creates unstable location and preference signals. Name the city, service and buyer condition.

Fix the test conditions

Record these fields for each run:

• date and time;
• engine and product;
• logged-in or logged-out state;
• device;
• stated location;
• exact prompt;
• whether web search was active;
• answer text;
• named businesses;
• citation links.

Use a clean chat for each prompt. Previous messages can steer the answer.

A logged-out test can reduce account history, but it does not remove every location or product signal. The goal is consistency, not laboratory certainty.

Score the answer

Use a simple sheet:

Metric	Score
Business named	0 or 1
Position among named firms	1, 2, 3 or lower
Website cited	0 or 1
Correct service	0 or 1
Correct location	0 or 1
Useful description	0 to 2
Serious factual error	0 or minus 2
Competitor named	Text

Add a notes column for unsupported claims.

Do not reward a mention that gives the wrong location or service. A false recommendation can create wasted enquiries and damage trust.

A monthly visibility rate can use:

prompts that named the business / prompts tested

Keep citation rate separate:

prompts that cited the website / prompts tested

A business can earn mentions through directories or social profiles while its own website receives no citation.

Test the engines apart

OpenAI tells publishers to allow OAI-SearchBot when they want site content included in ChatGPT search summaries and snippets. It also states that ChatGPT adds utm_source=chatgpt.com to referral links.

Perplexity documents two agents. PerplexityBot crawls for search, while Perplexity-User can fetch pages during a user request. Perplexity publishes IP ranges for verification.

Google says AI Overviews and AI Mode use the same core SEO requirements as Google Search. Google needs to index the pages and allow snippets. Google reports traffic from those AI features inside the Search Console Web search type rather than a separate AI report.

Those differences change the measurement:

• ChatGPT can produce identifiable referral URLs.
• Perplexity visibility needs manual answer tests and log review.
• Google AI feature exposure sits inside broader Search Console totals.

Do not merge the three engines into one number without keeping the underlying rows.

Connect answer tests to website evidence

Manual prompts show recommendation behaviour. Website tools show visits and actions.

Check four evidence sources.

Google Analytics

Open Traffic acquisition and filter Session source for known referrals such as chatgpt.com or perplexity.ai.

OpenAI’s current publisher guidance says ChatGPT appends its own UTM source. Other AI products may send a referrer, strip it or open a page in a way that appears as direct traffic.

Mark the website’s successful enquiry as generate_lead. A referral visit without a qualified action is visibility, not revenue.

Search Console

Use the Performance report for Google impressions and clicks. Google includes AI Overviews and AI Mode in the Web search type.

Search Console does not give South African site owners a clean prompt-by-prompt AI recommendation report. Use it for query and page direction, not a claimed AI ranking.

Server logs

Search logs for documented crawler and fetcher names. Confirm that the CDN or firewall does not block them.

Crawler activity proves access. It does not prove recommendation.

Enquiry records

Add one question to the lead process:

Where did you first hear about us?

Offer ChatGPT, Gemini, Perplexity, Google Search, social media, referral and other.

Do not force one answer when a client used several sources. A person may discover the firm in Gemini, verify it on Google and submit after a colleague’s recommendation.

Run the test each month

Repeat the same core prompts on the same day range.

Add new prompts when services change, but keep the original set so the business can compare periods.

A useful report contains:

1. prompts tested;
2. mention and citation rates;
3. factual errors;
4. pages cited;
5. competitors that gained mentions;
6. referral sessions;
7. leads and lead quality;
8. changes made since the last test.

Avoid daily checking. Frequent runs create noise and encourage teams to react to one answer.

Diagnose the result before changing content

A missing recommendation can come from several places:

• the engine cannot access the site;
• the site lacks a page for the question;
• the page makes an unsupported claim;
• the business information conflicts across sources;
• stronger independent sources support competitors;
• the engine did not trigger web retrieval;
• the business has little evidence for the requested location.

Fix the specific gap.

A city page cannot repair a weak service explanation. Schema cannot replace missing proof. Repeating the business name across thin articles can make the site worse.

Keep the scorecard honest

AI recommendation systems do not offer fixed positions. No agency controls the answer.

A credible report should show losses, errors and prompts where no business received a recommendation. It should also keep raw screenshots or answer exports so another person can inspect the result.

Use the scorecard to decide which pages need clearer facts, which third-party profiles need correction and which services lack public evidence.

IDJOY’s AI search visibility service tests named engines, records the raw answers and separates technical access from recommendation evidence.