When an AI assistant changes models, your brand’s mention rate can move even if your website and offers stayed the same. Run the old and new models side by side when both are available to see how much the two sets of answers differ.
Write down the two model names and the dates. Use the exact same buyer questions and keep the assistant’s search mode, language, market, and brand definition fixed. Start a fresh conversation for each answer. Capture both outputs close together and save the source links. Leave incomplete pairs out and record how many you excluded.
Mark whether each answer clearly names the right brand. Record official links in a separate column. A brand mention can stay in place even when the official link disappears.
In this fictional audit, 20 questions produced a usable answer on both model versions.
7 mentioned the brand on both versions.
2 mentioned it only on the old version.
6 mentioned it only on the new version.
5 mentioned it on neither version.
The old model mentioned the brand on 9 of 20 prompts (45%). The new model mentioned it on 13 of 20 (65%). That is a 20-percentage-point increase, with 6 gains and 2 losses across 8 changed pairs. Review the lost answers and citations even if the overall rate rose.
Report the two rates, the number of matched prompts, and the counts that gained or lost a mention. If the old model is already unavailable, mark the date as a break in the series and start a new baseline. A before-and-after comparison by itself cannot tell you how much of the movement came from the model update.
Read the VisibilityOS research and measurement method
Build a two-round audit plan
Download the audit template