VisibilityOS by PodPitch tracks how brands and official links appear in AI answers. Two people can read the same answer and record different results, especially when a brand name is shared by multiple companies. Use this worksheet to test whether your team applies the same rules.
Choose 12 saved AI answers. Include clear mentions, clear absences, and a few cases with ambiguous names or links.
Give both reviewers the same brand definition, official domain, answer text, and citation list. Each reviewer records Brand named and Official domain linked as separate yes or no labels, with a short note for unclear cases.
Compare the labels after both reviewers finish. Count agreement separately for each label. Read the disputed answers together, record the rule you used to resolve them, and test the rule on a fresh sample.
On 12 fictional answers, both reviewers marked Brand named yes six times and no three times. Only the first reviewer marked yes twice, and only the second reviewer marked yes once. That gives nine agreements out of 12, or 75% agreement. Repeat the same calculation separately for Official domain linked.
If an answer says “Larkfield” and links to a different business with the same name, record the disputed evidence. Check the linked domain and the business described in the answer before attributing the mention. Save that decision with the audit.
For each answer, record the prompt, assistant and date, both reviewers’ labels, the deciding quote or URL, and the resolved label. Agreement measures consistency. Use the saved answer and cited pages to check whether your shared decision is correct.
Read the VisibilityOS study and measurement method · Review the citation labeling reference · Open the public audit template