Strategic Themes

Using AI to Pull Strategic Theme Quotes (Presentation)

Step by step instructions on how to pull quotes using AI tools.

Strategic Theme Training Videos

Editing the back-end for event strategic themes

Creating Strategic Theme Questions - Documentation

Introduction

About The Information Provided By Vendors

Recommendations for Composing STQs

Technical Steps to Create STQs and Make Them Available

1) Create Draft STQs

3) Share The Completed Questionnaire So Others Can Access It

4) Add Link To The Completed Questionnaire To The Salesforce Opportunity

5) Add The STQs To The Product In The PeerSpot Back End

6) Update the RaaS Strategic Theme Questionnaires List

7) Update The Trello Card

Special Circumstances

Introduction

Strategic Theme Questions (STQs) play an important part in gathering a greater breadth and depth of information, whether for RaaS or event reviews. The questions are created based on information provided by our vendors in which they outline issues that are of strategic importance to them, features or benefits that differentiate their products in the marketplace; topics they want reviewers to talk about. This is in contrast to our standard questions, which won’t necessarily address topics that are of strategic importance to a given vendor.

When reviewers answer STQs they provide extra information to our readers, beyond the answers to our standard questions, and very often they provide great material for our vendors’ marketing campaigns by giving positive feedback about those strategic topics that are of importance to the vendors.

What follows are guidelines on how to create STQs. The topics covered are the following:

• An overview of the questionnaire that is completed by the vendor

• Recommendations on how to compose questions

• Technical steps to write and make STQs available

About The Information Provided By Vendors

There are a number of ways that vendors provide the information we need to create STQs:

1. They fill out a ST Vendor Questionnaire

2. They write it out in a document/email

3. They describe what their strategic themes/topics are in a call with Customer Success and we receive a Gong video snippet for that part of the call.

Let's look briefly at each of these sources.

1. ST Vendor Questionnaire

The questionnaire asks the vendor to specify/describe four topics connected to their solution.

1. Two competitive/differentiating features and

Ideal customer quotes about these competitive/differentiating features

2. One new feature/functionality and

Ideal customer quotes about this new feature/functionality

3. One data-point/metric which would be valuable to know about the solution or its use and

Ideal customer quotes for this data-point/metric

4. A top sales objection from potential customers of the solution

Ideal customer quotes that would negate this sales objection

Here is an example of a questionnaire received from a vendor called Anvilogic. This is an example of a pretty good set of replies to the questionnaire. There’s a decent amount of detail without it being excessive. The text highlighted in blue shows the key parts of the information provided by the user that were used to compose the strategic questions.

What we get varies notably from vendor to vendor. Some will go overboard and provide more than this and some will provide less. 

If a vendor provides too little information, and the deadline is not tight, it’s worthwhile asking the Customer Success person handling the vendor’s account to ask the vendor for more detail/input. 

If a vendor provides too much information, we can’t ask the reviewer to relate to all of it because there just isn’t enough time in the average review interview. Instead of trying to figure out what to include/eliminate we can either

1. Ask CS if they would like us to do our best to choose what we think are the key topics

2. Ask CS to have the vendor choose the most important topics.

2. Document/Email From Vendor

In this case, it's a matter of determining whether the information provides us what we need to create STQs or not. If not, we need to ask CS to get clarification from the vendor.

3. Gong video snippet

A Gong video snippet usually contains good information from the vendor on the topics they want us to get content about from reviewers. Generally, the format of the questions we ask the vendor on the call should mirror the questions we ask in the ST Vendor Questionnaire.

Recommendations for Composing STQs

1. The first step in composing STQs is to get a brief familiarity with the product in question. Generally, this shouldn’t require more than going to Perplexity.ai and asking “What is ProductName?” Get a sense for the type of product it is and its overall purpose. You don’t need to be an expert about the product to create STQs (and I’m living proof). 

For example, Google shows the following in its results for “What is Nasuni?”

Anvilogic is an AI-powered Detection Engineering and Hunting Platform that automates the process of creating, tuning, and deploying security detections across disparate data sources and tools without the need to centralize data or learn new coding languages.

From that information you know that Anvilogic is multi-platform security solution, and that should be enough to get started. If you need more infomation or clarification on a part of the answer provided by Perplexity, just ask it for clarification in the chat.

2. Compose the STQs in a draft document, based on the following instructions/suggestions/tips. (Most of the examples below are from the Nasuni Vendor ST form and the Nasuni Draft STQs).

• As a starting point, read through the Nasuni Vendor ST form and the Nasuni Draft STQs that were created from it. This will give you an overview of the result of the draft question-creation process and an idea of what words from the Vendor ST Form were used in the questions, and which were left out.

General Tips

• Boil down the information the vendor has provided about each feature to its most basic elements, as the basis for a question.

• Keep questions fairly simple. The longer a question gets, the less likely it is the reviewer will remember, and be able to relate to, all the elements of the question. For example, see how the first “competitive/differentiating feature” in the Nasuni Vendor ST form is split into two separate questions, the first two in the Nasuni Draft STQs.

• Avoid “yes/no” questions. We want to maximize the content reviewers provide. Ask them questions that lead them to describe things. Yes/no questions can be asked as an introduction to the more in-depth question. For example: Do you use the product's XYZ feature? If yes, … {this is where the in-depth question would go}

• Ask the reviewer to provide details and/or examples, when appropriate. This can get the reviewer to delve into the topic further and give real-life scenarios that illustrate how the product helps or doesn’t help.

• Use the word “helped” to make a question less “absolute”. For example in the “one data-point/metric” section of the Nasuni Vendor ST, it says “By eliminating the on-premises infrastructure…” The corresponding question in the Nasuni Draft STQs is “Has the solution helped to eliminate on-premises infrastructure?” The word “helped” turns it from an absolute question (which forces the user to answer whether the solution has completely eliminated on-premises infrastructure, or not) to one with room in the answer for nuance and degree.

• Keep questions neutral, as opposed to obviously “leading” to a particular answer. This often includes 

  1. Excluding marketing and “extreme” terms (like best, fastest, etc). For example, the second “competitive/differentiating feature” in the Nasuni Vendor ST form says “Nasuni can be rapidly configured to support organizational changes…” The fourth question In the Nasuni Draft STQs eliminates “rapidly” so as not to lead the reviewer to that conclusion.

  2. Asking about a feature using the “boiled down” wording about the feature: “Does Feature X do this and that? If yes, how important is this to you, and why?

  3. Using certain types of wording to introduce the question, such as 

• How easy or difficult is it to...

• How would you assess Feature X

• What is your impression of the solution’s Feature Y

• How does the solution’s Feature Z affect your IT operations?

All of these types of phrasing help to keep the question neutral and open to positive and negative responses.

By contrast, here are examples leading questions that we want to avoid:

• How easy is it to manage Feature X? (It doesn't say "easy or difficult" leading the reviewer to only consider that it is easy when answering).

• Does Feature Z improve your IT operations? (in contrast to "How does Feature Z affect your operations, the way this question is worded leads the reviewer to only consider how it improves them).

Section-Specific Tips

• For the “One new feature/functionality”, it’s often best to start with “Are you using...” and follow it with, “If yes, question about that feature....” Don’t assume the reviewer has used the new feature. (Sometimes the feature is new enough that users haven’t seen it yet or haven’t updated their product to the version with the new feature).

• For the “one data-point/metric” 

  1. When asking for metrics, always include at the end of the question, in parentheses “(Can be approximate).” For example, the third-last question in the Nasuni Draft STQs is: “Has the solution decreased capital costs because you don’t have to buy as much excess capacity? If yes, by how much has it reduced capital costs? (Can be approximate).”

  2. It’s often necessary to ask for a time period in relation to the metric in question. For example, (this is not from the Nasuni questions): “Has the solution helped to reduce costs? If yes, by how much per week or per month?” An answer that just says, “It has reduced our costs by $10,000”, without a timeframe, is not particularly helpful.

Technical Steps to Create STQs and Make Them Available

We use Trello to oversee the process of STQ-creation on the Reviews Delivery Catchall board (you may need to request access to this board). When a vendor sends us a its STQ information (via questionnaire, email, or Gong snippet) a card is created in the "Strategic Themes Requests" column. Once a card has been vetted by Davina it is moved to the “Strategic Themes Doing” column.

The info provided in each Trello card includes:

• Due date - This is the deadline by which the STQ document needs to be ready.

• The information provided by the vendor - A link to the questionnaire, the email/document, or a link to the Gong Snippet

• Link to ITCS product page(s) - This is where you will add the STQs to our backend.

• Link to Opp in SF - Each review program has an Opportunity associated with it in Salesforce. This Opportunity is where a link to the STQ questionnaire you create will be recorded.

• Pay attention to any requests/information provided at the top of the card in the Description field or in the Comments at the bottom of the card.

There are several steps involved in creating the STQ questionnaire and making the STQs available to both the interviewers who will be asking the questions and the editors who will be editing and excerpting the reviewer’s answers.

1) Create Draft STQs

** IF YOU ARE WORKING ON STQs FOR AN EVENT, ONLY CREATE 5-7 STQs (because we generally have less time at events to conduct reviews).

1. Add a shortcut to the Vendor ST Form in the Questionnaires Received folder. To do so, click the “Add shortcut to Drive” button at the top of the document, to the right of the document title (see screenshot snippet below) and then navigate to the folder to save the shortcut to the document there. This will ensure that all vendor forms that have been received are available in one place.

2. Go to the Draft Questionnaires folder in Google Drive and then open a blank Google Docs document (File > New > Document) in which you will create a draft of the STQs for a given product.

3. Give the document a title with the Vendor’s name and product + “Draft STQs” + ("Month Year"). For example: Nasuni Draft STQs or Tricentis Tosca Draft STQs (June 2024).

4. Use the above Recommendations for composing STQs to create the questions. Again, ** IF YOU ARE WORKING ON STQs FOR AN EVENT, ONLY CREATE 5-7 STQs (because we generally have less time at events to conduct reviews).

5. When you write each question in your draft document, precede it with an asterisk. Asterisks differentiate STQs from standard questions in the final questionnaire. If you write your draft questions with the asterisk, you can just copy the whole question into the final document.

6. When referring to specific product features, do your best to get the capitalization correct, according to the way the vendor presents the feature on its website.

7. When writing each draft question, highlight in light blue the key words/ideas on which you based the ST question, in the Vendor ST Form. Use the first light blue “Highlight color” in third column from the right, from the Google Docs toolbar as shown below:

8. Spell Check your draft questions.

** IF YOU ARE WORKING ON STQs FOR AN EVENT, skip to this step: Add the STQs To The Product In The PeerSpot Back End

2) Add the Draft STQs To Our Standard RaaS Questionnaire

The STQs you created need to be added to the Standard RaaS Questionnaire so that all the standard and strategic questions are in one document.

1. Go to the folder where the Standard Review Questionnaires are stored. (You may need to ask for access).

2. Create a copy of the Standard RaaS Questionnaire by right-clicking on the document and choosing “Make a copy.” Once the copy has been created a message with a link to the location of the copy will be displayed. Click on the “Show file location” link:

3. Open the copied file and rename it with the vendor’s name and product name in front of "RaaS Questionnaire" in the title, and update the month/year in parentheses. For example: Nasuni RaaS Questionnaire (July 2021).

4. Copy the STQs you created in your draft STQ document, one at a time, into the questionnaire you just renamed, as follows.

1. Each STQ should be preceded by an asterisk to identify the question as an STQ.

2. Questions that will generate answers about features of the product should be added after the standard question: “Which features have you found most valuable?” question.

Questions that will generate answers about how the product has benefited the reviewer/their company should be added after the standard question: “Can you provide an example of how this product has improved the way your organization functions?” 

Similarly, if an STQ is related to tech support, for example, place it before or after the standard tech support question, etc.

This placement helps the reviewer focus on features and then on benefits. It also helps review editors by grouping together the feature-related answers and then the benefits-related answers.

As an example, please see the Nasuni RaaS Questionnaire and look at where the questions from the Nasuni Draft STQs are positioned.

3. Double-check that you have copied each of the questions from your draft document into the final document.

3) Share The Completed Questionnaire So Others Can Access It

Move the completed questionnaire to this folder RaaS & Events - Strategic Theme Questionnaires.

4) Add Link To The Completed Questionnaire To The Salesforce Opportunity

1. In the Trello card, right-click the Link to Opp in SF and click “Copy”.

2. In your Salesforce browser, open a new tab and paste the link you just copied and press Enter.

3. Click the Details tab (see screenshot below).

4. Scroll down to the Reviews Delivery section (or just do Ctrl+f and enter “Themes”)

5. In the Strategic Themes box, click the pencil (highlighted in red, below) and then copy the title of the completed questionnaire into that box. Click Shft+Return and copy the URL (the link) of the completed questionnaire, and click "Save" at the bottom of the screen.

5) Add The STQs To The Product In The PeerSpot Back End

In this section are instructions for adding the STQs to the product in our backend. 

By adding them to the product in the backend, the STQs will be available in the review-editing template so that when an editor is editing the review he/she will be able to see the questions. As part of the editing process, the editor will excerpt and tag useful ST content provided by the reviewer.

Okay, on to the nitty-gritty:

1. In the Trello card, click on the Link to ITCS product page(s).

2. Click on the "cog" icon at the top-right of the page and click Edit Product

3. Click on the "More options" at the top-right of the page and click on "Strategic Questions"

4. Click the New Strategic Question link. If there are already questions on this page, the New Strategic Question link will be below them.

1. Copy and paste the first STQ, without the asterisk preceding it, from the final questionnaire into the “Long Label” box. If the question contains “Can be approximate” don’t copy that part.

2. In the Short Label box, create the Label that will appear in the STs dropdown box in the editing template. Ideally, try to keep it under 25 characters. It should also summarize the topic of the question. For example, the first STQ for Nasuni is

• Has the solution replaced multiple data silos and toolsets with a single global file system? If yes, how important is this to you, and why?

The Short Label is: “Replace Multiple Data Silos”. 

** IF YOU ARE WORKING ON STQs FOR AN EVENT the text of the short label MUST be preceded by the this text "EVENT - ". For example

EVENT - conf24 - Consolidate

where "conf" is a abbreviation of the event name (which was Splunk .Conf) and 24 is the year (2024), followed by the actual topic of the question ("Consolidate").

3. The “Active?” button is set to Yes by default. Doing so makes the Short Label appear in the editing template and the question itself appear in the list of questions in the editing template. Setting it to “N” removes these things from the editing template (but does not delete the question from our system).

4. Click the “Create Strategic question” button.

5. Repeat steps i - v for each STQ.

** IF YOU ARE WORKING ON STQs FOR AN EVENT skip the next step and go to "Update The Trello Card, below

6) Update the RaaS Strategic Theme Questionnaires List

1. Go to the RaaS Strategic Theme Questionnaires List.

2. Alphabetically add the company name of the vendor in the Vendor column

3. Add the product name to the Product Name column

4. Add the month and year to the Date Created column

5. Add a link to the completed questionnaire to the Link column

7) Update The Trello Card

1. Add a link to the completed questionnaire to the Trello card. 

   1. In the Comments section of the Trello card, in a new comment type @card. This will notify all member of the card of your update.

   2. Type "Please see" and copy the title of the completed questionnaire.

   3. Highlight the title of the questionnaire that you just added

   4. Go to questionnaire itself and copy its link

   5. Go back to the Trello card and do Ctrl+k

   6. Paste the link you just copied into the box.

   7. On the next line of the comment box, type "@jonathan" to inform him that the questionnaire is read and type "This ST questionnaire is ready for call center use."

   8. Save the comment to post it to the card.

2. Click the "Completed" check box at the top of the card.

3. Move the card to the "Strategic Themes Done" columns:

   1. At the top of the card, click the highlighted link shown in the screen shot here 

2. Select "Strategic Themes Done (added to site, worksheet, and SF)" and the card will be moved to that column.

Special Circumstances

Sometimes there are special circumstances or information that need to be noted at various locations, either to help interviewers or the editors.

1. Note To Editors 

When there are requirements or requests from the vendor that result in unusual questions, non-standard excerpting, or the deletion or modification of standard questions, it’s best to note this for editors. This is done by creating a Note To Editors as follows:

1. In the backend, for the product in question, create a new question (see step 6d, above). 

2. The Field Symbol should be: note_to_editors

3. The The Short Label should be “--Note To Editors--”

4. Type the contents of the note into the Long Label box.

5. Make sure the Note To Editors, once created in the list of questions, is at the top of the questions list.

Please see this product as an example.

2. Definitions

When there are technical acronyms or abbreviations that are commonly used by the vendor or users of the product/technology for which you are writing STQs, include definitions for them, for the benefit of interviewers who may not be familiar with them, as follows:

1. Just below the question that contains the acronym(s) or abbreviation(s) add the word “Definitions” and below that, one per line, the acronyns/abbreviations and their fully written out definition (ie. non-acronym/non-abbreviated version).

Here is an example:

2. The font size of the Definitions text should be one less than the rest of the text in the document.

3. The font type of the Definitions text should be Italics.

4. The highlighting of the text is done by selecting the text and choosing “light yellow #3” from the “Highlight Color” icon in the Google Docs toolbar:

3. “Please note” - Special notes for reviewers

Sometimes a ST question will require inclusion of additional information to enable the interviewer to understand the direction of the question (what the vendor is looking for) or to help the reviewer clarify or refine the question for the reviewer, or to clarify the structure of the question.

Here are some examples:

LogMeIn GoToConnect RaaS Questionnaire (August 2021)

Aruba Instant On RaaS Questionnaire (May 2021)

The font size of the “Note” text should be one less than the rest of the text in the document.

1. The Note should be enclosed by parentheses.

2. The font type of the Note should be Italics.

3. The highlighting of the text is done by selecting the text and choosing “light red #3” from the “Highlight Color” icon in the Google Docs toolbar: