Posted by Carlyn Foshee Chatfield on February 10, 2012

IT Surveys

Thanks to Alison Cruess from University of North Florida and Stuart Bradford from University of Arkansas at Little Rock for leading this morning's discussion!

We'll be talking about IT surveys - that is surveying your customers to get feedback you can use to direct your IT organization's future goals and strategies.

• • Alison says, "It's all about response."

  • We need to begin with our objectives and then develop questions. When we introduce a survey, we need to set expectations - how long will it take to complete the survey. Also include a way out, like "I don't know" as an answer.

  • Narrow the focus by the middle of the survey. As you edge towards the end of your survey, become specific. Use the hourglass structure.

  • Use branching - if you have skip logic or branching tools in your survey, use them!

  • Test your survey, use several sets of eyes. Does your survey provide the information you wanted to meet your objective?

  • Length of questions? -

  • 2-4 questions for a transactional survey is used as a quick follow up, designed to continue measuring customer service quality.

  • 5-10 questions for event evaluations

  • 10-20 questions for customer satisfaction

  • 20-30 questions for planning (future direction, product service)

  • 70-90 questions for employee satisfaction.

  • Wording of questions - compound questions are "double-barreled" and don't really work. ex: "training and career planning are avialable to me." If yes, does it mean both or just one. If no, which one is not available? Opposite problem is questions that are too general. Stay away from absolutes (always, never). Avoid negative questions or statements.

When to survey? Not beginning semester, not summer for students, but summer is great for staff. Best day to send inviation to survey is Tuesday, next best is Wednesday. Avoide Mondays and Fridays. Send the invitations in the late afternoon or early evening for students, mornings for faculty, staff.

More about hourglass - start broad, get people comfortable, go specific in middle, as their survey tolerance is waning, go broad again.

How do you use scales of value? example if you only have 4 options, you can't sit on the fence. Will you allow for a neutral point or do you want to force people to be a little satisfied or a little dis-satisfied?

How often are people choosing that neutral option? good thing to check.

How long do you leave a survey up? Depends on how long you want to collect input.

We run customer satisfaction surveys every other year.

Survey fatigue: our students are here 4 years, if they have to take the same survey every year, they may not be as likely to respond if they have to do it every year. Everybody is surveyed every other year, think about staggering them in future (students on odd year, fac/staff on even years).

• • Carlyn - Rice: Add questions to the chat

  • Carlyn - Rice: How long is "too long" for a survey?

  • Anita - UC Davis: What is the theory behind the hourglass structure? Why is this effective?

  • Anita - UC Davis: I have a couple of books on surveys, and just these few slides presented have been far more helpful.

  • Lynn-UGA: This slide is helpful. Is this a standard why to think of surveys or a breakdown Alison devloped?

  • Anita - UC Davis: What about scales on a survey and best practices

  • Anita - UC Davis: 1-4 or 1-5

  • Carlyn - Rice: how long (weeks) to leave survey up?

  • Chuck - Indiana: IU uses an annual survey process, archived and documented here - http://www.indiana.edu/~uitssur/

  • Chuck - Indiana: We are evaluating possible changes though - more frequent, targeted, smaller ones perhaps

  • Lynn-UGA: We are using Qualtrics. We just started this year with this tool. It is being used campus wide

  • Lisa: Is anyone participating in the TechQual+ survey?

  • Cathy @ Miami: We used Tech Qual+ last year and plan to use it again this spring.

  • Lisa: Any pitfalls we should be aware of, we're planning on piloting TQ+ this spring

  • Tiff - Pepperdine: We here at pepperdine use TechQual

  • Anita - UC Davis: we use surveymonkey, since our surverys are very limited

  • Lynn-UGA: We are using TechQual+ at UGA. Dr. Chester is our CIO so if there are questions be glad to answer

  • shandon: Qualtircs at UOregon.

  • Lisa: Vovici at UB (very old version)

  • Jen Gray, CU Boulder: CU-Boulder is currently implementing Qualtrics campuswide

  • Alison Cruess - Univ of North Florida: Vovici at UNF

  • Chuck - Indiana: I'm not sure what we use, our Sociology dept Center for Survey Research administers it

  • Anita - UC Davis: Question for all attendees: what is your response rate on surveys?

  • Tiff - Pepperdine: We use the TechQual survey to help shape our yearly initiatives at Pepperdine

  • Cathy @ Miami: The only "pitfall" I can think of from our first year with TechQual+ is be careful about your open ended questions. You get a list of the actual comments. It was a challenge to figure out how to make meaningful information from that data.

  • Alison Cruess - Univ of North Florida: the response rate is 23 to 29%

  • Lisa: Self select sample, around 30%

  • Cathy @ Miami: We had an 11% response rate for the TechQual+ - but we did it late in the spring semester, which was not a good idea.

  • Lisa: We also validate against departmental enrollement figures, typically engineering student are higher, arts& sciences a little lower (by 6%)

  • Chuck - Indiana: 35-40% response rate, smaple size 2000

  • Anita - UC Davis: Wow - 23% and 35-40% - those are amazing. I'm looking forward to looking at your surveys

  • Lynn-UGA: Univ of GA--We are using TechQual to students for the first time this year in March. We completed the faculty/staff in Nov. There are tools to analyze the qualitative opened questions

  • Carlyn - Rice: what's a wordal?

  • Lisa: Wow, using Wordl is is briliant!

  • Karen - EDUCAUSE: http://www.wordle.net/

  • Tiff - Pepperdine: no problem!

  • Chuck - Indiana: Never heard of wordle, thanks!

  • Lisa: How about use of incentives? What are your policies?

  • Lisa: (we used to use an iPod., etc - but tough budgets eliminated that and our response rate tanked until we used a random select for a "grab bag" giveaway. (lower cost)

  • Lynn-UGA: Is anyone participating in the EDUCAUSE ECAR Student Technology Survey. How have you used this on your campus?

  • Chuck - Indiana: what about single question poll on main IT site? Is anyone doign that?

  • Tiff - Pepperdine: we have participated in ECAR Tech survey but have not really done anything with the data.

  • Carlyn - Rice: what is a single question poll - is that like social networking sites use?

  • ITCOMM CG Leader 1: Stu: Is anyone using Banner survey?

  • ITCOMM CG Leader 1: Or Google forms?

  • Lynn-UGA: What type of questions have you asked through the Banner survey?

  • Cathy @ Miami: Great discussion - thanks again to Carlyn for keeping this going. I need to head off to my next meeting. See you all next month!

  • Chuck - Indiana: @Carlyn - yes, single question poll like you see on some sites, with immediate trends, and links to previous polls. I've floated the idea, but we haven't tried it yet.

  • Chuck - Indiana: Thanks everyone, I have to depart.

  • Tiff - Pepperdine: bye bye

  • Linda - Princeton U: Thanks so much everyone!