Yes, you are all experienced teachers, so you have been writing quizzes for quite a while. Nevertheless, you may have been making the same mistakes that many teachers make! Let's just review some quiz-making concepts and see how they influence the questionnaires you make for the seminar paper (and the Final Project of the M.Ed. program).
Here are just a few of the major issues about each question on a quiz:
Does the question truly examine the student's knowledge (or lack of knowledge) of the subject matter taught/learned? For example, does a question check the student's knowledge of English (grammar, vocabulary, spelling, etc.) or does the question merely pose a challenge to the student's use of logic? Does a question about reading comprehension relate to the given passage, or can it be merely answered from knowledge coming from outside the given passage? Think about these issues for each and every question.
In a multiple-choice question, is there only ONE possible answer? Or is there another possibility, which, with a bit of explanation, invalidates the question? Remember, students are experts in discovering unexpected possible correct answers, but do your best to limit this to a minimum. After creating a quiz, let it sit for a day, or so, and then come back to it with fresh eyes to catch the problems (or at least most of them).
Fill-in-the-blank questions are particularly dangerous (particularly in digital quizzes), in terms of possible alternative answers. How does the quiz tool handle variations in capitalization, punctuation, abbreviations, synonyms, dialects, and other usage differences? (e.g., harbor/harbour, traveling/travelling, truck/lorry, USA/U.S.A./U.S./United States/United States of America/America) In fact, unless a human is grading a fill-in-the-blank answer, it's probably better not to use this format!
Questions in a quiz should proceed from the easiest to the hardest. Let the weaker students feel good about showing what they do know before they give up in frustration.
On a deeper pedagogical level, let’s think about the structure of multiple-choice questions (digital or on paper). There are "stems" and "alternatives," each of which can be short or long. The four resulting possibilities suit very different realities - particularly in timed quizzes.
Short stems and short alternatives can be good for quick vocabulary checks, from L2 to L1 (easier) or from L1 to L2 (harder and demands more time). Nevertheless, although this form usually checks isolated points of knowledge, it does not afford the possibility of checking (or learning) a L2 in context of larger chunks of language. In fact, testing words with homonyms in this format may lead to misunderstandings. For example, does the word "score" mean a physical mark, a numerical mark, twenty years, achieving success, or something else? A poorly constructed question could cause a great deal of student frustration. You must be sure that there is only ONE correct answer option offered.
Short stems and long alternatives require more time to read and understand, which will be frustrating for weaker students. Think about the specific class you are presenting the quiz to.
Long stems and short alternatives are probably the best option for most classes. In particular, if you are giving the quiz in a f2f situation, you can read the stem and alternatives out-loud to help increase comprehension. (If you do read the questions and possible answers, try to keep a neutral voice! Your students are experts in understanding your tone of voice.)
Long stems and long alternatives require quite a lot of time for the students to read, understand, and reply to. Once again, think about the class you are giving the quiz to. 5-pointers may do very well, but younger (or weaker) students may be very frustrated.
Here is a summary of a guide to constructing multiple-choice questions by Cynthia Brame of Vanderbilt University. (You may find some of these suggestions to be overly demanding for some students.)
Use meaningful language in the stem, which presents a definite problem.
Avoid presenting irrelevant material in the stem.
Try to avoid negative language in the stems.
Try to avoid partial sentences in the stems.
Avoid obviously implausible alternatives.
Write clear and concise alternatives.
Write alternatives that are mutually exclusive, avoiding overlapping options.
Write homogenous alternatives, avoiding radically different content.
Use parallel structures for the alternatives: the same grammatical structure, approximately the same length, the same linguistic register.
Do not use "all of the above" or "none of the above" or a combination as alternatives.
List the alternatives in a logical way (e.g., alphabetically, numerically, etc.).
Try to avoid giving hints in one question for the answer of a subsequent question.
Once again, think about the class you are giving the quiz to. The correct tool and the correct options will vary from class to class. You should also think about these types of questions if you decide to use Google Forms for your seminar papers or Final M.Ed. Projects.
There are many other basic questions to think about. Some universities have entire courses about exam construction. Nevertheless, given our time constraints, we'll recognize these key problems and move on.
All the considerations about constructing quizzes, listed above, are crucial for constructing a high-quality questionnaire for your seminar paper (and Final Project for the M.Ed. program). With a high-quality questionnaire, you have a fairly good chance of receiving high-quality data in the responses. On the other hand, if you have a low-quality questionnaire, you will most probably receive worthless data in the responses.
Please note: A good questionnaire is difficult to construct. The first consideration is the length. If the questionnaire is too long, the respondents will lose interest and perhaps even sabotage their answers out of boredom or frustration. In general, try to keep questionnaires to students at 10 items or less. For adults, keep questionnaires under 25 items. Given these unofficial length limitations, you have to formulate the best questions possible.
Make sure each question is comprehensible and only asks for ONE data point. A question that asks two or more questions at the same time is worthless.
Make sure the possible responses do not overlap. For example, options "1-10, 10-20, and 20-30" overlap!
Make sure that there are no gaps between the possible responses. For example, options "1-10 and 20-30 have a gap!
Make sure that the possible responses cover all possibilities. For example, options "1-10, 11-20, and 21-30" probably also need "less than 1" and "more than 30"!
Make sure that each question (with the exception of biographical items) actually helps answer the research question!
In your questionnaire, you have to get the logic correct as well as the grammar, spelling, and diction. Therefore, do not use a questionnaire before you get my approval. You want to avoid the embarrassing situation of sending out a faulty questionnaire. Not only will such a faulty questionnaire present you in a less-than-perfect light, it will lead to gathering questionable data. Poorly designed questionnaires bring back poor (and often unusable) data.
The nature of your research question may require an interview as a "research instrument."
Interviews are a form of qualitative research. The questions in your interview should prompt the interviewees to respond to the Research Question. Constructing a reasonable number of high-quality questions will be a challenge.
Ask permission from the interviewees to record the interview for transcription. There are AI tools that will turn audio into text, but you might end up spending time transcribing the recordings yourself. This is quite a lot of work!
Then analyzing the texts of interviews is an additional major challenge. (It's a lot harder than dealing with quantitative data coming from questionnaires. Yes, your questionnaire may have some qualitative questions in it, making your study a mixed-methods project.) You will have to carefully consider the responses and let categories arise from the data. When you have discovered the categories, you have to assign the responses of the interviewees to these categories in order to report on the data in a meaningful way.
There are numerous approaches to qualitative research, but learning them usually requires advanced courses. Please understand that I am not an expert in qualitative research techniques. If you go the route of qualitative research, you should bring a draft of your questions (and the resultant data) to an instructor who is an expert.
A piece of advice: For the seminar paper, which is a micro-research project, do yourself a favor. Stick with quantitative research or a mixed-method approach (with just a few "open questions"). The Final Project for the M.Ed. program is a larger research project (i.e., a mini-research project), so you may want to consider interviews for that, after you have learned the basics while doing the seminar paper.
Work on the "research instrument" for your seminar paper and send a draft to me asap (assuming you haven't done so already).
There is no work to be uploaded to MOODLE this week.