Judges & Judging

Of all of the many important aspects of your role as coordinator, ensuring a quality judging experience is the most critical. The hard part is that judges are humans and humans do not always do what we hope they will. However, you can control recruitment, scheduling, and training. This section aims to increase your comfort level with these areas. 

First, you must understand the rules and the judging criteria on the evaluation form/rubric, be familiar with student entries (in general), and have thoroughly reviewed all judging instructions and the evaluation rubric. You must know the difference between a helpful comment and one that is hurtful, confusing, or just plain inappropriate. You must also know and be able to explain exactly what judges are asked to do, both logistically and with regard to entry evaluation.

As with all aspects of NHD, use the resources available to you. Consult the Affiliate agreement for the expectations regarding judging, use the template(s) from national staff, attend online coordinator training and discussion sessions about judging, and consult with your fellow coordinators. Where possible in your first year or two, attend a judges’ orientation conducted by a peer (or two). All of these resources will help you to develop and personalize your judge materials, tweaking them each year based on changes at the national level, and experiences and feedback from judges and participants. 

Start Early: Always invite more people than are needed, particularly if you likely will need multiple judging groups in some categories. Request their availability, assignment preferences, background information (see Qualifications below), and potential conflicts of interest. 

• National Contest Judge Recruitment Questions

• Utah History Day Sample Custom zFairs Judge Questions

• Minnesota History Day Judge Google Form

• Minnesota Judge Communication Plan

How and when will you know how many judges you need?: If you have advancement/registration caps on the number of entries per category that can advance to your contest from the previous level, you will be able to plan the rough number of entries that you'll have and, thus, the number of judges you'll need. Whether the previous level is schools or regionals, caps are recommended for other reasons, such as harmony among schools/regions, equality in access, and overall fairness. 

If advancement/registration caps do not already exist in your Affiliate, consult with your Advisory Committee or discuss with staff at your institution to create them. Two entries per category per school/region are a good minimum (or maximum), but you can go higher. If your schools or regions vary greatly in participation levels, you can create a formula so that more entries can attend from larger schools or regions, if desired. Just be sure to consider all options and announce any new procedures like this at the beginning of the program year.

Best practice is to have three judges per panel - so this is what you should aim for in your recruitment and logistics planning. If judges don't show or cancel, you can make a panel with only two judges, but it's better to aim high than to not have enough. Never send out a judging team with only one person.

Calculating the number of judges needed

• • A category with 1-10 entries = 1 team of 3 judges

  • A category with 11+ entries = at least 2 teams of 3 judges each + 1 final-round team of 3 separate judges (can be assigned to other categories in the first round)

  • Special awards: There are different ways to judge this. See the section on Special Awards to decide your method and calculate accordingly.

Judge Qualifications: At a minimum, a judge should be older than high school age and be comfortable working with kids. The ideal judge has:

• an open mind toward any topic and about working with students 

• a strong interest and/or experience in history and/or education 

• a positive attitude and enthusiasm for guiding students toward developing a strong passion for learning 

Background Areas: Judges can come from various personal and professional backgrounds. When creating judge panels, you will want a variety of different strengths, such as a historian, an educator, and a public historian. Recruiting from a wide pool of volunteers will help you to create balance in your judge teams. Look to recruit:

• Historians 

• Educators-School, collegiate, academic, etc.

• Pubic Historians-Museum professionals, librarians, archivists

• Librarians and media specialists

• Specialists in particular formats (e.g., web design, documentary, theater, journalists)

• Former NHD students who are now at least college-aged upperclassmen

• There will be VIPs to invite, such as sponsors, trustees, etc. 

Conflicts of Interest: Any of your judges may know students in your contest. People with conflicts may judge - and be fabulous judges. Examples include teachers who can only judge categories where they do not have competing students and judges who judged a particular category at a previous level that same year. Gather their potential conflicts when you recruit so that you can place them in categories where they DO NOT have conflicts later. View sample conflict of interest guidelines from other Affiliates. 

Be Strategic: Invite teachers and administrators in schools that you are targeting for future participation or people in organizations that you are targeting for sponsorship, etc. Serving as a judge is often all that is needed to hook someone to the program.

Strive for Diversity: As you likely are not starting from scratch, look at the demographics of your judging pool. What groups are underrepresented? Consider gender, ethnicity, age, occupation, education, geographical representation, and other factors. Compare the demographics of your judges to those of your students, both current and future. Then, consider where and how you are recruiting. Existing judges make excellent recruiters, but express your diversity goals when asking them to recruit on your behalf. Diversifying your judging pool is not easily done and it will not happen overnight. Develop a plan, stick with it, and evaluate demographics at the end of each program year. 

Should you be a judge?: No. Your job is to coordinate the event. You are too close to the entries and you know the schools and teachers; even if these are not the case, they are perceived to be. Besides, you do not have the time available in your day. 

Judge Team Captains: Always select an experienced judge as the captain. Judge captains don't overrule other judges, but they can provide important guidance, answer questions, and keep the event moving on time. If you have enough experienced folks for two of the three slots, that’s ideal, but if not, then a very strong captain can guide two novices. Judge captains usually have several additional responsibilities at the event:

• Letting you know if there are problems with any of the entries

• Turning in ranking forms and other team paperwork

• Guiding the team through the process, whether online or in-person

• If judging is done online in advance (or exclusively), scheduling the team meeting and entering the team's rankings and any special award nominations

Anticipate that a few judges will not show up. If more people sign up than are needed, start a waiting list and send them updates. You may need them. If in-person, always have some trained volunteers who can judge if needed.

What about finals teams?: Finals teams must be comprised of judges OTHER than those who judged the same category/division in the first-round. It may seem efficient to simply ask the first-round teams to review the finalists together and pick the winners. But, doing this injects bias into the process as each team will have spent more time with those they sent on to finals than with the other set. It is best to have a fresh set of people who do not know how the first-round teams ranked or evaluated the finalists. Do re-purpose judges, however, who have just a few entries in the first-round; ask them to judge finals in another category/division. 

To the greatest extent possible, identify your finals judges before the contest to avoid scrambling minutes before you need them. If you have people who want to judge finals but have a conflict if a particular school or region makes it to finals, keep them on standby and have others you can plug in, if needed. Tell your finals judges where and when to report for the final round. Be prepared with their cell phone numbers, if needed. It's usually a tight turnaround between determining the first-round advancers and launching the final round.

Samples from Affiliates

• National Contest Template for Judge Assignments and Teams

• NHD in Maine Judge Team Assignment Sample

• Virginia History Day Judge Team Assignment Sample

This section covers guidance about the broader considerations and concepts behind scheduling and judging paperwork. For mechanical aspects of this, follow the instructions in the zFairs Guide.

Scheduling

Scheduling entails assigning category-specific entry numbers to student entries and then determining judging time slots and room assignments. Generally, more than 10-12 entries per judging team are too many. Beyond that, it is tough for judges; thus, the risk is greater that each entry will not receive their full attention. As described above, divide a category into at least two judging teams and create a final round. It is okay to have some categories with a second (or final) round and others with just one.

The number of entries of each category type (see below) also is dependent upon the length of the judging period (e.g., 9:00 am-1:00 pm). Yes, you may be working in the humanities, but there is no escape from math!

Prepping the Paperwork

Using Online Evaluation Rubrics: zFairs' online judging portal enables judges to complete the evaluation forms electronically. You can then control the moment after the awards ceremony when students will be able to log in to their accounts and see their forms. If your contest is in person and you want to use online judging, ask judges in advance to bring laptops with them. You may have to rent or borrow a few extras, but it will save you a lot of paper and time. 

Using Paper Evaluation Rubrics: If you are going to print the evaluation forms for students, after scheduling, use zFairs to print entry numbers, titles, etc., onto each of the three forms for each judging room. Once the forms are all copied/printed, separate them by judge number (1, 2, or 3) and insert them into judge packets. For example, collect the entry forms for Judge 1, Junior Group Exhibit, in numerical order and put them into that judge’s packet. Mistakes made here could result in judges viewing entries that don’t match their forms. Always take the time to check these. 

How many evaluation rubrics should judges complete?: Resist the temptation to return only one evaluation rubric to the students. Three rubrics, one per judge, allow each judge to provide valuable feedback that is similar and complementary but not identical. For the final round, though, return only one form. The finals evaluation form does not have the checkboxes, and it says “finals” at the top. Since you will not be able to schedule the final round until you enter the first-round rankings at the contest, bring a printer and enough forms for each category with you and print them with the entry information of the finalists. Download the evaluation rubrics. 

Results Paperwork: At an Affiliate, Regional, or District Contest, you have several options to have judges rank entries and/or for judges to indicate which entries will advance to the next round/level of competition.

COLORS!

Using different colors of paper for each type of form will make your life easier and will significantly reduce errors. Use the official category colors for the evaluation forms (and their corresponding instruction booklets): 

• Paper: Yellow

• Exhibit: Pink

• Documentary: Green

• Performance: Blue

• Website: Purple

For the ranking and consensus forms, use one color for the first-round and another for finals. If you are running late to the awards ceremony and you have to grab the consensus forms to read on stage, having all of the finals consensus forms the same color will prevent you from accidentally grabbing those from the first-round and reading those names as the winners.

Before the Contest

Prior to the contest, whether in-person or virtual, judges must receive the theme narrative, category/rules instructions, and guidance/ expectations about writing comments and completing the evaluation form. This is especially critical if judges are doing online evaluation before the contest. If the contest will be in-person, judges also need a rough contest schedule and details about directions, parking and meals so that they arrive at the contest without any worries about when they'll be fed, when they'll have breaks, and when they're done for the day. 

Comprehensive Judge Page on Website

In addition to any emails or print mailings you may send, consider organizing your advance materials in a central place like the Judge's Main page on zFairs or another web-based page that you can add to as the contest approaches. On such a page, provide links to the materials. Just be sure to set the link permission to "anyone with the link can view." Include: 

• Contest Information:

  • Theme narrative (must), NHD's theme video (optional)

  • Sample evaluation forms and judging instruction for their category(ies)

  • Special prize list, if applicable

  • Registration links and info for workshops/practice sessions, virtual orientations; video recordings of these events, if online (after the fact)

    • Step-by-step videos for using the zFairs online judging portal

• Event Logistics:

  • A very basic schedule of the contest day, noting meals to be provided for them

  • Directions and parking information

  • Information on where and when to report and what to wear (e.g., comfortable shoes) 

Examples of Judge Pages:

• National Contest Judge's Main page. Note: The content of this page will change over time. 

• Utah History Day State Contest: Click on Judge's Main tab

• Minnesota History Day Judges Page

Email Communications

Use a mail merge to send customized information such as:

• Assignments (three or so weeks before the contest; include category, building/room, contact person, etc.)

• Reminder of Assignments (one week prior to the contest)

• See you soon! A day or so before the contest

In each of these emails, urge them to alert you ASAP if they are no longer able to judge. Filling vacancies before the contest is far easier than at the event itself.

Advance Workshops or Webinars 

Especially for new judges, a workshop or webinar that takes place outside of the contest day provides a low-stress opportunity to learn more and build confidence around their role at the event. When holding in-person events, these advance workshops or webinars are often voluntary, and cover slightly different or more in-depth orientation than each judge will attend on the day of the contest. Of course, you can provide incentives for attending. You might create a certificate or offer a door prize, if in person.   

• If you are recruiting a large number of volunteers from an organization, an advance workshop or lunch training session at their institution can be a great way to build camaraderie and a sense of excitement from that group!

Depending upon the size of your judging pool and available space, you could host an in-person workshop. Keep in mind that most of them will not be available during the weekday. That leaves evenings and weekends. Pack the time as full as you would with a teachers' workshop, but keep it to a half a day at most. 

At the Contest 

Check-in & Contest Packets

Judge check-in may be located at the general event check-in area, or in a separate area just for judges. Either way, it's best if the judge check-in is located near or immediately adjacent/in the judge room. This way you can immediately note and solve any problems with judges. You can hand a packet of these materials to each judge as they check-in, or have the materials already on tables and direct them to their seat. 

In-Person Event Orientation

NHD annually updates a PowerPoint that you and your regional coordinators are free to use as a template for your orientations. It contains the essential information and slots for you to personalize with logistical details, special awards, etc.

• View the National History Day Judge Orientation template

• Minnesota History Day 2019 Regional Judge Orientation (in-person)

• Virginia History Day 2021 State Orientation

Who should lead the orientation?  Keep this one task for yourself. You are the face of your contest, and judges need to see and hear from you. However, if you have someone with experience doing the orientation who is an excellent communicator, it’s okay for that person to do it after you provide the welcome and introduction. 

How long should it be? Ask your judges to arrive at least 15 minutes before you intend to begin the orientation so that they can visit the restroom, find the orientation room, select their muffins, and look through their packets. Allow 15 minutes for all of your logistical and procedural information. Then, allow 30-45 minutes for the contest orientation.  If desired and if time permits, split into groups to delve deeper into specific category rules or use open-ended questions to guide discussion. Regardless, at the end of the orientation, remind judges to refer to their instructions when they have any questions.

Essential components of an in-person judge orientation:

• Welcome

• Comfort Information: Schedules, building maps, location of restrooms, location and time of lunch, where and when to report to judging rooms and to the deliberation area

• Procedural Information: How and when to submit evaluation forms, ranking forms, etc.

• Role of the team captain

• The Theme: How it breaks down, whether students must adhere to all parts of it, how to look for relationship to the theme in student projects

• The Evaluation Rubric: Break down historical quality and clarity of presentation

• The Process Paper: What it should look like and contain

• The Annotated Bibliography: What it should look like and contain

• Rules for All Categories

• Category Rules: Be brief and tell them to carry their instruction booklets with them

• Rule Violations: The difference between minor and major infractions, the three reasons for disqualification, and how to handle violations

• Interviews: How to conduct them and example questions

• Importance of behaving the same with each entry, not offering ranking information, and maintaining their energy level 

• Things to Avoid: Using or even looking at their personal electronic devices, looking sleepy, personal bias regarding an entry topic, altering the schedule, thinking "aw those kids are so cute" or "those little kids couldn't possibly have made that"

• Consensus Judging: Stress that the three evaluation forms, when viewed together, should make sense to the students and their teachers

• Evaluation Rubrics: ow to complete and examples of good/bad comments

• Surveys: The evaluation of their experience, where to find their survey and how to return it

• Urgent Contact Information: For you and others who are on call during judging. Tell them to contact you without delay, in the moment, if there are any problems or questions.

• Emphasize NHD’s overall contest goals and encourage them to be professional, fair, and equal with every single entry.

• Thank you!

Discussion Questions

If your orientation follows a meal, use the dining time to your advantage so that the judges are talking about NHD at their tables. These questions work well as discussion items among small groups, preferably seated with their judging teammates. Just be sure to allow enough time to review the questions with the full group. 

• Can material found on a website be a primary source? If yes, what would be an example? If no, why not?

• How can you tell if a student/student group did the work himself/themselves?

• Is it acceptable for student-created text in an NHD entry to be written in the first person?

• What constitutes a major infraction? What is the penalty?

• What percentage of the total number of sources should be primary or secondary?

• What exactly does it mean for a topic to be “historically significant?”

• Is it okay to ask the same questions of each entry you are judging?  Perhaps you are an expert in the area researched by one of your entries. Would this change the kind of questions you would ask? If yes, how?

• How should you handle a group entry where one or more original team member(s) is/are missing?  

• What do you do when one student in a team entry dominates the question and answer period?

Problems During Orientation

What if judges discover conflicts of interest with their assigned entries? Gathering potential conflicts well before the contest will help you mitigate these in advance. As judges arrive, ask them to check the list of entries in their packets and let you know ASAP if they have a conflict. If anyone has a conflict, switch them with another judge, preferably within the same category. 

What if a judge tells you - or another judges - that they have a moral objection to a topic? A fair competition is essential to the integrity of your contest. You need to address this directly with the judge in question. Explain the importance of a fair contest to the judge and ask them if they feel that they can objectively evaluate the project using the judging rubric. If they cannot, try to change them to a different judge panel. If they feel they can be objective and do review the project in question, be sure to look carefully at their comment sheets for that project (and probably all others). 

What if a judge is a no-show? Take a deep breath and know that judge no-shows happen to all of us. 

1. Make a note. You'll want to follow-up with the judge to see if it was an issue with communication (something you can change) or an unavoidable conflict. You may want to make a note for your files, which may help you to place them in future contests or see if this is a habitual problem.

2. Triage the problem. NHD asks that all events have three judges on each panel, but sometimes this just isn't possible. Starting the day with three judges on each panel will give you the wiggle room you need to account for judge no-shows that may happen.

   • Bring in a new person. Are there other people who are not currently assigned to judge that can fill in? Someone who was going to serve as a room monitor? A teacher who doesn't have any conflicts in the category? Some contests recruit judges for "standby," who are not assigned to a specific judge team until all other judges have "checked-in" and are flexible in their assignments.

   • Avoid having judge one judge on a panel. This is from a safety of minors perspective (there's the potential to put one adult alone with a minor), but also from a quality of judging experience. You may need to pull one person off a three person team to ensure that there are enough on all the others. 

What if the judge misses the orientation? In the end, this is your call as the contest coordinator. With an experienced judge, they probably have enough background to be able to jump right in and judge, getting the logistical details from their co-judges. With a new judge, the quality of their judging (and their personal experience) depends on them having the background info they need. Get a sense of their comfort. Did they review printed materials or videos in advance? Can you move them to a team with a strong judge to guide them? If you do decide that they arrive too late (judging is in progress) or that they missed too much important information, thank them for coming, give them breakfast and their judge thank you as a gesture of goodwill.

Working the Room

Instruct your judges to return from their assigned rooms to the judges' room as soon as they finish with the last entry. You want them to all check-in and save their deliberations until they can speak securely away from participants and parents. Offering lunch or snacks gives them an incentive to come back as soon as possible. 

As they deliberate and fill out evaluation forms, make your or a key volunteer's presence known by circulating through the room. Your role is not to tell them how to evaluate or rank entries. Those must be their decisions, unless there is an accusation of one of the disqualifying violations (see section below). What you can do is offer guidance if requested or as you overhear something that your guidance can clarify. 

Also use this time to urge them to complete the consensus forms and ranking forms soonest so that you can launch final rounds and process winners for the awards ceremony. Do give them time to write their comments.

Potential Disqualifying Violations

If a judge or someone else brings a concern about a disqualifying offense to your attention, you must deal with it right away. 

• Most accusations relate to plagiarism and arise from judges who view papers and websites in advance. They have the time and Internet access to search for passages that appear to be copied word for word without attribution. 

• An accusation that an entry is reused from a previous contest year is harder to prove, unless the student willfully includes tell-tale information, such as interviews dated before the current year. If someone comes to you at the contest saying that he/she remembers a particular student doing the same topic previously, evidence gathering likely will take more time and resources than you have at that moment. If you have clear evidence, then follow the same procedure for disqualification as with plagiarism. 

• Also follow the same procedure if tampering is the accusation.

Investigation of Rule Violation that May Result in Disqualification

The NHD Board of Trustees adopted a NHD National Contest Entry Disqualification Procedure in 2015 that outlines a process for NHD staff to follow when an accusation occurs at the National Contest. This policy is summarized below and here. You are encouraged to use, or look at similar policies adopted by other Affiliates.

Investigation

When NHD staff receive an accusation of a disqualifying offense, as defined above, they will do the following:

1. Review the entry and evidence presented by the accusing party(ies).

2. Seek additional evidence or corroboration by other parties.

3. Determine the validity of the evidence.

4. If evidence of plagiarism is valid, contact the Affiliate coordinator and teacher to discuss the situation, present the evidence and seek their input in selecting an appropriate course of action.

5. Determine whether the offense was done with purpose and intent to mislead, misrepresent, deceive, or harm or if the offense was accidental, unintentional, or the result of poor guidance.

Action

1. Disqualify: If it is determined that the offense was done with purpose and intent to mislead, misrepresent, deceive or harm, the following action will be taken: The Affiliate coordinator, teacher and student(s) will be notified. The student(s) may attend the contest but the entry will be removed from the schedule, they will not interview with judges and will not receive evaluations or certificates of participation.

2. Teachable Moment: If it is determined that the offense was accidental, unintentional or the result of poor guidance, the following action will be taken: The student(s) will be permitted to stay in the contest and judges or others, as appropriate to the circumstances, will discuss the offense with him/them. Their work will be evaluated but not ranked. They will receive certificates of participation.

Problems During Judging

There is no such thing as a perfect contest. Every coordinator has stories about problems with judging and, importantly, what they learned from it. No matter how hard or well you train judges, there will be those who lecture kids, those with bias against particular topics, those who don’t get along with each other, or those who do not treat all entries equally (gushing over some entries but not others, etc.). 

Tell them to come to you with any problems or concerns without delay. Then, be prepared and alert for problems. This is one of the reasons why you do not want to assign yourself to contest tasks. Stay nimble and available to deal with the challenges that come up.

Judges' decisions are final. However, you can intervene if judges did not follow critical procedures, such as ignoring the annotated bibliographies, or if they clearly injected bias for or against entries, such as advancing an entry they liked best over one that was clearly better in terms of the judging criteria. When problems like this arise during the contest, you can step in and counsel your judges on proper procedure. If you learn of problems of this severity after the contest, you can still intervene. 

• See the Complaints section below for the investigation you'll need to conduct first. Then, call Kim for advice on how to handle the situation.

Auto-Advancement

Sometimes at your contest you will have entries that should not advance to the next level, even though they are the only ones in a category. This is more likely to happen at a small regional contest where there are three or fewer entries in a particular category though it’s also possible that there are more entries in a category but none that should advance. 

There are several different schools of thought about this situation, and what is most important is that you adopt a philosophy that applies to all students so you cannot be accused of favoring some or being too harsh on others. NHD does not have a written policy about advancement, but you can view sample contest advancement policies from other Affiliates. 

• You may be tempted to establish minimum requirements, without which an entry is “incomplete.” Do not. This seems like a good idea but could create problems when judges do not advance an entry that meets these requirements and you have to explain why. 

Whatever you decide, do your best to establish the guidelines prior to the contest to all participating teachers, parents, and students. Consider printing it in your contest program as well as on materials that go out in advance. If judges decide not to advance some entries, it will not be popular. Be prepared by disclosing the policy in advance and stand your ground.

• You or your judges do not have to advance these entries. It’s important to understand that advancing is winning a prize. It undermines the integrity of the program to reward an entry that is of very poor quality. Determining whether an entry is of sufficient quality is best handled by judges, whose decision is final. You want to be informed if judges do not advance entries, but other than providing guidance if judges are unsure about a decision, you should not be involved. Your role is to instruct judges that they do not have to advance entries by default. 

• Of course, there are shades of grey. You also must consider other factors. What do you do if all of the entries in your contest are of poor quality because it’s a newer contest? You do not want to jeopardize budding relationships with schools and teachers by shutting out entries. However, you also must consider the integrity of the contest and strike a consistent and fair balance.  

Options Other Than Advancing Entries

How you handle it at the awards ceremony is critical to how the news is received. Giving “Honorable Mention” to the students who are not advancing is also bestowing a prize. Instead, it is best to recognize the students who participated in the category by calling them forward and handing their certificates on stage. You could do this only in the category or two where this is necessary. It is better than skipping over the category or announcing that there are no “winners.” 

Complaints About Judging

Complaints happen. Most will come from parents and teachers, though you may have an unhappy judge as well. Some complaints are valid and others are not. But all must be taken seriously and answered respectfully. In advance, think about how you’ll respond to these. Handling them consistently and fairly and in a timely manner will increase your credibility and integrity.

First, ensure that all of your staff and volunteers, including judges, know that you are the point person for problems. You want to control the official response. Give everyone your cell phone number. Be clear whether you are or are not delegating this authority to anyone else.

The complaint may come at the contest and be extremely timely. For example, a teacher alerts you that a judge is harassing students or a judge knows a student or two in his/her judging room. You can immediately talk with the judges or, as the case may be and if judging has not yet begun, swap the judge with the conflict with one from another room. If judging is underway, you cannot make a swap but you can remove the judge with the conflict and leave the team with two judges.

If the complaint is about the outcome of judging, remember that the Contest Rule Book states that all judging decisions are final. You may wish you could change the final decisions, but you cannot. Explain that first to anyone who wants you to overrule a decision. Then, express your willingness to investigate the situation. Listen in a caring, non-defensive way and, provided the person is not threatening you or others, strive to put the person at ease by thanking him/her for bringing this concern to your attention. Take written notes and explain that you will investigate. 

In order to do a thorough investigation, you need information. Ask for judging forms or other evidence. If your encounter is face to face, you can do a lot of this right then. If not, then ask for documentation in writing and explain that you will respond in writing. You want to give yourself time to investigate and think about how you will respond when the heat of the moment has cooled. 

Talk with all parties concerned to sort out what happened. Assuming it’s a judging complaint, talk with each judge on the team and review their evaluation forms. Again, you are not going to undo what the judges have decided, but if you feel that the complaint is valid, you can explain that you will not invite the offending judges back next year. If you feel that the complaint is invalid, explain your position to the people who brought the concern.  

The National Office strongly recommends that you respond to all complaints in writing on letterhead, with your signature, and send your letter via postal mail. This may seem old-fashioned but it elevates the level of integrity and respect that you are showing for the person issuing the complaint. And, most significantly, it eliminates the inevitable back-and-forth that will happen with email. The complaint may initially come to you via email. Simply respond with a short message that you are thankful for the email, and you will investigate and respond via postal mail. Investigate and respond within two weeks. Please feel free to call the national staff for advice. 

How? 

At an in-person contest with paper forms, assign one or two volunteers to staff a check-out desk; their job includes scanning through evaluation sets before clearing judges to leave. They will not be able to read every single evaluation in this setting. But, they will hopefully spot those with sparse or no comments or any of the markings that will confuse students, such as a ranking with a big circle around it or higher checkmarks crossed out and replaced with lower ones. Keep some blank evaluation forms handy and ask these judges to rewrite them. 

Digital evaluations in zFairs are much easier to review, and you will also have more time to read the comments. Recruit volunteers to read the forms and log what they find so that you can reach out to judges before students can access the forms.