Festival Scoring

MPA DRAMA COMMITTEE
SCORING PROCEDURES

The MPA representatives shall have scoring sheets filled out with the name of each school for each judge.  These sheets should be put in a packet for each judge.  These folders should be kept securely at each site and should not leave the site for the entire weekend.

Judges should be given the packet at the initial pre-festival meeting, at which time the MPA and the MDC representatives will review the scoring rubric and procedures with the judges.  The judges may take the packets with them for each session but must return them to the MPA representative at the end of each session.  Judges are encouraged to use pencil and are prohibited from conferring with other judges.

After the last performance on Saturday night, the MPA representative will tally the final session score sheets before the final critiques and return the score sheets to the judges since the judges may wish to use their notes on the score sheets at the critiques.

After the critiques the final tally will be verified by the MPA representative, and the judge and the winners, runners-up and rank order will be determined by the final tally of the score sheets.  Up to, and including the preliminary final tally, judges are allowed to correct errors on their individual score sheets or to adjust their scores of earlier plays based on their judgements in later plays.  Judges are not allowed to confer with other judges during this process.

Each judge will individually score each play using the MDF Score Sheets. The numerical rankings of the three judges will be totaled.  After the top three scores in each class (the top 5 in the case of an all-A or all-B regional) have been obtained from the scoring rubric (raw score), the clerk will assign a #1, #2, or #3 from each judge to determine the final rank order.  No school will be given a rank score lower than #3 of the top schools.  In the case of a tie, the school that has the highest point total from the "Total Production Impact" category will be named champion. If there still remains a tie, then each judge will rank order the tied schools and the school with the lowest total will be declared the winner.

Disqualifications will be noted on the final tally sheet but neither the scores or the rank order will be changed.

After the award winners have been determined, the MPA representative will keep the master tally sheet and send it to the MPA office.  The master tally sheet may not be released to the schools, the public or the press.  The judges’ score sheets will be returned to the appropriate school.


Scoring Procedures Explained

Explanation from MDC Chair, Maura Smith (2012)

Because scores for a production can range from 0 to 100, a low score from one judge (for example a 60) could have a detrimental effect on the placing of a school that received high scores from the other two judges (for example a 90 and a 95.)  In this scenario, the school that scored the highest from two of the three judge might very well place second or third in the final tabulation because one judge scored that school in fifth or sixth place.  This did not seem fair that one judge’s opinion should have more importance than that of the other two.  In theory, that is why we have three judges: to prevent this scenario.  Our research showed that in the last few years, many final decisions were skewed by the score of one judge who differed, for whatever reason, with his or her colleagues. Ideally, the rubric was supposed to align judges’ opinions.  This has not happened.

Therefore, the MDC proposed that a second step be added to the final decision.  The rubric will still be employed as the first step. Then a rank order process will be added as a second and final step (In the case of ties, the present rules for breaking a tie would be used to determine a winner.) 

Step One:  The MPA Clerk totals the judges’ rubric scores as always.  Nothing in this process changes.  After all schools have been scored and critiqued and the judges have had the opportunity to readjust any scores they wish to readjust, the clerk adds the  scores.  He then lists all schools from highest to lowest according to the rubric scores.  This is where the process would have ended in the past. 

But now we move to Step Two:  The clerk now takes the top three schools (in each class or 5 schools in the case of a regional with only one class) and moves them on to the final round of the decision-making process.  (In case of a tie, four (or 6) schools might advance.)  The clerk then rank orders each school according to how it was ranked by each individual judge.  No school in this process can be ranked lower than third.  From the example above, the school receiving a 60, 90, and 95 would be rank ordered 3, 1, 1 for a total rank order of 5.  This school will undoubtedly place first as it now is clearly favored by a majority of the judges.  The runner-up will be determined in the same manner by rank ordering all of the schools which have advanced to Step Two. 

The attached chart shows the process more visually.  These are not real numbers but are a close approximation of what the MPA Clerk often sees. 

Our role as directors will be to educate ourselves and our students on how this new step will be used.  It may be confusing at first, for example, to see that the school with the highest rubric totals might not be the winning school in the end.  However, when we look closely at how rubric scores can be skewed by subjectivity, we can see that the rank order phase of the process lends an equity which rewards the school that has been chosen by two of the three judges.

Bear in mind as you review these changes that the process has been studied carefully and is the result of decades of trying to perfect a technique which will be fair to all.  Every voter at the spring MDC meeting agreed that the two-step process is the way to go.  Then the MPA agreed, after lengthy debate, to accept the MDC’s recommendation.

As long as we engage in competition in the arts, we must accept that the “winners” will be chosen with some degree of subjectivity.  No rank order or rubric will be 100% accurate in scoring what is a product of the heart.  However, we can always attempt to make the scoring process as transparent and fair as possible.   If you have further questions about this matter, please contact Maura Smith, MDC chair, at Skowhegan Area High School:  msmith@msad54.org.

Chart illustrating the new procedure:


Comments