Author Topic: Judge Pref Update from the Nichols  (Read 2768 times)


  • Administrator
  • Sr. Member
  • *****
  • Posts: 335
Judge Pref Update from the Nichols
« on: January 04, 2014, 08:39:49 PM »
Jon Bruschke, Chris Palmer and myself have been working to implement the judging diversity policy announced prior to the swing. This is some information to help keep everyone informed about how this has developed.

Given that judge placement with diversity-enhancing concerns is still in the experimental stages, we thought it useful to share the results of the USC tournament.  To briefly summarize prelim procedures, judges were asked to self-identify as diversity-enriching, and a 1-9 preference scale was used with normal MPJ placement procedures with care taken to ensure the placement of diversity-enriching judges where preference targets were met.  Those targets were top-5 ratings and mutuality off by no more than 1 (which were somewhat relaxed in debates where neither team was eligible to clear and in JV division).

15 judges self-identified as diversity-enriching, 10.64% of the total pool.  Those judges had an average rating on the 1-9 pref scale of 4.9 compared to 5.3 for all other judges.  Both pools of judges heard over 96% of their committed rounds.  On average, the teams judged by diversity-enriching judges had 3.0 wins compared to 2.6 for other judges.  40.2% of the teams that diversity-enriching coaches judged had 4 or more wins compared to 30.9% for other judges.  The raw data are posted below.

We do not seek to over-interpret the results, but it does appear that in this case we were able to place self-identified in all their obligated debates and that they heard debates that tended to be at least as meaningful as the rest of the pool.  We also note the overall paucity of diversity-enriching judges; with only 15 such judges, it was not possible to place at least 1 judge in each of the 16 doubles debates, nor will it be possible to place 2 such judges in each octos debate as that would require at least 16 diversity-enriching judges.

We found the exercise informative and useful, and while the main difficulty at this point is the sheer lack of self-identified diversity-enriching judges, paying some attention to the placement of those critics did seem like it provided better representation in the pool and it did not appear to result in having debates compete in front of lower-rating critics in the aggregate.

        Diversity-enriching judges      15
        Percentage of pool that's diverse       10.64%
        Avg pref of diversity-enriching judges  4.9
        Total committed rounds: 42
        Total rounds judged:    41
        Percentage of commited rds judged:      97.62%
        Avg wins per team judged for diverse judges     3.0
        % teams judged with 4+ wins     40.2%
        Non-diverse judges      126
        Avg pref of non-diverse judges  5.3
        Total committed rounds (non diverse):   366
        Total rounds judged:    354
        Percentage of commited rds judged:      96.72%
        Avg wins per team judged for non-diverse judges 2.6
        % teams judged with 4+ wins     30.9%
Gordon Stables
Assistant Dean for Student Affairs
Director of Debate & Forensics
Annenberg School for Communication & Journalism
University of Southern California