College Policy Debate Forums
December 12, 2018, 02:55:55 AM *
Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
News: IF YOU EXPERIENCE PROBLEMS WITH THE SITE, INCLUDING LOGGING IN, PLEASE LET ME KNOW IMMEDIATELY.  EMAIL ME DIRECTLY OR USE THE CONTACT US LINK AT THE TOP.
 
   Home   Help Search Login Register  
Pages: [1]
  Print  
Author Topic: Speaker Points at GSU, KENTUCKY, HARVARD and WAKE  (Read 2098 times)
glarson
Sr. Member
****
Posts: 477


« on: November 21, 2013, 07:08:16 PM »

attached

As promised prior to the tournament, I am now releasing two sets of data for the four listed tournaments.  I ran three of them and therefore had the data natively.  Harvard provided their data to me in CAT xml format and I translated that data into STA this evening (my apologies if any datapoints didn't translate correctly).

The sheets have two tabs per tournament.  The first lists the top 50 speakers using the tiebreakers in force for the tournament.  The sheets are sortable so that you can observe the difference of using different kinds of sort strategies.  I discussed at some length in an earlier post what the difference between Z-score and ZSCR2 is.  In the former case, judge score distributions are normalized to the overall tournament population.  In the latter case, judge score distributions are normalized to the sample of ALL scores received during the tournament by the students that each individual judge hears.  In most cases, a strong conceptual and statistical argument could be made that the latter measure proves to be the best normalization, but I'm willing to discuss it with those who might have a question.  That said, by far the best solution is for there to be general consistency between judges in the range and average of scores they give (in comparison to all of the scores given to the student they hear).  The fact that that consensus has broken down (assuming it ever existed fully) probably means that tournament directors should consider statistical normalizations to award speaker awards and probably to seed teams as well (though additional tools are available for the latter task).

I'll leave it to all of you to determine what the data may or may not mean but I will make two observations, neither of which should surprise too many.  First, there was clear evidence of point inflation at Harvard.  Most of individuals involved have been called out and/or have publicly acknowledged what they did and why.  My personal opinion as one who has run hundreds of tournaments is that this is an absolutely inappropriate strategy since it further erodes the value of points as a measure of performance and erodes confidence in the outcomes of the event.  But a second conclusion is equally inescapable.  Both prior to and after the Harvard tournament, there has been a somewhat consistent group of coaches who for whatever reasons (many of which they have publicly identified) also have given points significantly higher than community averages.  This practice has also had an unacceptable impact on the reliability and validity of the speaker point aggregations.  I'm not as confident as Justin was in his post that it impacted who broke at Wake, but it has had a clear impact on speaker awards this fall.  While each coach needs to evaluate the reasons they give the points that they do, I believe that whether policy coaches or others choose to intentionally not follow community norms, we harm the integrity of the activity.

So as I said earlier, if we can't establish some common norms for assigning points, I believe that we would have to adopt statistical methods to normalize them.  Fairness dictates as much.

GARY

* Point Distributions and Speaker Rankings.xlsx (874.18 KB - downloaded 435 times.)
Logged
Pages: [1]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.21 | SMF © 2015, Simple Machines Valid XHTML 1.0! Valid CSS!