org.dkpro.statistics.agreement.coding

Class RandolphKappaAgreement

java.lang.Object

org.dkpro.statistics.agreement.AgreementMeasure

org.dkpro.statistics.agreement.coding.CodingAgreementMeasure

org.dkpro.statistics.agreement.coding.RandolphKappaAgreement

All Implemented Interfaces:

ICodingAgreementMeasure, IAgreementMeasure, IChanceCorrectedAgreement

public class RandolphKappaAgreement
extends CodingAgreementMeasure
implements IChanceCorrectedAgreement

Generalization of Bennett et al.'s (1954) S-measure for calculating a chance-corrected inter-rater agreement for multiple raters, which is known as Randolph's (2005) kappa. The measure assumes a uniform probability distribution for all raters and categories. It yields thus identical results as Bennett's S.

References:

• Bennett, E.M.; Alpert, R. & Goldstein, A.C.: Communications through limited response questioning. Public Opinion Quarterly 18(3):303-308, 1954.
• Randolph, J.J.: Free-marginal multirater kappa (multirater kappa_free): An alternative to Fleiss' fixed-marginal multirater kappa. In: Proceedings of the 5th Joensuu University Learning and Instruction Symposium, 2005.
• Warrens, M.J.: Inequalities between multi-rater kappas. Advances in Data Analysis and Classification 4(4):271-286, 2010.

Author:

Christian M. Meyer

Field Summary

Fields inherited from class org.dkpro.statistics.agreement.coding.CodingAgreementMeasure

study

Constructor Summary

Constructors

Constructor and Description
RandolphKappaAgreement(ICodingAnnotationStudy study) Initializes the instance for the given annotation study.

Method Summary

Modifier and Type	Method and Description
double	calculateExpectedAgreement() Calculates the expected inter-rater agreement that assumes a uniform distribution over all raters and annotations.

Methods inherited from class org.dkpro.statistics.agreement.coding.CodingAgreementMeasure

calculateObservedAgreement, doCalculateItemAgreement, ensureTwoRaters, warnIfMissingValues

Methods inherited from class org.dkpro.statistics.agreement.AgreementMeasure

calculateAgreement

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.dkpro.statistics.agreement.IChanceCorrectedAgreement

calculateObservedAgreement

Methods inherited from interface org.dkpro.statistics.agreement.IAgreementMeasure

calculateAgreement

Constructor Detail

RandolphKappaAgreement

public RandolphKappaAgreement(ICodingAnnotationStudy study)

Initializes the instance for the given annotation study. The study may never be null.

Method Detail

calculateExpectedAgreement

public double calculateExpectedAgreement()

Calculates the expected inter-rater agreement that assumes a uniform distribution over all raters and annotations.

Specified by:

calculateExpectedAgreement in interface IChanceCorrectedAgreement

Overrides:

calculateExpectedAgreement in class AgreementMeasure

Throws:

NullPointerException - if the annotation study is null.

ArithmeticException - if there are no annotation categories.