org.dkpro.statistics.agreement.coding

Class CohenKappaAgreement

java.lang.Object

org.dkpro.statistics.agreement.AgreementMeasure

org.dkpro.statistics.agreement.coding.CodingAgreementMeasure

org.dkpro.statistics.agreement.coding.CohenKappaAgreement

All Implemented Interfaces:

ICodingAgreementMeasure, IAgreementMeasure, ICategorySpecificAgreement, IChanceCorrectedAgreement

public class CohenKappaAgreement
extends CodingAgreementMeasure
implements IChanceCorrectedAgreement, ICategorySpecificAgreement

Implementation of Cohen's kappa (1960) for calculating a chance-corrected inter-rater agreement for two raters. The measure assumes a different probability distribution for all raters.

References:

• Artstein, R. & Poesio, M.: Inter-Coder Agreement for Computational Linguistics. Computational Linguistics 34(4):555-596, 2008.
• Cohen, J.: A Coefficient of Agreement for Nominal Scales. Educational and Psychological Measurement 20(1):37-46, 1960.

Author:

Christian M. Meyer

Field Summary

Fields inherited from class org.dkpro.statistics.agreement.coding.CodingAgreementMeasure

study

Constructor Summary

Constructors

Constructor and Description
CohenKappaAgreement(ICodingAnnotationStudy study) Initializes the instance for the given annotation study.

Method Summary

Modifier and Type	Method and Description
double	calculateCategoryAgreement(Object category) Calculates the inter-rater agreement for the given category.
double	calculateExpectedAgreement() Calculates the expected inter-rater agreement that assumes a different probability distribution for all raters.
double	calculateMaximumAgreement() Computes the maximum possible value of the kappa coefficient for the provided study.
protected double	calculateMaximumObservedAgreement()

Methods inherited from class org.dkpro.statistics.agreement.coding.CodingAgreementMeasure

calculateObservedAgreement, doCalculateItemAgreement, ensureTwoRaters, warnIfMissingValues

Methods inherited from class org.dkpro.statistics.agreement.AgreementMeasure

calculateAgreement

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.dkpro.statistics.agreement.IChanceCorrectedAgreement

calculateObservedAgreement

Methods inherited from interface org.dkpro.statistics.agreement.IAgreementMeasure

calculateAgreement

Constructor Detail

CohenKappaAgreement

public CohenKappaAgreement(ICodingAnnotationStudy study)

Initializes the instance for the given annotation study. The study may never be null.

Method Detail

calculateExpectedAgreement

public double calculateExpectedAgreement()

Calculates the expected inter-rater agreement that assumes a different probability distribution for all raters.

Specified by:

calculateExpectedAgreement in interface IChanceCorrectedAgreement

Overrides:

calculateExpectedAgreement in class AgreementMeasure

Throws:

NullPointerException - if the annotation study is null.

ArithmeticException - if there are no items in the annotation study.

calculateMaximumObservedAgreement

protected double calculateMaximumObservedAgreement()

calculateMaximumAgreement

public double calculateMaximumAgreement()

Computes the maximum possible value of the kappa coefficient for the provided study. In case of balanced off-marginals (i.e., an equal disagreement for each pair of categories), the maximum kappa is 1. In other cases, it decreases with a higher discrepancy of the distribution of disagreements.

calculateCategoryAgreement

public double calculateCategoryAgreement(Object category)

Description copied from interface: ICategorySpecificAgreement

Calculates the inter-rater agreement for the given category.

Specified by:

calculateCategoryAgreement in interface ICategorySpecificAgreement

See Also:

ICategorySpecificAgreement