org.dkpro.statistics.agreement

Interface IChanceCorrectedAgreement

All Superinterfaces:

IAgreementMeasure

All Known Implementing Classes:

BennettSAgreement, CohenKappaAgreement, FleissKappaAgreement, RandolphKappaAgreement, ScottPiAgreement

public interface IChanceCorrectedAgreement
extends IAgreementMeasure

Generic interface to be implemented by all inter-rater agreement measures that perform a chance correction and thus distinguish between observed agreement and expected agreement. The basic idea is that if raters made a random decision on each annotation unit, then the final inter-rater agreement should be zero. See also IChanceCorrectedDisagreement for the analogous definition of chance corrected measures with observed and expected disagreement.

References:

• Artstein, R. & Poesio, M.: Inter-Coder Agreement for Computational Linguistics. Computational Linguistics 34(4):555-596, 2008.

Author:

Christian M. Meyer

See Also:

IAgreementMeasure, AgreementMeasure, IChanceCorrectedDisagreement

Method Summary

Modifier and Type	Method and Description
double	calculateExpectedAgreement() Returns the expected agreement of an annotation study.
double	calculateObservedAgreement() Returns the observed agreement of an annotation study.

Methods inherited from interface org.dkpro.statistics.agreement.IAgreementMeasure

calculateAgreement

Method Detail

calculateObservedAgreement

double calculateObservedAgreement()

Returns the observed agreement of an annotation study. The observed agreement is basically the proportion of annotation units that the raters agree on divided by the number of units in the given study.

calculateExpectedAgreement

double calculateExpectedAgreement()

Returns the expected agreement of an annotation study. The expected agreement is the proportion of agreement that would be expected by chance alone. The expected agreement should be equal to the observed agreement if each rater makes a random decision for each unit.