org.dkpro.statistics.agreement

Interface IChanceCorrectedDisagreement

All Superinterfaces:

IAgreementMeasure

All Known Implementing Classes:

KrippendorffAlphaAgreement, WeightedKappaAgreement

public interface IChanceCorrectedDisagreement
extends IAgreementMeasure

Generic interface to be implemented by all inter-rater agreement measures that perform a chance correction and thus distinguish between observed disagreement and expected disagreement. The basic idea is that if raters made a random decision on each annotation unit, then the final inter-rater agreement should be zero. See also IChanceCorrectedAgreement for the analogous definition of chance corrected measures with observed and expected agreement.

References:

• Artstein, R. & Poesio, M.: Inter-Coder Agreement for Computational Linguistics. Computational Linguistics 34(4):555-596, 2008.

Author:

Christian M. Meyer

See Also:

IAgreementMeasure, DisagreementMeasure, IChanceCorrectedAgreement

Method Summary

Modifier and Type	Method and Description
double	calculateExpectedDisagreement() Returns the expected disagreement of an annotation study.
double	calculateObservedDisagreement() Returns the observed disagreement of an annotation study.

Methods inherited from interface org.dkpro.statistics.agreement.IAgreementMeasure

calculateAgreement

Method Detail

calculateObservedDisagreement

double calculateObservedDisagreement()

Returns the observed disagreement of an annotation study. The observed disagreement is basically the proportion of annotation units that the raters disagree on divided by the number of units in the given study.

calculateExpectedDisagreement

double calculateExpectedDisagreement()

Returns the expected disagreement of an annotation study. The expected disagreement is the proportion of disagreement that would be expected by chance alone. The expected disagreement should be equal to the observed disagreement if each rater makes a random decision for each unit.