org.dkpro.statistics.agreement

Class AgreementMeasure

java.lang.Object

org.dkpro.statistics.agreement.AgreementMeasure

All Implemented Interfaces:

IAgreementMeasure

Direct Known Subclasses:

CodingAgreementMeasure

public abstract class AgreementMeasure
extends Object
implements IAgreementMeasure

Default implementation of the IAgreementMeasure interface providing computational hooks for calculating the observed and the expected agreement. The values for observed and expected agreement are combined using Scott's (1955)(A_O - A_E) / (1.0 - A_E) formula (where A_O denotes the observed agreement and A_E denotes the expected agreement. See also DisagreementMeasure for the analogous definition of a measure based on the observed and expected disagreement.

References:

• Artstein, R. & Poesio, M.: Inter-Coder Agreement for Computational Linguistics. Computational Linguistics 34(4):555-596, 2008.
• Scott, W.A.: Reliability of content analysis: The case of nominal scale coding. Public Opinion Quaterly 19(3):321-325, 1955.

Author:

Christian M. Meyer

See Also:

IAgreementMeasure, DisagreementMeasure

Constructor Summary

Constructors

Constructor and Description
AgreementMeasure()

Method Summary

Modifier and Type	Method and Description
double	calculateAgreement() Returns the inter-rater agreement score.
protected double	calculateExpectedAgreement()
protected abstract double	calculateObservedAgreement()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AgreementMeasure

public AgreementMeasure()

Method Detail

calculateAgreement

public double calculateAgreement()

Description copied from interface: IAgreementMeasure

Returns the inter-rater agreement score. Unless otherwise indicated, the result is between -1 and +1, where +1 indicates perfect agreement, 0 indicates no agreement or that the agreement can be fully explained by chance, and -1 indicates perfect disagreement.

Specified by:

calculateAgreement in interface IAgreementMeasure

calculateObservedAgreement

protected abstract double calculateObservedAgreement()

calculateExpectedAgreement

protected double calculateExpectedAgreement()