org.openimaj.experiment.evaluation.agreement

Class CohensKappaInterraterAgreement

java.lang.Object

org.openimaj.experiment.evaluation.agreement.CohensKappaInterraterAgreement

public class CohensKappaInterraterAgreement
extends Object

Calculates the interrater agreement for a given dataset between two (and only two) raters.

Cohen's Kappa is defined as: (PrA - PrE) / (1 - PrE) where PrA is the percentage agreement, and PrE is the probability of random agreement. PrA = agreement / total and PrE = PrX + PrY where PrX and PrY are the probability of both agreeing on X or both agreeing on Y randomly (that is, Pr(r1,x)*Pr(r2,x) ... )

Author:

David Dupplaw (dpd@ecs.soton.ac.uk)

See Also:

"http://en.wikipedia.org/wiki/Cohen's_kappa"

Constructor Summary

Constructors

Constructor and Description
CohensKappaInterraterAgreement()

Method Summary

Methods

Modifier and Type	Method and Description
static <K,A> double	calculate(Map<K,A> rater1, Map<K,A> rater2) The input should be a Map for each rater where the keys represent all the subjects that were rated by the raters and the values represent the annotations given by the raters.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CohensKappaInterraterAgreement

public CohensKappaInterraterAgreement()

Method Detail

calculate

public static <K,A> double calculate(Map<K,A> rater1,
                     Map<K,A> rater2)

The input should be a Map for each rater where the keys represent all the subjects that were rated by the raters and the values represent the annotations given by the raters. Agreement between the raters is determined by Object.equals(Object) for the INSTANCE type. Annotations for subjects which are not in both sets are ignored.

Parameters:

rater1 - The annotations from rater 1

rater2 - The annotations from rater 2

Returns:

Cohen's Kappa [0,1]

See Also:

"http://en.wikipedia.org/wiki/Cohen's_kappa"