org.lemurproject.ireval

Class SetRetrievalEvaluator

java.lang.Object

org.lemurproject.ireval.SetRetrievalEvaluator

public class SetRetrievalEvaluator
extends Object

Computes summary statistics over a set of queries.

Author:

Trevor Strohman, Jonathon Hare (jsh2@ecs.soton.ac.uk)

Constructor Summary

Constructors

Constructor and Description
SetRetrievalEvaluator(Collection<RetrievalEvaluator> evaluators) Creates a new instance of SetRetrievalEvaluator

Method Summary

Methods

Modifier and Type	Method and Description
Map<String,Double>	evaluateAll(String metric) Get a Map containing a particular metric value for each query.
double	geometricMeanAveragePrecision()
Collection<RetrievalEvaluator>	getEvaluators()
static int[]	getFixedPoints()
double[]	interpolatedPrecision()
double	meanAveragePrecision()
double	meanBinaryPreference()
double	meanNormalizedDiscountedCumulativeGain()
double	meanNormalizedDiscountedCumulativeGain(int documentsRetrieved)
double	meanPrecision(int documentsRetrieved)
double	meanReciprocalRank()
double	meanRPrecision()
int	numberRelevant()
int	numberRelevantRetrieved()
int	numberRetrieved()
double[]	precisionAtFixedPoints()
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

SetRetrievalEvaluator

public SetRetrievalEvaluator(Collection<RetrievalEvaluator> evaluators)

Creates a new instance of SetRetrievalEvaluator

Parameters:

evaluators -

Method Detail

getEvaluators

public Collection<RetrievalEvaluator> getEvaluators()

Returns:

Returns a collection of evaluators.

getFixedPoints

public static int[] getFixedPoints()

Returns:

the levels at which precision is calculated

precisionAtFixedPoints

public double[] precisionAtFixedPoints()

Returns:

the precision at fixed values

interpolatedPrecision

public double[] interpolatedPrecision()

Returns:

the interpolated precision values

meanAveragePrecision

public double meanAveragePrecision()

Returns:

the mean average precision; the mean of the average precision values for all queries.

meanBinaryPreference

public double meanBinaryPreference()

Returns:

the mean of the binary preference values for all queries.

geometricMeanAveragePrecision

public double geometricMeanAveragePrecision()

Returns:

the geometric mean of average precision values for all queries.

meanPrecision

public double meanPrecision(int documentsRetrieved)

Parameters:

documentsRetrieved -

Returns:

the mean of the precision values for all queries.

meanReciprocalRank

public double meanReciprocalRank()

Returns:

the mean of the reciprocal rank values for all queries.

meanRPrecision

public double meanRPrecision()

Returns:

the mean of the R-precision values for all queries.

meanNormalizedDiscountedCumulativeGain

public double meanNormalizedDiscountedCumulativeGain()

Returns:

the mean of the NDCG values for all queries.

meanNormalizedDiscountedCumulativeGain

public double meanNormalizedDiscountedCumulativeGain(int documentsRetrieved)

Parameters:

documentsRetrieved -

Returns:

the mean of the NDCG values for all queries.

evaluateAll

public Map<String,Double> evaluateAll(String metric)
                               throws IllegalArgumentException

Get a Map containing a particular metric value for each query. For instance, if metric == "averagePrecision", this returns a map where the keys are query identifiers and the values are the average precision metric evaluated for each query.

Parameters:

metric - the metric name.

Returns:

A Map containing a particular metric value for each query.

Throws:

IllegalArgumentException

numberRetrieved

public int numberRetrieved()

Returns:

The number of documents retrieved for all queries.

numberRelevant

public int numberRelevant()

Returns:

The total number of relevant documents to any of the queries.

numberRelevantRetrieved

public int numberRelevantRetrieved()

Returns:

The total number of relevant documents retrieved for any of the queries.

toString

public String toString()

Overrides:

toString in class Object