X- The type of accumulator used by this metric.
public interface Metric<X> extends Closeable
Metrics use contexts to track and accumulate data for experimental conditions. For each experimental condition (algorithm / data set pair), the evaluator will do the following:
createContext(Attributed, TTDataSet, Recommender).
measureUser(TestUser, Object), passing in the context in which the user should be evaluated.
The context will general consist of accumulators for the aggregate results reported by a metric over the entire experimental condition, such as the average of all user measurements. It may also contain additional relevant information, such as anything needed from the algorithm and data set for the measurements, or additional output tables for recording extra data.
Metrics themselves are generally stateless, with all state contained in the context. In this
case, there is a single instance of the metric, or an instance per parameterization, and the
Closeable.close() method is a no-op. Some metrics need state or output resources; such metrics
should define a
MetricFactory to instantiate them,
and clean up and release resources or state in
Metrics may be used from multiple threads. LensKit does not currently use multiple threads with the same context, but that may change in the future.
AbstractMetric provides a base implementation of this interface that allows user and
aggregate measurements to be defined in plan Java objects, so metrics do not need to handle
creating table rows themselves.
|Modifier and Type||Method and Description|
Create the context for an experimental condition (algorithm/data set pair).
Get labels for the aggregate columns output by this evaluator.
Get the aggregate results from an accumulator.
Get labels for the per-user columns output by this evaluator.
Measure a user in the evaluation.
@Nullable X createContext(Attributed algorithm, TTDataSet dataSet, Recommender recommender)
algorithm- The algorithm.
dataSet- The data set.
recommender- The LensKit recommender, if applicable. This can be null for an external algorithm that does not provide a LensKit recommender.
measureUser(TestUser, Object). If the metric does not accumulate any results, this method can return
user- The user to evaluate.
context- The context for the active experimental condition.