lenskit.metrics.ranking.LogRBP#

class lenskit.metrics.ranking.LogRBP(n=None, *, k=None, patience=0.85)#

Bases: lenskit.metrics.ranking._base.RankingMetricBase

Evaluate recommendations with the natural logarithm of rank-biased precision, aggregated as \(\log(\overline{\operatorname{RBP}})\).

This metric measures the same quantity as RBP, but carries each list’s score in log space so that the summary can be computed as \(\log(\operatorname{mean}(\operatorname{RBP}))\) rather than \(\operatorname{mean}(\operatorname{RBP})\). The two are different quantities: the summary is obtained by log-sum-exp accumulation,

\[\log\!\left(\frac{1}{n}\sum_i \operatorname{RBP}_i\right) = \operatorname{logsumexp}_i \ln \operatorname{RBP}_i - \ln n,\]

which keeps resolving for scores far below the float64 subnormal floor. With geometric weighting and patience \(p\), a single relevant item at rank \(r\) scores \((1-p)p^{r-1}\); at \(p=0.85\) and \(r=5001\) this underflows to exactly 0.0 in normal space (its log is \(\approx -814.5\)) and is therefore still finite here.

Only the geometric rank weighting used by RBP is supported. Custom RankWeight objects and weight_field are deliberately not accepted: the base log_weight() falls back to \(\ln w = \ln(e^{\ln w})\), which exponentiates the very discounts this metric exists to avoid.

Like RBP, it distinguishes two different cases that must not be conflated:

  • a list with test items but no retrieved items scores \(0\), and is reported as \(-\infty\) and retained in the aggregate (it lowers the mean);

  • a list with an empty test list has no defined score, is reported as NaN, and is excluded from both the numerator and the denominator — matching RBP and the existing accumulator behaviour.

Consequently a collection whose valid lists all score zero aggregates to \(-\infty\), while a collection with no valid lists at all aggregates to NaN rather than a fabricated zero.

Note

The summary key is provisionally named logmean and appears in collected results as LogRBP.logmean. The name is not settled upstream; see 🐞 1159.

Warning

Experimental prototype.

Parameters:

  • n (int | None) – The maximum recommendation list length.

  • patience (float) – The patience parameter \(p\), the probability that the user continues browsing at each point. The default is 0.85.

  • k (int | None) – Deprecated alias for n.

Stability:

Experimental

weight: lenskit.metrics.ranking._weighting.GeometricRankWeight#
patience: float#
measure_list(recs, test)#

Compute measurements for a single list.

Returns:

  • A float for simple metrics

  • Intermediate data for decomposed metrics

  • A dict mapping metric names to values for multi-metric classes

Parameters:
Return type:

float

extract_list_metrics(data)#

Expose the per-list log-RBP value in the collected list metrics.

Parameters:

data (float)

Return type:

float

create_accumulator()#

Create the accumulator implementing this metric’s summary semantics.

The aggregate is \(\log(\operatorname{mean}(\operatorname{RBP}))\), computed as \(\operatorname{logsumexp}_i \ell_i - \ln n_{\text{valid}}\) over the retained per-list log measurements \(\ell_i\). A measurement is retained when it is neither None nor NaN; \(-\infty\) is retained, since it encodes a genuine zero-valued RBP and contributes \(e^{-\infty} = 0\) to the sum while still counting in the denominator. That is what separates this aggregate from the mean of the logs, which would be \(-\infty\) whenever any list scores zero.

Returns n and logmean only. It deliberately reports no mean, median or std of the log values: those statistics are not meaningful once \(-\infty\) is a legitimate observation. logmean is NaN when there are no valid measurements, and \(-\infty\) when every valid measurement came from a zero-scoring list.

Storage is delegated to ValueStatAccumulator, whose add drops exactly None and NaN and keeps every other value. test_accumulator_drops_missing_keeps_neginf pins that \(-\infty\) is retained, so a future narrowing of add to finite values fails loudly instead of silently discarding zero-RBP lists.

Return type:

LogRBPAccumulator