lenskit.data.ItemList#

class lenskit.data.ItemList(source=None, *, ordered=None, vocabulary=None, _init_dict=None, _init_array=None, **kwargs)#

Bases: object

Representation of a (usually ordered) list of items, possibly with scores and other associated data; many components take and return item lists. Item lists are to be treated as immutable — create a new list with modified data, do not do in-place modifications of the list itself or the arrays or data frame it returns.

To get the length of an item list, use the standard len() function.

An item list logically a list of rows, each of which is an item with multiple fields. A designated field, score, is available through the scores() method, and is always single-precision floating-point.

Item lists can be subset as an array (e.g. items[selector]), where integer indices (or arrays thereof), boolean arrays, and slices are allowed as selectors.

When an item list is pickled, it is pickled compactly but only for CPUs: the vocabulary is dropped (after ensuring both IDs and numbers are computed), and all arrays are pickled as NumPy arrays. This makes item lists compact to serialize and transmit, but does mean that that serializing an item list whose scores are still on the GPU will deserialize on the CPU in the receiving process. This is usually not a problem, because item lists are typically used for small lists of items, not large data structures that need to remain in shared memory.

In a few places, the “items” in an item list may be other entities, such as users, tags, authors, etc. This seems less confusing than calling it EntityList, but having a very different use case and feature set than EntitySet.

Note

Naming for fields and accessor methods is tricky, because the usual convention for a data frame is to use singular column names (e.g. “item_id”, “score”) instead of plural (“item_ids”, “scores”) — the data frame, like a database table, is a list of instances, and the column names are best interpreted as naming attributes of individual instances.

However, when working with a list of e.g. item IDs, it is more natural — at least to this author — to use plural names: item_ids. Since this class is doing somewhat double-duty, representing a list of items along with associated data, as well as a data frame of columns representing items, the appropriate naming is not entirely clear. The naming convention in this class is therefore as follows:

  • Field names are singular (item_id, score).

  • Named accessor methods are plural (item_ids(), scores()).

  • Both singular and plural forms are accepted for item IDs numbers, and scores in the keyword arguments. Other field names should be singular.

Todo

Right now, selection / subsetting only happens on the CPU, and will move data to the CPU for the subsetting operation. There is no reason, in principle, why we cannot subset on GPU. Future revisions may add support for this.

Parameters:

  • source (ItemList | IDSequence | None) – A source item list. If provided and an ItemList, its fields and data are used to initialize any aspects of the item list that are not provided in the other arguments. Otherwise, it is interpreted as item_ids.

  • item_ids – A list or array of item identifiers. item_id is accepted as an alternate name.

  • item_nums – A list or array of item numbers. item_num is accepted as an alternate name.

  • vocabulary (Vocabulary | None) – A vocabulary to translate between item IDs and numbers.

  • ordered (bool) – Whether the list has a meaningful order.

  • scores – An array of scores for the items. Pass the value False to remove the scores when copying from a source list.

  • fields – Additional fields, such as score or rating. Field names should generally be singular; the named keyword arguments and accessor methods are plural for readability (“get the list of item IDs”). Pass the value False to remove the field when copying from a source list.

  • _init_dict (dict[str, Any] | None)

  • _init_array (pa.StructArray | None)

  • kwargs (Any)

Stability:
Caller (see Stability Levels).
__init__(source=None, *, ordered=None, vocabulary=None, _init_dict=None, _init_array=None, **kwargs)#
Parameters:

  • source (ItemList | IDSequence | None)

  • ordered (bool | None)

  • vocabulary (Vocabulary | None)

  • _init_dict (dict[str, Any] | None)

  • _init_array (pa.StructArray | None)

  • kwargs (Any)

Methods

__init__([source, ordered, vocabulary, ...])

arrow_types(*[, ids, numbers, ranks])

Get the Arrow data types for this item list.

clone()

Make a shallow copy of the item list.

concat(other)

Concatenate this item list with another.

field(name[, format, index])

from_arrow(tbl, *[, vocabulary])

Create a item list from a Pandas table or structured array.

from_df(df, *[, vocabulary, keep_user])

Create a item list from a Pandas data frame.

from_vocabulary(vocab)

ids(*[, format])

Get the item IDs.

isin(other)

Return a boolean mask identifying the items of this list that are in the other list.

numbers([format, vocabulary, missing])

Get the item numbers.

ranks([format])

Get an array of ranks for the items in this list, if it is ordered.

remove(*[, ids, numbers])

Return an item list with the specified items removed.

scores([format])

Get the item scores (if available).

to_arrow(*[, ids, numbers, ranks, type, columns])

Convert the item list to a Pandas table.

to_df(*[, ids, numbers])

Convert this item list to a Pandas data frame.

top_n([n, scores])

Get the top _N_ items in this list, sorted in decreasing order.

update(other)

Create a copy of the item list, updated with new data from another list.

Attributes

ordered

Whether this list has a meaningful order.

vocabulary

Get the item list's vocabulary, if available.
ordered: bool = False#

Whether this list has a meaningful order.

classmethod from_df(df, *, vocabulary=None, keep_user=False)#

Create a item list from a Pandas data frame. The frame should have item_num and/or item_id columns to identify the items; other columns (e.g. score or rating) are added as fields. If the data frame has user columns (user_id or user_num), those are dropped by default.

Parameters:

  • df (DataFrame) – The data frame to turn into an item list.

  • vocabulary (Vocabulary | None) – The item vocabulary.

  • keep_user (bool) – If True, keeps user ID/number columns instead of dropping them.

Return type:

ItemList

classmethod from_arrow(tbl, *, vocabulary=None)#

Create a item list from a Pandas table or structured array. The table should have item_num and/or item_id columns to identify the items; other columns (e.g. score or rating) are added as fields. If the data frame has user columns (user_id or user_num), those are dropped by default.

Parameters:
Return type:

ItemList

clone()#

Make a shallow copy of the item list.

Return type:

ItemList

property vocabulary: Vocabulary | None#

Get the item list’s vocabulary, if available.

ids(*, format='numpy')#

Get the item IDs.

Returns:

An array of item identifiers.

Raises:

RuntimeError – if the item list was not created with IDs or a Vocabulary.

Parameters:

format (Literal['numpy', 'arrow'])

numbers(format='numpy', *, vocabulary=None, missing='error')#

Get the item numbers.

Parameters:

  • format (LiteralString) – The array format to use.

  • vocabulary (Vocabulary | None) – A alternate vocabulary for mapping IDs to numbers. If provided, then the item list must have IDs (either stored, or through a vocabulary).

  • missing (Literal['error', 'negative', 'null'])

Returns:

An array of item numbers.

Raises:
Return type:

numpy.typing.ArrayLike

scores(format='numpy', **kwargs)#

Get the item scores (if available).

Parameters:

format (LiteralString)

Return type:

ArrayLike | None

ranks(format='numpy')#

Get an array of ranks for the items in this list, if it is ordered. Unordered lists have no ranks. The ranks are based on the order in the list, not on the score.

Item ranks start with 1, for compatibility with common practice in mathematically defining information retrieval metrics and operations.

Returns:

An array of item ranks, or None if the list is unordered.

Parameters:

format (LiteralString)

Return type:

ArrayLike | None

isin(other)#

Return a boolean mask identifying the items of this list that are in the other list.

This is equivalent to numpy.isin() applied to the ID arrays, but is much more efficient in many cases.

Parameters:

other (ItemList)

Return type:

ndarray[tuple[int], dtype[bool]]

to_df(*, ids=True, numbers=True)#

Convert this item list to a Pandas data frame. It has the following columns:

  • item_id — the item IDs (if available and ids=True)

  • item_num — the item numbers (if available and numbers=True)

  • score — the item scores

  • rank — the item ranks (if the list is ordered)

  • all other defined fields, using their field names

Parameters:
Return type:

DataFrame

to_arrow(*, ids=True, numbers=False, ranks=True, type='table', columns=None)#

Convert the item list to a Pandas table.

Parameters:
arrow_types(*, ids=True, numbers=False, ranks=True)#

Get the Arrow data types for this item list.

Parameters:
Return type:

dict[str, DataType]

top_n(n=None, *, scores=None)#

Get the top _N_ items in this list, sorted in decreasing order.

If any scores are undefined (NaN), those items are excluded.

Parameters:

  • n (int | None) – The number of items. If None or negative, returns all items sorted by score.

  • scores (str | ndarray[tuple[int], dtype[V]] | None) – The name of a field containing the scores, or a NumPy vector of scores, for selecting the top _N_. If None, the item list’s scores are used.

Returns:

An ordered item list containing the top n items.

Return type:

ItemList

concat(other)#

Concatenate this item list with another.

If self has a vocabulary that accomodates the item IDs in other, the returned item list will share the vocabulary.

Parameters:

other (ItemList)

Return type:

ItemList

update(other)#

Create a copy of the item list, updated with new data from another list.

This creates a new item list with the same items as self, but with new or modified fields from other. For any item in both self and other, and each field present in other, the values of that field in other are used instead of self. That is:

  • If a field is only present in self, it is used unchanged.

  • If a field is only present in other, its values are used for the items that appear in both sets, and items only in self have an unset / null value for the field (NaN for foating arrays).

  • If a field is present in both lists, then its values from other are used for items appearing in other, and the values from self are used for items that appear only in self.

  • Items that appear only in other are ignored.

Note

Only the presence or absence of an item in self and other is considered, not the nullity of individual field values. If a field appears in both self and other, and some of its values in other are null or NaN, then they will be null or NaN in the resulting item list, regardless of the value of that field for those items on self.

That is, this method does not do item-level coalescing of null field values.

Parameters:

other (ItemList) – The item list to merge into this one.

Returns:

The updated item list.

Return type:

ItemList

remove(*, ids=None, numbers=None)#

Return an item list with the specified items removed.

The items to remove are not required to be in the list.

Parameters:

  • ids (IDSequence | None) – The item IDs to remove.

  • numbers (pa.Int32Array | NPVector[np.integer] | pd.Series[int] | None) – The item numbers to remove.

Return type:

ItemList