lenskit.config ============== .. py:module:: lenskit.config .. autoapi-nested-parse:: LensKit general configuration Submodules ---------- .. autoapisummary:: lenskit.config.common .. toctree:: :hidden: :maxdepth: 1 /api/lenskit/config/common/index Classes ------- .. autoapisummary:: lenskit.config.LenskitSettings lenskit.config.MachineSettings lenskit.config.ParallelSettings lenskit.config.PowerQueries lenskit.config.PrometheusSettings lenskit.config.RandomSettings lenskit.config.TuneSettings Functions --------- .. autoapisummary:: lenskit.config.load_config_data lenskit.config.locate_configuration_root lenskit.config.lenskit_config lenskit.config.configure Package Contents ---------------- .. py:function:: load_config_data(path: pathlib.Path | os.PathLike[str], model: None = None) -> pydantic.JsonValue load_config_data(path: pathlib.Path | os.PathLike[str], model: type[M]) -> M General-purpose function to automatically load configuration data and optionally validate with a model. :param path: The path to the configuration file. :param model: The Pydantic model class to validate. .. py:function:: locate_configuration_root(*, cwd = None, abort_at_pyproject = True, abort_at_gitroot = True) Search for a configuration root containing a ``lenskit.toml`` file. This searches for a ``lenskit.toml`` file, beginning in the current working directory (or the alternate ``cwd`` if provided), and searching upward until one is found. Search stops if a ``pyproject.toml`` file or ``.git`` directory is found without encountering ``lenskit.toml``. .. py:class:: LenskitSettings :canonical: lenskit.config._schema.LenskitSettings Bases: :py:obj:`pydantic_settings.BaseSettings` Definition of LensKit settings. LensKit supports loading various settings from configuration files and the environment as a consistent way to control LensKit's various control surfaces. :Stability: Experimental .. py:attribute:: model_config .. py:attribute:: random :type: RandomSettings Random number generator configuration. .. py:attribute:: parallel :type: ParallelSettings .. py:attribute:: machine :type: str | None :value: None The name of the machine running experiments. This is usually set in ``lenskit.local.toml`` or the ``LK_MACHINE`` environment variable. .. py:attribute:: prometheus :type: PrometheusSettings Prometheus settings for task metric collection. .. py:attribute:: machines :type: dict[str, MachineSettings] Description of different machines used in the experiment(s), to support things like collecting power metrics. .. py:attribute:: tuning :type: TuneSettings LensKit tuning settings. .. py:property:: current_machine :type: MachineSettings | None .. py:method:: settings_customise_sources(settings_cls, init_settings, env_settings, dotenv_settings, file_secret_settings) :classmethod: .. py:method:: finish_setup() Finalize settings, resolving default values etc. as needed. .. py:class:: MachineSettings :canonical: lenskit.config._schema.MachineSettings Bases: :py:obj:`pydantic.BaseModel` Definition for a single machine. A “machine” is a computer (or cluster) that is in use for running LensKit experiments. Many users won't use this, but if you want to use the power consumption monitoring, you will need to define how to measure power for the different machines in use. .. py:attribute:: description :type: str | None :value: None Short description for this machine. .. py:attribute:: power_queries :type: PowerQueries Prometheus queries to collect power metrics for this machine. .. py:class:: ParallelSettings :canonical: lenskit.config._schema.ParallelSettings Bases: :py:obj:`pydantic_settings.BaseSettings` Configuration for LensKit's parallel processing. These settings are in the ``[parallel]`` table in ``lenskit.toml``. .. seealso:: :ref:`parallel-config` .. py:attribute:: model_config .. py:attribute:: num_cpus :type: int :value: 0 The number of CPUs LensKit should consider using. This is auto-detected from the system environment, and should only be configured manually if you want to override LensKit's CPU detection for some reason. Note that the auto-detected values *do* account for operating system scheduling affinities and CPU limits. This value is not used directly as a limit, but is used to derive the default values for the other concurrency controls (threads, etc.). .. py:attribute:: use_ray :type: bool :value: False Use Ray to parallelize batch operations, hyperparameter tuning, etc. .. py:attribute:: num_batch_jobs :type: int :value: 0 Number of batch inference jobs to run in parallel. Can be overridden with the :envvar:`LK_NUM_BATCH_JOBS` environment variable. .. py:attribute:: num_procs :type: int :value: 0 Number of processes to use. .. py:attribute:: num_threads :type: int :value: 0 Number of threads to use. Can be overridden with the :envvar:`LK_NUM_THREADS` environment variable. Specify -1 to use all available threads. .. py:attribute:: num_backend_threads :type: int :value: 0 Number of threads for compute backends to use. Can be overridden with the :envvar:`LK_NUM_BACKEND_THREADS` environment variable. Specify -1 to leave threading limits unmodified. .. py:property:: total_threads .. py:method:: resolve_defaults() Resolve default values for thread/process counts. .. py:method:: env_vars() Get the parallel configuration as a set of environment variables to configure a child process. The set also includes ``OMP_NUM_THREADS`` and related variables for BLAS and MKL to configure OMP early. .. py:class:: PowerQueries :canonical: lenskit.config._schema.PowerQueries Bases: :py:obj:`TypedDict` Queries for requesting power consumption data from Prometheus. Each entry is a Python format string (for :meth:`str.format`), that will be used to format a dictionary ``{"elapsed": time_ms}``. The result should be a valid Prometheus query that returns the power consumption in Joules (watt seconds) over the last ``time_ms`` milliseconds. .. py:attribute:: system :type: str Total (chassis or system) power consumption. .. py:attribute:: cpu :type: str CPU power consumption. .. py:attribute:: gpu :type: str GPU power consumption. .. py:class:: PrometheusSettings :canonical: lenskit.config._schema.PrometheusSettings Bases: :py:obj:`pydantic.BaseModel` Prometheus configuration settings. LensKit's task logging supports querying Prometheus for task-related metrics such as power consumption. .. py:attribute:: url :type: str | None :value: None .. py:class:: RandomSettings :canonical: lenskit.config._schema.RandomSettings Bases: :py:obj:`pydantic.BaseModel` Random number generator configuration. .. py:attribute:: seed :type: int | None :value: None The root RNG seed. .. py:class:: TuneSettings :canonical: lenskit.config._schema.TuneSettings Bases: :py:obj:`pydantic.BaseModel` LensKit hyperparameter tuning settings. .. py:attribute:: jobs :type: int | None :value: None Number of allowed hyperparameter tuning jobs. .. py:attribute:: gpu_mult :type: Annotated[float, Gt(0), Le(1.0)] :value: 1.0 Multiplier for tuning job GPU requirements. This is to coarsely adapt GPU requirements from configuration files to the local machine. If a tuning specificataion requires 1 GPU, but your machine has enough capacity to run two jobs in parallel on a single GPU, you can set this to 0.5 to modify the tuning jobs to require 0.5 GPUs each. .. py:attribute:: max_points :type: int | None :value: None Maximum number of search points for hyperparameter tuning. .. py:function:: lenskit_config() Get the LensKit configuration. If no configuration has been specified, returns a default settings object. .. py:function:: configure(cfg_dir: pathlib.Path | None = None) -> _schema.LenskitSettings configure(cfg_dir: pathlib.Path | None = None, *, settings_cls: type[SettingsClass]) -> SettingsClass configure(settings: _schema.LenskitSettings, /) -> _schema.LenskitSettings Initialize LensKit configuration. LensKit does **not** automatically read configuration files — if this function is never called, then configuration will entirely be done through defaults and environment varibles. This function will automatically configure the global RNG, if a seed is specified. It does **not** configure logging. :param cfg_dir: The directory in which to look for configuration files.. If not provided, uses the current directory. :param settings_cls: The base LensKit settings class. Rarely used, only needed if a project wants to to extend LensKit settings with their own settings. :returns: The configured LensKit settings. Exported Aliases ---------------- .. py:exception:: lenskit.config.ConfigWarning Re-exported alias for :py:exc:`lenskit.diagnostics.ConfigWarning`. .. py:function:: lenskit.config.get_logger Re-exported alias for :py:func:`lenskit.logging.get_logger`. .. py:function:: lenskit.config.init_global_rng Re-exported alias for :py:func:`lenskit.random.init_global_rng`.