API: Model experiments（API：模型实验(Model experiments)）¶

The model experiments Python API provides a set of methods for writing metrics and hyperparameters to experiments. This page contains information on available functions and classes in the palantir_models.experiments package.

`palantir_models.experiments.Experiment`¶

A class for tracking model training experiments. An experiment can be created from any ModelOutput class.

model_output.create_experiment(name="experiment-name")

Function	Description
`log_param(key, value)`	Log a single parameter to an experiment.
`log_params(parameter_map)`	Log a set of parameters to an experiment.
`log_metric(name, value, step)`	Log a metric value to an experiment under the given series.
`log_metrics(metric_values, step)`	Log a set of metric values to an experiment under the given series.
`log_image(name, image, caption, step)`	Log an image to an experiment under the given series.
`as_mlflow_run()`	Creates a context block and sets the active MLflow run to write to the experiment.

`palantir_models.experiments.ErrorHandlerType`¶

An enum for controlling how errors are handled when running experiment code. This specifically governs errors caused by non-user code such as network errors when writing metrics, or experiment size overflows.

Variant	Description
`ErrorHandlerType.FAIL`	Fail on any error. Will re-raise the error back to the caller.
`ErrorHandlerType.WARN`	Warn on the first error, then suppress the rest.
`ErrorHandlerType.SUPPRESS`	Do not log anything.

The desired error handler type can be passed to ModelOutput.create_experiment when creating an experiment as shown below:

from palantir_models.experiments import ErrorHandlerType
model_output.create_experiment(
    name="mnist-experiment",
    error_handler_type=ErrorHandlerType.WARN
)

`Experiment.log_param`¶

palantir_models.experiments.Experiment.log_param(key: str, value: Union[bool, date, datetime, float, int, str])

Logs the given parameter value to the given key.
Parameters can be overwritten by logging to the same parameter key.

experiment.log_param(key="learning_rate", value=1e-3)

Parameters¶

key: str
The name of the parameter to write.
value: Union[bool, date, datetime, float, int, str]
The value to write to the parameter.

`Experiment.log_params`¶

palantir_models.experiments.Experiment.log_params(parameter_map: Dict[str, Union[bool, date, datetime, float, int, str]])

The batched version of log_param.

experiment.log_params(parameter_map={
    "learning_rate": 1e-3,
    "model_type": "CNN"
})

Parameters¶

parameter_map: Dict[str, Union[bool, date, datetime, float, int, str]]
A mapping of parameter name to parameter value.

`Experiment.log_metric`¶

palantir_models.experiments.Experiment.log_metric(metric_name: str, metric_value: float, step: Optional[int] = None)

Logs the metric value in the series.
If the series does not exist (has not been logged to yet), one will be created.
If step is provided, an attempt to log at that step will be made. If the step has already been logged to, an error will be raised.
If step is not provided, the current step of the series will be auto-incremented by 1.

experiment.log_metric(metric_name="train/loss", metric_value=1.5)
experiment.log_metric(metric_name="train/loss", metric_value=0.9) # will auto-increment the step

Parameters¶

metric_name: str
The name of the metric series to write the value to.
metric_value: float
The value to write.
step: Optional[int]
Optional step to write to. If the provided step is less than the current step, an error will be raised. If the step is not provided, the step will be auto-incremented.

`Experiment.log_metrics`¶

palantir_models.experiments.Experiment.log_metrics(values: Dict[str, float], step: Optional[int] = None)

The batched version of log_metric.
Logs all provided values to their respective series at the same step.
If any series does not exist (has not been logged to yet), it will be created.
If step is provided, an attempt to log at that step will be made. If the step has already been logged to on any series, an error will be raised.
If step is not provided, the current step of the all the series will be independently auto-incremented by 1.

experiment.log_metrics(values={
    "train/loss": 1.5,
    "train/acc", 0.7
})

Parameters¶

values: Dict[str, float]
Mapping of metric name to metric value.
step: Optional[int]
Optional step to write to. If the provided step is less than the current step of any series being written to, an error will be raised. If the step is not provided, the steps of each series will be independently auto-incremented.

`Experiment.log_image`¶

palantir_models.experiments.Experiment.log_image(series_name: str, image: Union[str, bytes, "PIL.Image.Image"], caption: Optional[str] = None, step: Optional[int] = None)

Logs the image in the series.
The image must be a path to a PNG image, PNG image bytes, or a Pillow ↗ image.
If the series does not exist (has not been logged to yet), one will be created.
If step is provided, an attempt to log at that step will be made. If the step has already been logged to, an error will be raised.
If step is not provided, the current step of the series will be auto-incremented by 1.

experiment.log_image("bounding_boxes", image="/path/to/image.png", caption="My caption", step=1)

Parameters¶

series_name: str
The name of the image series to write the value to.
image: Union[str, bytes, "PIL.Image.Image"]
The image to write.
caption: Optional[str]
Optional caption to write alongside the image.
step: Optional[str]
Optional step to write to. If the provided step is less than the current step, an error will be raised. If the step is not provided, the step will be auto-incremented.

`Experiment.log_plot`¶

palantir_models.experiments.Experiment.log_plot(series_name: str, plot_data: "plotly.graph_objects.Figure", caption: Optional[str] = None, step: Optional[int] = None)

Logs the plot in the series.
The plot must be a Plotly ↗ plotly.graph_objects.Figure.
If the series does not exist (has not been logged to yet), one will be created.
If step is provided, an attempt to log at that step will be made. If the step has already been logged to, an error will be raised.
If step is not provided, the current step of the series will be auto-incremented by 1.

import plotly.express as px

fig = px.line(x=[1, 2, 3], y=[4, 3, 2])
experiment.log_plot("training/accuracy", plot_data=fig, caption="Accuracy over epochs", step=10)

Parameters¶

series_name: str
The name of the plot series to write the value to.
plot_data: "plotly.graph_objects.Figure"
The plot to write.
caption: Optional[str]
Optional caption to write alongside the plot.
step: Optional[int]
Optional step to write to. If the provided step is less than the current step, an error will be raised. If the step is not provided, the step will be auto-incremented.

`Experiment.log_table`¶

palantir_models.experiments.Experiment.log_table(table_name: str, table: Union["polars.DataFrame", "pandas.DataFrame"], description: Optional[str] = None)

Logs the table to the experiment.
The table must be either a pandas ↗ or Polars ↗ DataFrame.

import pandas as pd

leaderboard = pd.DataFrame({
    "model_type": ["xgboost", "decision_tree", "torch_nn"],
    "score": [0.75, 0.6, 0.93],
    "rank": [2, 3, 1]
})

experiment.log_table("model_leaderboard", leaderboard)

Parameters¶

table_name: str
The name of the table.
table: Union["polars.DataFrame", "pandas.DataFrame"]
The table data to write.
description: Optional[str]
Optional description to write alongside the table.

`Experiment.as_mlflow_run`¶

palantir_models.experiments.Experiment.as_mlflow_run()

Creates an active MLflow run which will write to the experiment
Requires MLflow >= 2.0.0

with experiment.as_mlflow_run() as run:
    ...

中文翻译¶

API：模型实验(Model experiments)¶

模型实验 Python API 提供了一组用于将指标(Metrics)和超参数(Hyperparameters)写入实验的方法。本页包含 palantir_models.experiments 包中可用函数和类的信息。

`palantir_models.experiments.Experiment`¶

用于跟踪模型训练实验的类。可以从任何 ModelOutput 类创建实验。

model_output.create_experiment(name="experiment-name")

函数	描述
`log_param(key, value)`	将单个参数记录到实验。
`log_params(parameter_map)`	将一组参数记录到实验。
`log_metric(name, value, step)`	将指标值记录到实验的指定序列(Series)下。
`log_metrics(metric_values, step)`	将一组指标值记录到实验的指定序列下。
`log_image(name, image, caption, step)`	将图像记录到实验的指定序列下。
`as_mlflow_run()`	创建一个上下文块，并将活动的 MLflow 运行设置为写入该实验。

`palantir_models.experiments.ErrorHandlerType`¶

用于控制运行实验代码时如何处理错误的枚举。这专门管理由非用户代码引起的错误，例如写入指标时的网络错误或实验大小溢出。

变体	描述
`ErrorHandlerType.FAIL`	任何错误都失败。将重新引发错误给调用者。
`ErrorHandlerType.WARN`	在第一个错误时发出警告，然后抑制其余错误。
`ErrorHandlerType.SUPPRESS`	不记录任何内容。

创建实验时，可以将所需的错误处理程序类型传递给 ModelOutput.create_experiment，如下所示：

from palantir_models.experiments import ErrorHandlerType
model_output.create_experiment(
    name="mnist-experiment",
    error_handler_type=ErrorHandlerType.WARN
)

`Experiment.log_param`¶

palantir_models.experiments.Experiment.log_param(key: str, value: Union[bool, date, datetime, float, int, str])

将给定的参数 value 记录到给定的 key。
通过记录到相同的参数键可以覆盖参数。

experiment.log_param(key="learning_rate", value=1e-3)

参数¶

key: str
要写入的参数名称。
value: Union[bool, date, datetime, float, int, str]
要写入参数的值。

`Experiment.log_params`¶

palantir_models.experiments.Experiment.log_params(parameter_map: Dict[str, Union[bool, date, datetime, float, int, str]])

log_param 的批量版本。

experiment.log_params(parameter_map={
    "learning_rate": 1e-3,
    "model_type": "CNN"
})

参数¶

parameter_map: Dict[str, Union[bool, date, datetime, float, int, str]]
参数名称到参数值的映射。

`Experiment.log_metric`¶

palantir_models.experiments.Experiment.log_metric(metric_name: str, metric_value: float, step: Optional[int] = None)

将指标值记录到序列中。
如果序列不存在（尚未记录过），则会创建一个。
如果提供了 step，将尝试在该步骤进行记录。如果该步骤已被记录过，将引发错误。
如果未提供 step，序列的当前步骤将自动递增 1。

experiment.log_metric(metric_name="train/loss", metric_value=1.5)
experiment.log_metric(metric_name="train/loss", metric_value=0.9) # 将自动递增步骤

参数¶

metric_name: str
要写入值的指标序列的名称。
metric_value: float
要写入的值。
step: Optional[int]
可选的写入步骤。如果提供的步骤小于当前步骤，将引发错误。如果未提供步骤，步骤将自动递增。

`Experiment.log_metrics`¶

palantir_models.experiments.Experiment.log_metrics(values: Dict[str, float], step: Optional[int] = None)

log_metric 的批量版本。
将所有提供的值在相同步骤记录到各自的序列。
如果任何序列不存在（尚未记录过），则会创建它。
如果提供了 step，将尝试在该步骤进行记录。如果该步骤在任何序列上已被记录过，将引发错误。
如果未提供 step，所有序列的当前步骤将独立自动递增 1。

experiment.log_metrics(values={
    "train/loss": 1.5,
    "train/acc", 0.7
})

参数¶

values: Dict[str, float]
指标名称到指标值的映射。
step: Optional[int]
可选的写入步骤。如果提供的步骤小于任何正在写入的序列的当前步骤，将引发错误。如果未提供步骤，每个序列的步骤将独立自动递增。

`Experiment.log_image`¶

palantir_models.experiments.Experiment.log_image(series_name: str, image: Union[str, bytes, "PIL.Image.Image"], caption: Optional[str] = None, step: Optional[int] = None)

将图像记录到序列中。
图像必须是 PNG 图像的路径、PNG 图像字节或 Pillow ↗ 图像。
如果序列不存在（尚未记录过），则会创建一个。
如果提供了 step，将尝试在该步骤进行记录。如果该步骤已被记录过，将引发错误。
如果未提供 step，序列的当前步骤将自动递增 1。

experiment.log_image("bounding_boxes", image="/path/to/image.png", caption="My caption", step=1)

参数¶

series_name: str
要写入值的图像序列的名称。
image: Union[str, bytes, "PIL.Image.Image"]
要写入的图像。
caption: Optional[str]
可选的标题，与图像一起写入。
step: Optional[str]
可选的写入步骤。如果提供的步骤小于当前步骤，将引发错误。如果未提供步骤，步骤将自动递增。

`Experiment.log_plot`¶

palantir_models.experiments.Experiment.log_plot(series_name: str, plot_data: "plotly.graph_objects.Figure", caption: Optional[str] = None, step: Optional[int] = None)

将图表记录到序列中。
图表必须是 Plotly ↗ 的 plotly.graph_objects.Figure。
如果序列不存在（尚未记录过），则会创建一个。
如果提供了 step，将尝试在该步骤进行记录。如果该步骤已被记录过，将引发错误。
如果未提供 step，序列的当前步骤将自动递增 1。

import plotly.express as px

fig = px.line(x=[1, 2, 3], y=[4, 3, 2])
experiment.log_plot("training/accuracy", plot_data=fig, caption="Accuracy over epochs", step=10)

参数¶

series_name: str
要写入值的图表序列的名称。
plot_data: "plotly.graph_objects.Figure"
要写入的图表。
caption: Optional[str]
可选的标题，与图表一起写入。
step: Optional[int]
可选的写入步骤。如果提供的步骤小于当前步骤，将引发错误。如果未提供步骤，步骤将自动递增。

`Experiment.log_table`¶

palantir_models.experiments.Experiment.log_table(table_name: str, table: Union["polars.DataFrame", "pandas.DataFrame"], description: Optional[str] = None)

将表格记录到实验。
表格必须是 pandas ↗ 或 Polars ↗ DataFrame。

import pandas as pd

leaderboard = pd.DataFrame({
    "model_type": ["xgboost", "decision_tree", "torch_nn"],
    "score": [0.75, 0.6, 0.93],
    "rank": [2, 3, 1]
})

experiment.log_table("model_leaderboard", leaderboard)

参数¶

table_name: str
表格的名称。
table: Union["polars.DataFrame", "pandas.DataFrame"]
要写入的表格数据。
description: Optional[str]
可选的描述，与表格一起写入。

`Experiment.as_mlflow_run`¶

palantir_models.experiments.Experiment.as_mlflow_run()

创建一个活动的 MLflow 运行，该运行将写入实验。
需要 MLflow >= 2.0.0

with experiment.as_mlflow_run() as run:
    ...

API: Model experiments（API：模型实验(Model experiments)）¶

palantir_models.experiments.Experiment¶

palantir_models.experiments.ErrorHandlerType¶

Experiment.log_param¶

Parameters¶

Experiment.log_params¶

Parameters¶

Experiment.log_metric¶

Parameters¶

Experiment.log_metrics¶

Parameters¶

Experiment.log_image¶

Parameters¶

Experiment.log_plot¶

Parameters¶

Experiment.log_table¶

Parameters¶

Experiment.as_mlflow_run¶

中文翻译¶

API：模型实验(Model experiments)¶

palantir_models.experiments.Experiment¶

palantir_models.experiments.ErrorHandlerType¶

Experiment.log_param¶

参数¶

Experiment.log_params¶

参数¶

Experiment.log_metric¶

参数¶

Experiment.log_metrics¶

参数¶

Experiment.log_image¶

参数¶

Experiment.log_plot¶

参数¶

Experiment.log_table¶

参数¶

Experiment.as_mlflow_run¶

`palantir_models.experiments.Experiment`¶

`palantir_models.experiments.ErrorHandlerType`¶

`Experiment.log_param`¶

`Experiment.log_params`¶

`Experiment.log_metric`¶

`Experiment.log_metrics`¶

`Experiment.log_image`¶

`Experiment.log_plot`¶

`Experiment.log_table`¶

`Experiment.as_mlflow_run`¶

`palantir_models.experiments.Experiment`¶

`palantir_models.experiments.ErrorHandlerType`¶

`Experiment.log_param`¶

`Experiment.log_params`¶

`Experiment.log_metric`¶

`Experiment.log_metrics`¶

`Experiment.log_image`¶

`Experiment.log_plot`¶

`Experiment.log_table`¶

`Experiment.as_mlflow_run`¶