Experiments（实验（Experiments））

An experiment is an artifact representing a collection of metrics produced during a model training job.

Integrate experiments into model training code

Experiments can be created from any environment used to create models in Foundry. The ModelOutput class in Jupyter® Code Workspaces and Code Repositories provides hooks for creating and writing to experiments. These experiments can then be published alongside the model and are instantly viewable in the model page.

Users can also take advantage of MLflow to streamline the integration of experiments into their existing code.

Code Workspaces

The below code snippet demonstrates experiment usage in Jupyter® Code Workspaces:

from palantir_models.code_workspaces import ModelOutput
# `my-alias` is an alias to a model in the current workspace
model_output = ModelOutput("my-alias")

experiment = model_output.create_experiment(name="my-experiment")

# log parameters
experiment.log_param("learning_rate", 1e-4)

# log metrics
experiment.log_metric("train/loss", loss)
experiment.log_metric("train/loss", loss, step=step)

# publish alongside model to persist in the models page
model_output.publish(ModelAdapter(model), experiment=experiment)

Code Repositories

The below code snippet demonstrates experiment usage in Code Repositories:

from transforms.api import configure, transform, Input
from palantir_models.transforms import ModelOutput

@transform(
    input_data=Input("..."),
    model_output=ModelOutput("..."),
)
def compute(input_data, model_output):
    experiment = model_output.create_experiment(name="my-experiment")

    # log parameters
    experiment.log_param("learning_rate", 1e-4)

    # log metrics
    experiment.log_metric("train/loss", loss)
    # can also provide optional step value
    experiment.log_metric("train/loss", loss, step=step)

    # publish alongside model to persist in the models page
    model_output.publish(ModelAdapter(model), experiment=experiment)

Learn more about creating and visualizing experiments.

---

中文翻译

---

实验（Experiments）

实验是一种工件（artifact），用于表示模型训练任务过程中产生的指标集合。

将实验集成到模型训练代码中

实验可以从 Foundry 中用于创建模型的任何环境中生成。在 Jupyter® Code Workspaces 和 Code Repositories 中，ModelOutput 类提供了创建和写入实验的钩子（hooks）。这些实验可以与模型一同发布，并立即在模型页面中查看。

用户还可以利用 MLflow 来简化将实验集成到现有代码中的过程。

Code Workspaces

以下代码片段展示了在 Jupyter® Code Workspaces 中使用实验的方法：

from palantir_models.code_workspaces import ModelOutput
# `my-alias` 是当前工作空间中某个模型的别名
model_output = ModelOutput("my-alias")

experiment = model_output.create_experiment(name="my-experiment")

# 记录参数
experiment.log_param("learning_rate", 1e-4)

# 记录指标
experiment.log_metric("train/loss", loss)
experiment.log_metric("train/loss", loss, step=step)

# 与模型一同发布，以在模型页面中持久化保存
model_output.publish(ModelAdapter(model), experiment=experiment)

Code Repositories

以下代码片段展示了在 Code Repositories 中使用实验的方法：

from transforms.api import configure, transform, Input
from palantir_models.transforms import ModelOutput

@transform(
    input_data=Input("..."),
    model_output=ModelOutput("..."),
)
def compute(input_data, model_output):
    experiment = model_output.create_experiment(name="my-experiment")

    # 记录参数
    experiment.log_param("learning_rate", 1e-4)

    # 记录指标
    experiment.log_metric("train/loss", loss)
    # 也可以提供可选的步长值
    experiment.log_metric("train/loss", loss, step=step)

    # 与模型一同发布，以在模型页面中持久化保存
    model_output.publish(ModelAdapter(model), experiment=experiment)

了解更多关于创建和可视化实验的信息。