Legacy language models within functions（函数中的旧版语言模型）¶

:::callout{title="Legacy" theme="danger"} This is documentation for the legacy language models within functions. The updated language models in functions offer more robust capabilities, such as vision and streaming. Upgrade your language models in functions to take advantage of the latest AIP offerings. :::

Palantir provides a set of language models which can be used within functions. Read more about Palantir-provided LLMs.

:::callout{title="Prerequisites" theme="neutral"} To use Palantir-provided language models, AIP must first be enabled on your enrollment. You also must have permissions to use AIP builder capabilities. :::

Import a language model¶

To begin using a language model, you must import the specific model into the code repository where you are writing your functions by following the steps below:

Navigate and open the Model Imports side panel to view all existing imported models.

Model import sidebar.

To import a new language model, select Add in the top-right corner of the Resource Imports panel and select Models. This will open a new window where you will be able to view the Palantir-provided models that are available to you.

Model import dialog showing a few Palantir-provided LLMs.

You will also see a tab where you can view custom models which have been created through the Modeling Objectives app or direct model deployments previously. More information on using those models can be found in the functions on models documentation.
Select the models you would like to import, then select Confirm selection to import these models into your repository. Task runner will execute the localDev task, generating code bindings to interact with these models.
After importing the language models, you can now use them in your repository by adding the following import statement, replacing GPT_4o with the name of the language model you have imported into your repository:

import { GPT_4o } from "@foundry/models-api/language-models"

Writing a function that uses a language model¶

At this stage, we can now write a function that uses a language model we imported. For this example, we assume that we have imported GPT_4o as described above.

We begin by adding the following import statement to our file:

import { GPT_4o } from "@foundry/models-api/language-models"

Each language model will have generated methods available with strongly typed inputs and outputs. For example, the GPT_4o model provides a createChatCompletion method which allows the user to pass a set of messages along with additional parameters to modify the model’s behavior, such as the temperature or maximum number of tokens.

In the following illustrative example, we use the provided GPT_4o model to run a simple sentiment analysis on a piece of text provided by a user. The function will classify the text as "Good", "Bad", or "Uncertain".

@Function()
public async sentimentAnalysis(userPrompt: string): Promise<string> {
    const systemPrompt = "Provide an estimation of the sentiment the text the user has provided. \
    You may respond with either Good, Bad, or Uncertain. Only choose Good or Bad if you are overwhelmingly \
    sure that the text is either good or bad. If the text is neutral, or you are unable to determine, choose Uncertain."

    const systemMessage = { role: "SYSTEM", contents: [{ text: systemPrompt }] };
    const userMessage = { role: "USER", contents: [{ text: userPrompt }] };
    const gptResponse = await GPT_4o.createChatCompletion({messages: [systemMessage, userMessage], params: { temperature: 0.7 } });
    return gptResponse.choices[0].message.content ?? "Uncertain";
}

This function can then be used throughout the platform.

Embeddings¶

Along with generative language models, Palantir also provides models which can be used to generate embeddings. A simple example is as follows:

@Function()
public async generateEmbeddingsForText(inputs: string[]): Promise<Double[][]> {
    const response = await TextEmbeddingAda_002.createEmbeddings({ inputs });
    return response.embeddings;
}

This is most commonly used to perform semantic search workflows.

Performance considerations¶

Certain models may have rate limits applied to them, limiting the number of tokens which may be passed over a certain time period. This will be enforced along with any standard limits that apply to functions.

Note: AIP feature availability is subject to change and may differ between customers.

中文翻译¶

函数中的旧版语言模型¶

:::callout{title="旧版" theme="danger"} 本文档介绍的是函数中的旧版语言模型。函数中的更新版语言模型提供了更强大的功能，例如视觉和流式传输。升级函数中的语言模型以利用最新的 AIP 产品。 :::

Palantir 提供了一组可在函数中使用的语言模型。详细了解 Palantir 提供的 LLM。

:::callout{title="前提条件" theme="neutral"} 要使用 Palantir 提供的语言模型，必须先在您的注册中启用 AIP。您还需要拥有使用 AIP 构建者功能的权限。 :::

导入语言模型¶

要开始使用语言模型，请按照以下步骤将特定模型导入到您编写函数的代码仓库中：

导航并打开模型导入侧面板，查看所有已导入的模型。

模型导入侧边栏。

要导入新的语言模型，请在资源导入面板的右上角选择添加，然后选择模型。这将打开一个新窗口，您可以在其中查看可供使用的 Palantir 提供的模型。

模型导入对话框，显示几个 Palantir 提供的 LLM。

您还会看到一个选项卡，可以查看之前通过建模目标应用程序或直接模型部署创建的自定义模型。有关使用这些模型的更多信息，请参阅模型上的函数文档。
选择您要导入的模型，然后选择确认选择将这些模型导入到您的仓库中。任务运行器将执行 localDev 任务，生成用于与这些模型交互的代码绑定。
导入语言模型后，您现在可以通过添加以下导入语句在仓库中使用它们，将 GPT_4o 替换为您导入到仓库中的语言模型名称：

import { GPT_4o } from "@foundry/models-api/language-models"

编写使用语言模型的函数¶

现在，我们可以编写一个使用已导入语言模型的函数。在此示例中，我们假设已按上述方式导入了 GPT_4o。

我们首先在文件中添加以下导入语句：

import { GPT_4o } from "@foundry/models-api/language-models"

每个语言模型都会提供具有强类型输入和输出的生成方法。例如，GPT_4o 模型提供了一个 createChatCompletion 方法，允许用户传递一组消息以及附加参数（如温度或最大令牌数）来修改模型的行为。

在以下示例中，我们使用提供的 GPT_4o 模型对用户提供的文本进行简单的情感分析。该函数会将文本分类为"好"、"坏"或"不确定"。

@Function()
public async sentimentAnalysis(userPrompt: string): Promise<string> {
    const systemPrompt = "请评估用户提供的文本的情感倾向。\
    您只能回复“好”、“坏”或“不确定”。只有在您非常确定文本是正面或负面时，才选择“好”或“坏”。\
    如果文本是中性的，或者您无法确定，请选择“不确定”。"

    const systemMessage = { role: "SYSTEM", contents: [{ text: systemPrompt }] };
    const userMessage = { role: "USER", contents: [{ text: userPrompt }] };
    const gptResponse = await GPT_4o.createChatCompletion({messages: [systemMessage, userMessage], params: { temperature: 0.7 } });
    return gptResponse.choices[0].message.content ?? "不确定";
}

此函数随后可在整个平台中使用。

嵌入¶

除了生成式语言模型，Palantir 还提供可用于生成嵌入的模型。一个简单的示例如下：

@Function()
public async generateEmbeddingsForText(inputs: string[]): Promise<Double[][]> {
    const response = await TextEmbeddingAda_002.createEmbeddings({ inputs });
    return response.embeddings;
}

这最常用于执行语义搜索工作流。

性能考量¶

某些模型可能应用了速率限制，限制了在特定时间段内可传递的令牌数量。这将与适用于函数的任何标准限制一起强制执行。

注意：AIP 功能的可用性可能会发生变化，并且可能因客户而异。