Use Palantir-provided language models within transforms（在转换中使用 Palantir 提供的语言模型）¶

:::callout{title="Prerequisites" theme="neutral"} To use Palantir-provided language models, AIP must first be enabled on your enrollment. :::

Palantir provides a set of language and embedding models which can be used within Python transforms. The models can be used through the palantir_models library. This library provides a set of FoundryInputParams that can be used with the transforms.api.transform decorator.

Repository setup¶

To add language model support to your transforms, open the library search panel on the left side of your Code Repository. Search for palantir_models and choose Add and install library within the Library tab. Repeat this process with language-model-service-api to add that library as well.

Your Code Repository will then resolve all dependencies and run checks again. Checks may take a moment to complete, after which you will be able to start using the library in your transforms.

Transform setup¶

:::callout{title="Prerequisites" theme="neutral"} The palantir_model classes can only be used with the transforms.api.transform decorator. :::

In this example, we will use the palantir_models.transforms.OpenAiGptChatLanguageModelInput. First, import OpenAiGptChatLanguageModelInput into your Python file. This class can now be used to create our transform. Then, follow the prompts to specify and import the model and dataset that you wish to use as input.

from transforms.api import transform, Input, Output
from palantir_models.transforms import OpenAiGptChatLanguageModelInput
from palantir_models.models import OpenAiGptChatLanguageModel

@transform(
    source_df=Input("/path/to/input/dataset")
    model=OpenAiGptChatLanguageModelInput("ri.language-model-service..language-model.gpt-4_azure"),
    output=Output("/path/to/output/dataset"),
)
def compute_generic(ctx, source_df, model: OpenAiGptChatLanguageModel, output):
    ...

As you begin typing the resource identifier, a dropdown menu will automatically appear to indicate models available for use. You may choose your desired option from the dropdown.

Dropdown menu showing available LLMs for selection following a user's input of "ri.".

Use language models to generate completions¶

For this example, we will be using the language model to determine the sentiment for each review in the input dataset. The OpenAiGptChatLanguageModelInput provides an OpenAiGptChatLanguageModel to the transform at runtime which can then be used to generate completions for reviews.

from transforms.api import transform, Input, Output
from palantir_models.transforms import OpenAiGptChatLanguageModelInput
from palantir_models.models import OpenAiGptChatLanguageModel
from language_model_service_api.languagemodelservice_api_completion_v3 import GptChatCompletionRequest
from language_model_service_api.languagemodelservice_api import ChatMessage, ChatMessageRole


@transform(
    reviews=Input("/path/to/reviews/dataset"),
    model=OpenAiGptChatLanguageModelInput("ri.language-model-service..language-model.gpt-4_azure"),
    output=Output("/output/path"),
)
def compute_sentiment(ctx, reviews, model: OpenAiGptChatLanguageModel, output):
    def get_completions(review_content: str) -> str:
        system_prompt = "Take the following review determine the sentiment of the review"
        request = GptChatCompletionRequest(
            [ChatMessage(ChatMessageRole.SYSTEM, system_prompt), ChatMessage(ChatMessageRole.USER, review_content)]
        )
        resp = model.create_chat_completion(request)
        return resp.choices[0].message.content

    reviews_df = reviews.pandas()
    reviews_df['sentiment'] = reviews_df['review_content'].apply(get_completions)
    out_df = ctx.spark_session.createDataFrame(reviews_df)
    return output.write_dataframe(out_df)

Embeddings¶

Along with generative language models, Palantir also provides an embedding model. The following example shows how we can use the palantir_models.transforms.GenericEmbeddingModelInput to calculate embeddings on the same reviews dataset. The GenericEmbeddingModelInput provides a GenericEmbeddingModel to the transform at runtime which can be used to calculate embeddings for each review. The embeddings are explicitly cast to floats because the ontology vector property requires this.

from transforms.api import transform, Input, Output
from language_model_service_api.languagemodelservice_api_embeddings_v3 import GenericEmbeddingsRequest
from palantir_models.models import GenericEmbeddingModel
from palantir_models.transforms import GenericEmbeddingModelInput
from pyspark.sql.types import ArrayType, FloatType


@transform(
    reviews=Input("/path/to/reviews/dataset"),
    embedding_model=GenericEmbeddingModelInput("ri.language-model-service..language-model.text-embedding-ada-002_azure"),
    output=Output("/path/to/embedding/output")
)
def compute_embeddings(ctx, reviews, embedding_model: GenericEmbeddingModel, output):

    def internal_create_embeddings(val: str):
        return embedding_model.create_embeddings(GenericEmbeddingsRequest(inputs=[val])).embeddings[0]

    reviews_df = reviews.pandas()
    reviews_df['embedding'] = reviews_df['review_content'].apply(internal_create_embeddings)
    spark_df = ctx.spark_session.createDataFrame(reviews_df)
    out_df = spark_df.withColumn('embedding', spark_df['embedding'].cast(ArrayType(FloatType())))
    return output.write_dataframe(out_df)

Use vision language models to extract PDF document content¶

For PDF document extraction workflows using vision language models, we recommend using AIP Document Intelligence. AIP Document Intelligence provides an intuitive interface to test different extraction strategies on your documents, compare results, and evaluate extraction quality before deploying to production.

After you deploy a configuration, a Python transform code repository is created and automatically configured with your selected extraction method, model, and any custom prompts. You do not need to manually write transform code for document extraction. Learn more about deploying to Python transforms.

中文翻译¶

在转换中使用 Palantir 提供的语言模型¶

:::callout{title="前提条件" theme="neutral"} 要使用 Palantir 提供的语言模型，必须先在您的注册环境中启用 AIP。 :::

Palantir 提供了一组语言模型和嵌入模型(embedding models)，可在 Python 转换(transforms)中使用。这些模型可通过 palantir_models 库使用。该库提供了一组 FoundryInputParams，可与 transforms.api.transform 装饰器配合使用。

代码仓库设置¶

要为您的转换添加语言模型支持，请打开代码仓库左侧的库搜索面板。搜索 palantir_models，然后在库选项卡中选择添加并安装库。对 language-model-service-api 重复此过程以添加该库。

您的代码仓库随后将解析所有依赖项并重新运行检查。检查可能需要一些时间才能完成，之后您就可以开始在转换中使用该库了。

转换设置¶

:::callout{title="前提条件" theme="neutral"} palantir_model 类只能与 transforms.api.transform 装饰器一起使用。 :::

在本示例中，我们将使用 palantir_models.transforms.OpenAiGptChatLanguageModelInput。首先，将 OpenAiGptChatLanguageModelInput 导入到您的 Python 文件中。现在可以使用此类来创建我们的转换。然后，按照提示指定并导入您希望用作输入的模型和数据集。

from transforms.api import transform, Input, Output
from palantir_models.transforms import OpenAiGptChatLanguageModelInput
from palantir_models.models import OpenAiGptChatLanguageModel

@transform(
    source_df=Input("/path/to/input/dataset")
    model=OpenAiGptChatLanguageModelInput("ri.language-model-service..language-model.gpt-4_azure"),
    output=Output("/path/to/output/dataset"),
)
def compute_generic(ctx, source_df, model: OpenAiGptChatLanguageModel, output):
    ...

当您开始输入资源标识符时，会自动出现一个下拉菜单，显示可供使用的模型。您可以从下拉菜单中选择所需的选项。

下拉菜单显示用户输入"ri."后可选择的可用LLM。

使用语言模型生成补全¶

在本示例中，我们将使用语言模型来确定输入数据集中每条评论的情感倾向。OpenAiGptChatLanguageModelInput 在运行时向转换提供一个 OpenAiGptChatLanguageModel，然后可用于为评论生成补全。

from transforms.api import transform, Input, Output
from palantir_models.transforms import OpenAiGptChatLanguageModelInput
from palantir_models.models import OpenAiGptChatLanguageModel
from language_model_service_api.languagemodelservice_api_completion_v3 import GptChatCompletionRequest
from language_model_service_api.languagemodelservice_api import ChatMessage, ChatMessageRole


@transform(
    reviews=Input("/path/to/reviews/dataset"),
    model=OpenAiGptChatLanguageModelInput("ri.language-model-service..language-model.gpt-4_azure"),
    output=Output("/output/path"),
)
def compute_sentiment(ctx, reviews, model: OpenAiGptChatLanguageModel, output):
    def get_completions(review_content: str) -> str:
        system_prompt = "请分析以下评论并判断其情感倾向"
        request = GptChatCompletionRequest(
            [ChatMessage(ChatMessageRole.SYSTEM, system_prompt), ChatMessage(ChatMessageRole.USER, review_content)]
        )
        resp = model.create_chat_completion(request)
        return resp.choices[0].message.content

    reviews_df = reviews.pandas()
    reviews_df['sentiment'] = reviews_df['review_content'].apply(get_completions)
    out_df = ctx.spark_session.createDataFrame(reviews_df)
    return output.write_dataframe(out_df)

嵌入(Embeddings)¶

除了生成式语言模型，Palantir 还提供嵌入模型(embedding model)。以下示例展示了如何使用 palantir_models.transforms.GenericEmbeddingModelInput 对相同的 reviews 数据集计算嵌入。GenericEmbeddingModelInput 在运行时向转换提供一个 GenericEmbeddingModel，可用于计算每条评论的嵌入。嵌入被显式转换为浮点数，因为本体论(Ontology)向量属性要求如此。

from transforms.api import transform, Input, Output
from language_model_service_api.languagemodelservice_api_embeddings_v3 import GenericEmbeddingsRequest
from palantir_models.models import GenericEmbeddingModel
from palantir_models.transforms import GenericEmbeddingModelInput
from pyspark.sql.types import ArrayType, FloatType


@transform(
    reviews=Input("/path/to/reviews/dataset"),
    embedding_model=GenericEmbeddingModelInput("ri.language-model-service..language-model.text-embedding-ada-002_azure"),
    output=Output("/path/to/embedding/output")
)
def compute_embeddings(ctx, reviews, embedding_model: GenericEmbeddingModel, output):

    def internal_create_embeddings(val: str):
        return embedding_model.create_embeddings(GenericEmbeddingsRequest(inputs=[val])).embeddings[0]

    reviews_df = reviews.pandas()
    reviews_df['embedding'] = reviews_df['review_content'].apply(internal_create_embeddings)
    spark_df = ctx.spark_session.createDataFrame(reviews_df)
    out_df = spark_df.withColumn('embedding', spark_df['embedding'].cast(ArrayType(FloatType())))
    return output.write_dataframe(out_df)

使用视觉语言模型提取PDF文档内容¶

对于使用视觉语言模型的PDF文档提取工作流，我们建议使用 AIP Document Intelligence。AIP Document Intelligence 提供直观的界面，让您可以在文档上测试不同的提取策略、比较结果，并在部署到生产环境前评估提取质量。

部署配置后，系统会创建一个 Python 转换代码仓库，并自动配置您选择的提取方法、模型和任何自定义提示。您无需手动编写文档提取的转换代码。了解更多关于部署到 Python 转换的信息。