Add a geotemporal series output（添加地理时间序列输出）

You can add a geotemporal series sync output to pipelines to make your batch or streaming geotemporal data in Foundry available in downstream applications, such as the Map application. Geotemporal observations from Foundry are written in Pipeline Builder. Learn more about different output types and geotemporal series.

Prerequisites

To use geotemporal series in Pipeline Builder, they must be enabled for your enrollment. Contact your Palantir representative for more information about enabling this feature.

Create a geotemporal series output

Open the Pipeline outputs panel located on the right side of the Pipeline Builder interface. Then, select Add > Geotemporal series sync > Add to create a new geotemporal series.

:::callout{theme="neutral"} If you cannot see the geotemporal series output type in Pipeline Builder, check whether your enrollment meets the prerequisite stated above. :::

Mapping primary fields

All geotemporal series outputs in Pipeline Builder have primary fields, or intrinsic fields, that must be produced as part of your pipeline's upstream transformation logic. These fields control the translation from rows in a pipeline to geotemporal observations. Please refer to geotemporal series data modeling for more information about primary fields. These primary fields include:

• Series ID: A string value used as a unique identifier for each track in an integration, such as a flight identifier used to distinguish unique flights. Refer to geotemporal series data modeling for more examples on choosing a series ID.
• Timestamp: A value representing the time of a given observation.
• Position: A GeoPoint value that represents the location at the time of an observation. GeoPoint values in Pipeline Builder can be created from latitude and longitude coordinates using the Construct geopoint column expression in upstream transformation logic.
• Altitude: An optional value of type Double that represents the height above the surface of the Earth in meters.

Mapping additional fields

In addition to the required fields mentioned above, you can map custom fields to output observations. To do so, navigate to the Properties section of the geotemporal series output panel and select Add Property. Select an upstream transformation logic output column to have the corresponding value present in downstream observations. You must specify a name, whether the mapping should be live or static, and optionally, documentation for what the field represents. Live properties change with each observation, such as the heading of a plane. Static properties are constant across the series and are used to denormalize for search and filtering capabilities, such as aircraft type.

In addition to the standard data types in Pipeline Builder, geotemporal series outputs support mapping special geometric types to their equivalents in the geotemporal series data model. Please see Pipeline Builder's geospatial documentation for more detail on these types:

• GeoPoint: A single point, which can be rendered on downstream maps and can be different from the point set in the "Position" primary field.
• Geometry: A GeoJSON object which can be rendered on downstream maps.

Advanced settings

In addition to basic output mappings, geotemporal series outputs support the following advanced and optional settings:

• Time to live: A long type value representing the amount of time in milliseconds after which data will be considered stale or expired. Stale data will not be rendered in downstream applications. Note that the time to live value does not affect data indexed to the dataset archive.
• Indexing settings: Allows users to specify how they would like to index geotemporal series observations. There are two supported data storage options: live streaming and dataset archive. Live streaming enables rendering points and tracks in real time, while dataset archive retains all series data in permanent storage indexed for geotemporal search.
• Observation type: A string type value representing a logical categorization used for filtering data in downstream applications, for example "GPS ping" or "manual report".
• Style configuration: Allows users to configure the appearance of an observation in downstream applications. Supported attributes include icon size, opacity, rotation, symbology, title, geometry width, geometry color, geometry fill, geometry opacity, and track color.
• Observation schema: You can optionally select an existing observation schema as part of your geotemporal series. Doing so will pre-populate all fields associated with that schema as properties to be mapped. To support backwards compatibility, you will not be able to delete, change, or toggle an existing field between live and static. If you do not select an existing schema, a new schema will be created for your integration upon first successful deployment of the pipeline output.
• Source system: Source systems are groupings of different geotemporal series syncs that you can optionally reuse. If you select an existing source system, the resulting geotemporal series integration will be upserted to the specified source system upon successful deployment. Existing integrations in the source system will not be modified. If you do not select an existing source system, a new one will be created upon first successful deployment of the pipeline output. By default, all geotemporal series outputs within a given pipeline will be grouped into a source system corresponding to that pipeline.
• Destination namespace: Users can specify the destination namespace that a pipeline will export observations to, if they have access to multiple namespaces with geotemporal output series configured. More details can be found below. Contact your Palantir representative for guidance on configuring additional destinations.

Security

The security of downstream geotemporal series observations is derived from your pipeline input data:

• Mandatory permissions:
• Downstream observations will be secured by the latest output transaction or streaming view on the geotemporal series output dataset in Pipeline Builder. This resource will propagate mandatory or classification Markings inherited from input datasets to the pipeline.
• If upstream input data Markings change between builds of a geotemporal series output, the subsequent build will fail in order to prevent unintended changes in previously integrated data Markings. In this case, you must delete and re-create the geotemporal series ouput with the appropriate Markings.
• All pipeline inputs must have organization Markings with access to the destination namespace configured for the corresponding geotemporal series output. Organization Markings present on the geotemporal series output must be a superset of the organizations with access to the output's namespace.
• Discretionary permissions:
• When a geotemporal series output is first deployed, all groups with read permissions and above on the parent project will be granted view access to the downstream geotemporal series output. Subsequent changes to discretionary permissions of the parent project will not propagate to downstream observations. Please contact your Palantir representative if your geotemporal series workflow requires more granular discretionary permissions.

Consult the documentation below for more information on mandatory and discretionary permissions:

• Securing a data foundation
• Protecting sensitive data

Troubleshooting

Observations should flow into your geotemporal series shortly after a pipeline has been deployed and a build has been triggered. If data is not appearing in downstream applications, you can troubleshoot using the following steps:

• Verify that data is appearing in the observations output dataset in Pipeline Builder by navigating to the Outputs panel on the right side and selecting the three dots next to your output, then selecting View backing dataset.

• Check for errors in the error dataset by selecting View errors dataset in the Outputs panel mentioned above.
• Check for errors logs indicating a runtime failure in the currently running pipeline build. This is accessible from the History tab in Pipeline Builder.
• Verify that there are no security configuration errors in downstream applications by ensuring that downstream resources contain the necessary classification Markings, mandatory Markings, and discretionary groups to view the underlying geotemporal series.

---

中文翻译

添加地理时间序列输出

您可以为流水线添加地理时间序列同步输出，使 Foundry 中的批处理或流式地理时间序列数据可供下游应用程序（如地图应用程序）使用。Foundry 中的地理时间序列观测数据通过 Pipeline Builder 写入。了解更多关于不同输出类型和地理时间序列的信息。

前提条件

要在 Pipeline Builder 中使用地理时间序列，必须为您的注册环境启用该功能。请联系您的 Palantir 代表以获取有关启用此功能的更多信息。

创建地理时间序列输出

打开位于 Pipeline Builder 界面右侧的流水线输出面板。然后，选择添加 > 地理时间序列同步 > 添加以创建新的地理时间序列。

:::callout{theme="neutral"} 如果您在 Pipeline Builder 中看不到地理时间序列输出类型，请检查您的注册环境是否满足上述前提条件。 :::

映射主要字段

Pipeline Builder 中的所有地理时间序列输出都有主要字段（primary fields），或称内在字段（intrinsic fields），这些字段必须作为流水线上游转换逻辑的一部分生成。这些字段控制从流水线中的行到地理时间序列观测数据的转换。请参考地理时间序列数据建模以获取有关主要字段的更多信息。这些主要字段包括：

• 系列 ID（Series ID）： 一个字符串值，用作集成中每条轨迹的唯一标识符，例如用于区分不同航班的航班标识符。请参考地理时间序列数据建模以获取更多关于选择系列 ID 的示例。
• 时间戳（Timestamp）： 表示给定观测时间的值。
• 位置（Position）： 一个 GeoPoint 值，表示观测时的位置。Pipeline Builder 中的 GeoPoint 值可以通过在上游转换逻辑中使用构建地理点列表达式，从纬度和经度坐标创建。
• 海拔（Altitude）： 一个可选的 Double 类型值，表示高于地球表面的高度（以米为单位）。

映射附加字段

除了上述必填字段外，您还可以将自定义字段映射到输出观测数据。为此，请导航到地理时间序列输出面板的属性部分，然后选择添加属性。选择一个上游转换逻辑输出列，使相应的值出现在下游观测数据中。您必须指定一个名称，映射应为实时（live）还是静态（static），以及可选地提供该字段含义的文档说明。实时属性随每次观测而变化，例如飞机的航向。静态属性在整个系列中保持不变，用于反规范化以实现搜索和过滤功能，例如飞机类型。

除了 Pipeline Builder 中的标准数据类型外，地理时间序列输出还支持将特殊几何类型映射到地理时间序列数据模型中的对应类型。有关这些类型的更多详细信息，请参阅 Pipeline Builder 的地理空间文档：

• GeoPoint： 一个单点，可以在下游地图上渲染，并且可以与"位置"主要字段中设置的点不同。
• 几何（Geometry）： 一个 GeoJSON 对象，可以在下游地图上渲染。

高级设置

除了基本输出映射外，地理时间序列输出还支持以下高级和可选设置：

• 生存时间（Time to live）： 一个 long 类型值，表示数据被视为过期或失效的时间量（以毫秒为单位）。过期数据将不会在下游应用程序中渲染。请注意，生存时间值不会影响索引到数据集归档的数据。
• 索引设置（Indexing settings）： 允许用户指定如何索引地理时间序列观测数据。支持两种数据存储选项：实时流式传输和数据集归档。实时流式传输支持实时渲染点和轨迹，而数据集归档将所有系列数据保留在永久存储中，并针对地理时间序列搜索进行索引。
• 观测类型（Observation type）： 一个 string 类型值，表示用于在下游应用程序中过滤数据的逻辑分类，例如"GPS 信号"或"手动报告"。
• 样式配置（Style configuration）： 允许用户配置观测数据在下游应用程序中的外观。支持的属性包括图标大小、不透明度、旋转、符号、标题、几何宽度、几何颜色、几何填充、几何不透明度和轨迹颜色。
• 观测模式（Observation schema）： 您可以选择性地选择一个现有的观测模式作为地理时间序列的一部分。这样做将预填充与该模式关联的所有字段作为要映射的属性。为了支持向后兼容性，您将无法删除、更改或将现有字段在实时和静态之间切换。如果您不选择现有模式，则在流水线输出首次成功部署时，将为您的集成创建一个新模式。
• 源系统（Source system）： 源系统是不同地理时间序列同步的分组，您可以选择性地重复使用。如果您选择现有的源系统，则生成的地理时间序列集成将在成功部署时被更新到指定的源系统。源系统中的现有集成将不会被修改。如果您不选择现有的源系统，则在流水线输出首次成功部署时将创建一个新的源系统。默认情况下，给定流水线内的所有地理时间序列输出将被分组到与该流水线对应的源系统中。
• 目标命名空间（Destination namespace）： 如果用户有权访问多个配置了地理时间序列输出命名空间，则可以指定流水线将观测数据导出到的目标命名空间。更多详细信息请参见下文。请联系您的 Palantir 代表以获取配置其他目标的指导。

安全性

下游地理时间序列观测数据的安全性源自您的流水线输入数据：

• 强制权限：
• 下游观测数据将由 Pipeline Builder 中地理时间序列输出数据集的最新输出事务或流式视图保护。此资源将传播从输入数据集继承到流水线的强制或分类标记。
• 如果上游输入数据标记在地理时间序列输出的两次构建之间发生变化，则后续构建将失败，以防止先前集成数据标记发生意外更改。在这种情况下，您必须使用适当的标记删除并重新创建地理时间序列输出。
• 所有流水线输入必须具有组织标记，并且能够访问为相应地理时间序列输出配置的目标命名空间。地理时间序列输出上的组织标记必须是能够访问输出命名空间的组织的超集。
• 自由裁量权限：
• 当首次部署地理时间序列输出时，对父项目具有读取权限及以上的所有组将被授予对下游地理时间序列输出的查看权限。父项目自由裁量权限的后续更改不会传播到下游观测数据。如果您的地理时间序列工作流需要更细粒度的自由裁量权限，请联系您的 Palantir 代表。

请查阅以下文档以获取有关强制权限和自由裁量权限的更多信息：

• 保护数据基础
• 保护敏感数据

故障排除

在流水线部署并触发构建后，观测数据应很快流入您的地理时间序列。如果数据未出现在下游应用程序中，您可以使用以下步骤进行故障排除：

• 通过导航到右侧的输出面板，选择输出旁边的三个点，然后选择查看支持数据集，验证数据是否出现在 Pipeline Builder 的观测输出数据集中。

• 通过在上述输出面板中选择查看错误数据集来检查错误数据集中的错误。
• 检查错误日志，查看当前正在运行的流水线构建中是否存在运行时故障。这可以通过 Pipeline Builder 中的历史记录选项卡访问。
• 通过确保下游资源包含必要的分类标记、强制标记和自由裁量组以查看底层地理时间序列，验证下游应用程序中没有安全配置错误。