Breaking changes between Object Storage V1 and Object Storage V2(对象存储 V1 与对象存储 V2 之间的重大变更)¶
The transition from Object Storage V1 (Phonograph) to Object Storage V2 (OSv2) is a major architectural shift that requires a number of breaking changes. The legacy Object Storage V1 (Phonograph) has a large API surface area, exposing significant amounts of low-level database functionality directly. Conversely, the new Object Storage V2 architecture syncs objects through the Object Data Funnel service into specialized object databases to gain scale, performance, flexibility, and security improvements. This architectural transition leads to two main types of breaking changes:
- Specialized object databases, being designed for specific use cases, may not offer the same functionality as Object Storage V1. This may require workflows to be modeled differently in OSv2 than in OSv1.
- Separating dimensions of concern that had been consolidated in Object Storage V1 requires refactoring of some existing queries that interact directly with OSv1 APIs.
These breaking changes may block some OSv1-backed object types from migrating to OSv2 without refactoring or remediation. The documentation on migrating to OSv2 contains more information on how these issues are surfaced to the user, as well as how to perform a migration from OSv1 to OSv2.
Permanent breaking changes¶
This section provides a list of breaking changes between Object Storage V1 (Phonograph) and Object Storage V2 (OSv2). These breaking changes should be addressed before trying to migrate any object type to OSv2. All of these changes between OSv1 and OSv2 are intended to improve the quality of data going into the Ontology, ensure more deterministic behavior, and increase legibility across the platform.
Actions and edits¶
- OSv2 only supports user edits via Actions. All existing direct queries on OSv1 edit APIs must be refactored to use Actions before migrating to OSv2.
- OSv2 renames "writeback datasets" as "materializations". In OSv1, writeback datasets are required; in OSv2, materializations are optional. See Materializations for more details on materialized datasets.
Edit history¶
- The migration from OSv1 to OSv2 provides the option to preserve edit history on the object.
- Enabling this option will include edit history with the migration process. Note that this will incur additional compute costs for processing and storage.
- If this is not enabled, previous edits from OSv1 will persist; however, the full edit history from OSv1 will not be available in OSv2. Once the migration is completed, it will not be possible to recover the full edit history from OSv1. Users are required to acknowledge that when migrating to Object Storage V2, the history of user edits (Actions) will not be preserved except for Action Logs.
- Track user edit history will need to be enabled for object types in OSv2 within the Ontology Manager and object views will need to be updated to add the Edit History widget to display user edit histories.
- Users may also configure Action Log object types to otherwise capture how Actions affect objects; also, see the Action Log Timeline widget to display a history of Actions across targeted objects.
Data restrictions¶
OSv2 enforces data restrictions on primary keys and property types to ensure the quality of data going into the ontology. For the full list of restrictions, see the documentation on OSv2 data restrictions.
Changelog and incremental updates¶
- For object types with a changelog datasource, OSv2 does not consider the "latest timestamp wins" semantic used by legacy changelog datasets.
- OSv2 indexes all pipelines incrementally by default; all relevant changelog calculations are automatically performed in the background by Funnel, rendering the "changelog python decorator" obsolete.
- OSv2 has stricter validations on geopoint properties.
Monitoring¶
- OSv2 object types can only be monitored by monitoring views, which replaces the Object Storage V1 (Phonograph) sync status health check.
Object Set Service (OSS) changes¶
- OSS APIs do not have query string support; OSv1 does have query string support.
- OSS cardinality metrics are not supported on object sets that contain objects from both Object Storage V1 and Object Storage V2, such as multi-backend object sets.
Temporary breaking changes¶
This section lists out the current feature gap between Object Storage V1 (Phonograph) and Object Storage V2 (OSv2). These features are in development and the list will be updated as the breaking changes are resolved.
- OSv2 currently does not support connecting schedules where the path between the upstream and downstream targets would go through a materialization.
- OSv2 currently does not support custom analyzers.
- Foundry Rules Archetypes currently does not support the full feature set of OSv2. For example, object types backed by multiple datasources and object types with multiple materializations are not supported. You can refer to the Foundry Rules documentation for more details.
中文翻译¶
对象存储 V1 与对象存储 V2 之间的重大变更¶
从对象存储 V1(Phonograph)过渡到对象存储 V2(OSv2)是一项重大的架构变革,需要引入多项重大变更。旧版对象存储 V1(Phonograph)拥有庞大的 API 接口面积,直接暴露了大量底层数据库功能。相比之下,新的对象存储 V2 架构通过对象数据漏斗服务(Object Data Funnel)将对象同步到专门的对象数据库中,从而在规模、性能、灵活性和安全性方面获得提升。这一架构转型主要带来两类重大变更:
- 专门的对象数据库针对特定用例设计,可能无法提供与对象存储 V1 相同的功能。这可能导致在 OSv2 中需要以不同于 OSv1 的方式对工作流进行建模。
- 将对象存储 V1 中整合的关注维度进行分离,需要对一些直接与 OSv1 API 交互的现有查询进行重构。
这些重大变更可能会阻碍部分基于 OSv1 的对象类型在未经重构或修复的情况下迁移到 OSv2。关于迁移到 OSv2 的文档包含了更多关于如何向用户呈现这些问题,以及如何执行从 OSv1 到 OSv2 迁移的信息。
永久性重大变更¶
本节列出了对象存储 V1(Phonograph)与对象存储 V2(OSv2)之间的重大变更。在尝试将任何对象类型迁移到 OSv2 之前,应解决这些重大变更。OSv1 与 OSv2 之间的所有这些变更旨在提高进入本体论(Ontology)的数据质量,确保更确定性的行为,并提升整个平台的可读性。
操作与编辑¶
- OSv2 仅支持通过操作(Actions)进行用户编辑。在迁移到 OSv2 之前,所有对 OSv1 编辑 API 的现有直接查询都必须重构为使用操作(Actions)。
- OSv2 将"回写数据集(writeback datasets)"重命名为"物化(materializations)"。在 OSv1 中,回写数据集是必需的;在 OSv2 中,物化是可选的。有关物化数据集的更多详细信息,请参阅物化。
编辑历史¶
- 从 OSv1 到 OSv2 的迁移提供了保留对象编辑历史的选项。
- 启用此选项将在迁移过程中包含编辑历史。请注意,这将产生额外的计算成本用于处理和存储。
- 如果未启用此选项,OSv1 中的先前编辑将保留;但 OSv1 的完整编辑历史将无法在 OSv2 中使用。迁移完成后,将无法恢复 OSv1 的完整编辑历史。用户必须确认,在迁移到对象存储 V2 时,用户编辑(操作)的历史将不会被保留,操作日志(Action Logs)除外。
- 需要在 Ontology Manager 中为 OSv2 中的对象类型启用跟踪用户编辑历史,并且需要更新对象视图以添加编辑历史小部件来显示用户编辑历史。
- 用户也可以配置操作日志(Action Log)对象类型,以其他方式捕获操作如何影响对象;另请参阅操作日志时间线小部件,以显示跨目标对象的操作历史。
数据限制¶
OSv2 对主键和属性类型强制执行数据限制,以确保进入本体论的数据质量。有关限制的完整列表,请参阅 OSv2 数据限制文档。
变更日志与增量更新¶
- 对于具有变更日志数据源的对象类型,OSv2 不考虑旧版变更日志数据集使用的"最新时间戳获胜"语义。
- OSv2 默认对所有管道进行增量索引;所有相关的变更日志计算均由 Funnel 在后台自动执行,使得"变更日志 Python 装饰器"变得过时。
- OSv2 对地理点属性有更严格的验证。
监控¶
- OSv2 对象类型只能通过监控视图进行监控,这取代了对象存储 V1(Phonograph)同步状态健康检查。
对象集服务(OSS)变更¶
- OSS API 不支持查询字符串;OSv1 支持查询字符串。
- OSS 基数指标不支持包含来自对象存储 V1 和对象存储 V2 的对象集,例如多后端对象集。
临时性重大变更¶
本节列出了对象存储 V1(Phonograph)与对象存储 V2(OSv2)之间当前存在的功能差距。这些功能正在开发中,随着重大变更的解决,此列表将进行更新。
- OSv2 当前不支持连接调度,其中上游和下游目标之间的路径需要经过物化。
- OSv2 当前不支持自定义分析器。
- Foundry Rules Archetypes 当前不支持 OSv2 的完整功能集。例如,不支持由多个数据源支持的对象类型以及具有多个物化的对象类型。您可以参考 Foundry Rules 文档了解更多详细信息。