跳转至

Phase 4: Roles and responsibilities（阶段四：角色与职责）

The following are new roles and responsibilities for Phase 4 of the Foundry Program team, in addition to the roles established in Phase 1, Phase 2, and Phase 3.

Production Engineer

Responsibilities

  • Own production pipelines; manage the release process to ensure that production pipelines adhere to the organization's standards and execute reliably.
  • Collaborate closely with SMEs to address any requested changes to the Ontology and intake feedback on pipeline performance and usability in production workflows.
  • Own the approval process of Pull Requests (PRs) into the master branch of the production pipeline, and manage approvals to run pipeline builds on the master branch.
  • Work with Pipeline and Data Source Developers to define health checks across key datasets and implement continuous monitoring of the health checks across releases.
  • The Production Engineer fundamentally owns the monitoring of the healthy execution of the production pipeline. This includes monitoring of data refresh on different sources, execution of pipeline builds, and assisting to triage support requests when applicable.

Profile and required skills

  • Data engineering skills (SQL, Python, PySpark/Spark)
  • Knowledge of data cleaning and quality best practices
  • Knowledge of version control best practices
  • Familiarity with agents / VMs / connecting systems

Relevant Foundry applications and resources

Pipeline Developer

Responsibilities

  • Build a common ontological layer that provides data assets for use by all use cases.
  • Define this common ontological layer and build the transformations required to take data from the source structure to the ontology structure.
  • Work closely alongside Production Engineers and Ontology Managers to implement the ontological layer that is designed by Ontology Managers and is leveraged in the data provided to use case teams.

Profile and required skills

  • Data engineering skills (SQL, Python, PySpark/Spark)
  • Knowledge of data cleaning and quality best practices
  • Knowledge of version control best practices
  • Familiarity with agents / VMs / connecting systems

Relevant Foundry applications and resources

Object View Developer

Responsibilities

  • Object View Developers develop object views based on the Ontology.
  • Intake requests from platform management, use case teams, and even business owners and end users for new features within object views, as well as new object views for new objects within the Ontology. Work closely with these teams to understand business requirements and ensure maximum utility for the object views.
  • Collaborate heavily with Ontology Managers to identify what objects and object views are expected to be built, and what backing datasets are required to create them.
  • Work with Production Engineers and Pipeline Developers to create processes for information sharing and bilateral awareness of development initiatives.
  • Build out prototypical views in Contour for eventual promotion to the production object view.
  • Build a process for how to make changes to ontology objects and object views to enable scaling to multiple developers and to provide visibility into object view development requirements, constraints, and best practices.

Profile and required skills

  • Data engineering skills (SQL, Python, PySpark/Spark)
  • Knowledge of version control best practices
  • Database knowledge
  • Business domain knowledge
  • Project management and change management skills
  • Proven ability to write comprehensive technical documentation
  • Ability to communicate between technical and non-technical stakeholders

Relevant Foundry applications and resources

中文翻译

阶段四：角色与职责

以下是在阶段一、阶段二和阶段三已确立角色基础上，为Foundry项目团队阶段四新增的角色与职责。

生产工程师（Production Engineer）

职责

  • 负责生产流水线（production pipelines）的运维；管理发布流程，确保生产流水线符合组织标准并稳定执行。
  • 与领域专家（SMEs）紧密协作，处理对本体（Ontology）的变更请求，并收集关于流水线性能及在生产工作流中可用性的反馈。
  • 负责审批将拉取请求（Pull Requests, PRs）合并至生产流水线主分支（master branch）的流程，并管理在主分支上运行流水线构建的审批。
  • 与流水线开发者和数据源开发者合作，为关键数据集定义健康检查（health checks），并在各版本发布中实施持续监控。
  • 生产工程师从根本上负责监控生产流水线的健康运行，包括监控不同数据源的数据刷新、流水线构建的执行，并在适用时协助对支持请求进行分类处理。

背景与所需技能

  • 数据工程技能（SQL、Python、PySpark/Spark）
  • 了解数据清洗与质量最佳实践
  • 了解版本控制最佳实践
  • 熟悉代理（agents）/虚拟机（VMs）/系统连接

相关Foundry应用与资源

流水线开发者（Pipeline Developer）

职责

  • 构建通用的本体层（ontological layer），为所有用例提供数据资产。
  • 定义该通用本体层，并构建所需的数据转换（transformations），将数据从源结构转换为本体结构。
  • 与生产工程师和本体管理者（Ontology Managers）紧密协作，实施由本体管理者设计、并在提供给用例团队的数据中使用的本体层。

背景与所需技能

  • 数据工程技能（SQL、Python、PySpark/Spark）
  • 了解数据清洗与质量最佳实践
  • 了解版本控制最佳实践
  • 熟悉代理/虚拟机/系统连接

相关Foundry应用与资源

对象视图开发者（Object View Developer）

职责

  • 基于本体开发对象视图（object views）。
  • 接收来自平台管理、用例团队乃至业务负责人和最终用户的需求，涉及对象视图的新功能，以及为本体中新增对象创建新对象视图。与这些团队紧密合作，理解业务需求，确保对象视图的最大效用。
  • 与本体管理者密切协作，确定需要构建哪些对象和对象视图，以及创建它们所需的支撑数据集（backing datasets）。
  • 与生产工程师和流水线开发者合作，建立信息共享和开发计划双向感知的流程。
  • 在Contour中构建原型视图，以便最终升级为生产对象视图。
  • 建立对本体对象和对象视图进行变更的流程，以支持多开发者扩展，并提供对象视图开发需求、约束和最佳实践的可见性。

背景与所需技能

  • 数据工程技能（SQL、Python、PySpark/Spark）
  • 了解版本控制最佳实践
  • 数据库知识
  • 业务领域知识
  • 项目管理和变更管理技能
  • 具备编写全面技术文档的成熟能力
  • 能够在技术与非技术利益相关者之间进行沟通

相关Foundry应用与资源