BigQuery compute pushdown（BigQuery 计算下推 (compute pushdown)）

To use compute pushdown with BigQuery, create a Python repository and install the most recent version of the transforms-tables library.

An Ibis ↗ connection to BigQuery will be established based on the connection details of the BigQuery tables configured as inputs and/or outputs. Data can be transformed using the Ibis DataFrame API. For complete guidance on the Ibis API, consult the Ibis documentation ↗.

:::callout{theme="neutral"} Note the use of Foundry lightweight API syntax rather than Foundry Spark syntax. :::

An example of a BigQuery-backed Ibis transform is shown below:

from transforms.api import transform
from transforms.tables import (
    IbisInput,
    IbisOutput,
    TableInput,
    TableOutput,
    BigQueryTable
)


@transform.bigquery.using(
    source_table=TableInput("ri.tables.main.table.1234"),
    output_table=TableOutput(
        "ri.tables.main.table.5678",
        "ri.magritte..source.1234",
        BigQueryTable("PROJECT", "DATASET", "TABLE"),
    ),
)
def compute(source_table: IbisInput, output_table: IbisOutput):
    # Data transformation with Ibis
    table = source_table.table()
    output_table.write(table)

:::callout{theme="neutral"} Incremental computation using the @incremental decorator is not currently supported when using compute pushdown to BigQuery. :::

---

中文翻译

---

BigQuery 计算下推 (compute pushdown)

要使用 BigQuery 的计算下推功能，请创建一个 Python 仓库，并安装最新版本的 transforms-tables 库。

系统将根据配置为输入和/或输出的 BigQuery 表的连接详情，建立与 BigQuery 的 Ibis ↗ 连接。数据可以使用 Ibis DataFrame API 进行转换。有关 Ibis API 的完整指南，请查阅 Ibis 文档 ↗。

:::callout{theme="neutral"} 请注意，此处使用的是 Foundry 轻量级 API 语法，而非 Foundry Spark 语法。 :::

以下是一个基于 BigQuery 的 Ibis 转换示例：

from transforms.api import transform
from transforms.tables import (
    IbisInput,
    IbisOutput,
    TableInput,
    TableOutput,
    BigQueryTable
)


@transform.bigquery.using(
    source_table=TableInput("ri.tables.main.table.1234"),
    output_table=TableOutput(
        "ri.tables.main.table.5678",
        "ri.magritte..source.1234",
        BigQueryTable("PROJECT", "DATASET", "TABLE"),
    ),
)
def compute(source_table: IbisInput, output_table: IbisOutput):
    # 使用 Ibis 进行数据转换
    table = source_table.table()
    output_table.write(table)

:::callout{theme="neutral"} 在使用 BigQuery 计算下推时，当前不支持使用 @incremental 装饰器进行增量计算。 :::