A Flyte SDK (v2) version of this plugin is available as flyteplugins-bigquery.

Bigquery

flytekitplugins-bigquery

FlytekitDatabases & Warehousesbigquerygooglesqlwarehouse

BigQuery enables us to build data-intensive applications without operational burden. Flyte backend can be connected with the BigQuery service. Once enabled, it can allow you to query a BigQuery table.

Install

pip install flytekitplugins-bigquery

Quick Start(example, may need adjustment)

See full examples

pip install flytekitplugins-bigquery

from flytekit import task, workflow
from flytekitplugins.bigquery import BigQueryConnector, BigQueryConfig, BigQueryTask

@task(task_config=BigQueryTask(...))
def my_task() -> None:
    ...

@workflow
def my_workflow() -> None:
    my_task()

Available Imports (3)

connectorBigQueryConnector

Backend connector for Bigquery.

from flytekitplugins.bigquery import BigQueryConnector

configBigQueryConfig

BigQueryConfig should be used to configure a BigQuery Task.

extends dataclass — configuration or data structure for plugin setup

from flytekitplugins.bigquery import BigQueryConfig

taskBigQueryTask

This is the simplest form of a BigQuery Task, that can be used even for tasks that do not produce any output.

extends SQLTask — a flyte task that can be used in workflows

from flytekitplugins.bigquery import BigQueryTask

Dependencies

google-cloud-bigquerygoogle-cloud-bigquery-storagenumpy

Related Plugins

BigQuery

Databases & Warehouses

This plugin provides BigQuery integration for Flyte, enabling you to run BigQuery queries as Flyte tasks.

Snowflake

Databases & Warehouses

Snowflake enables us to build data-intensive applications without operational burden. Flyte backend can be connected with the Snowflake service. Once enabled, it can allow you to query a Snowflake service.