Hive

flytekitplugins-hive

FlytekitDatabases & Warehouseshivehadoopsqlbig-data

Flyte backend can be connected with various Hive services. Once enabled, it allows you to query a Hive service (e.g., Qubole) and retrieve typed schema (optionally).

Install

pip install flytekitplugins-hive

Quick Start(example, may need adjustment)

See full examples

pip install flytekitplugins-hive

from flytekit import task, workflow
from flytekitplugins.hive import HiveConfig, HiveSelectTask, HiveTask

@task(task_config=HiveSelectTask(...))
def my_task() -> None:
    ...

@workflow
def my_workflow() -> None:
    my_task()

Available Imports (3)

configHiveConfig

HiveConfig should be used to configure a Hive Task.

extends dataclass — configuration or data structure for plugin setup

from flytekitplugins.hive import HiveConfig

taskHiveSelectTask

Task for Hive.

from flytekitplugins.hive import HiveSelectTask

taskHiveTask

This is the simplest form of a Hive Task, that can be used even for tasks that do not produce any output.

extends SQLTask — a flyte task that can be used in workflows

from flytekitplugins.hive import HiveTask

Related Plugins

Bigquery

Databases & Warehouses

BigQuery enables us to build data-intensive applications without operational burden. Flyte backend can be connected with the BigQuery service. Once enabled, it can allow you to query a BigQuery table.