AWS Athena
flytekitplugins-athena
Flyte backend can be connected with Athena. Once enabled, it allows you to query AWS Athena service (Presto + ANSI SQL Support) and retrieve typed schema (optionally).
pip install flytekitplugins-athenaQuick Start(example, may need adjustment)
See full examplespip install flytekitplugins-athena
from flytekit import task, workflow
from flytekitplugins.athena import AthenaConfig, AthenaTask
@task(task_config=AthenaTask(...))
def my_task() -> None:
...
@workflow
def my_workflow() -> None:
my_task()Available Imports (2)
AthenaConfig should be used to configure a Athena Task.
extends dataclass — configuration or data structure for plugin setup
from flytekitplugins.athena import AthenaConfig
This is the simplest form of a Athena Task, that can be used even for tasks that do not produce any output.
extends SQLTask — a flyte task that can be used in workflows
from flytekitplugins.athena import AthenaTask
Related Plugins
AWS Batch
Flyte backend can be connected with AWS batch. Once enabled, it allows you to run flyte task on AWS batch service
AWS SageMaker
The plugin currently features a SageMaker deployment connector.
Databricks
This plugin provides Databricks integration for Flyte, enabling you to run Spark jobs on Databricks as Flyte tasks.
Kubernetes Pod
By default, Flyte tasks decorated with @task are essentially single functions that are loaded in one container. But often, there is a need to run a job with more than one container.
