Kubeflow MPI
flytekitplugins-kfmpi
This plugin uses the Kubeflow MPI Operator and provides an extremely simplified interface for executing distributed training.
pip install flytekitplugins-kfmpiQuick Start(example, may need adjustment)
See full examplespip install flytekitplugins-kfmpi
from flytekit import task, workflow
from flytekitplugins.kfmpi import CleanPodPolicy, HorovodJob, Launcher, MPIJob
@task(task_config=HorovodJob(...))
def my_task() -> None:
...
@workflow
def my_workflow() -> None:
my_task()Available Imports (7)
CleanPodPolicy describes how to deal with pods when the job is finished.
extends Enum — enumeration of predefined options
from flytekitplugins.kfmpi import CleanPodPolicy
Configuration for an executable [`Horovod Job using MPI operator`](https://github.com/kubeflow/mpi-operator).
extends dataclass — configuration or data structure for plugin setup
from flytekitplugins.kfmpi import HorovodJob
Launcher replica configuration.
extends dataclass — configuration or data structure for plugin setup
from flytekitplugins.kfmpi import Launcher
Configuration for an executable [`MPI Job`](https://github.com/kubeflow/mpi-operator).
extends dataclass — configuration or data structure for plugin setup
from flytekitplugins.kfmpi import MPIJob
RestartPolicy describes how the replicas should be restarted.
extends Enum — enumeration of predefined options
from flytekitplugins.kfmpi import RestartPolicy
RunPolicy describes some policy to apply to the execution of a kubeflow job.
extends dataclass — configuration or data structure for plugin setup
from flytekitplugins.kfmpi import RunPolicy
Worker replica configuration.
extends dataclass — configuration or data structure for plugin setup
from flytekitplugins.kfmpi import Worker
Related Plugins
Dask
Flyte can execute dask jobs natively on a Kubernetes Cluster, which manages the virtual dask cluster's lifecycle
Dask
Flyte can execute dask jobs natively on a Kubernetes Cluster, which manages the virtual dask cluster's lifecycle
Kubeflow PyTorch
This plugin uses the Kubeflow PyTorch Operator and provides an extremely simplified interface for executing distributed training using various PyTorch backends.
Kubeflow TensorFlow
This plugin uses the Kubeflow TensorFlow Operator and provides an extremely simplified interface for executing distributed training using various TensorFlow backends.
