Model Serving

flytekitplugins-dgxc-lepton

A professional Flytekit plugin that enables seamless deployment and management of AI inference endpoints using Lepton AI infrastructure within Flyte workflows.

flytekitplugins-inference

Serve models natively in Flyte tasks using inference providers like NIM, Ollama, and others.

flytekitplugins-onnxpytorch

This plugin allows you to generate ONNX models from your PyTorch models.

flytekitplugins-onnxscikitlearn

This plugin allows you to generate ONNX models from your ScikitLearn models.

flytekitplugins-onnxtensorflow

This plugin allows you to generate ONNX models from your TensorFlow Keras models.

flytekitplugins-openai

The plugin currently features ChatGPT and Batch API connectors.

flyteplugins-openai

This plugin provides a drop-in replacement for OpenAI packages. It provides

flyteplugins-sglang

Serve large language models using SGLang with Flyte Apps.

flyteplugins-vllm

Serve large language models using vLLM with Flyte Apps.