SGLang

flyteplugins-sglang

Flyte SDK (v2)Model Servingsglanginferencellmservinggpu

Serve large language models using SGLang with Flyte Apps.

Install

pip install flyteplugins-sglang

Quick Start(example, may need adjustment)

pip install flyteplugins-sglang

from flytekit import task, workflow
from flyteplugins.sglang import DEFAULT_SGLANG_IMAGE, SGLangAppEnvironment

config = DEFAULT_SGLANG_IMAGE(...)

@task
def my_task() -> None:
    ...

@workflow
def my_workflow() -> None:
    my_task()

Available Imports (2)

typeDEFAULT_SGLANG_IMAGE

Configuration type for SGLang.

from flyteplugins.sglang import DEFAULT_SGLANG_IMAGE

configSGLangAppEnvironment

App environment backed by SGLang for serving large language models.

extends dataclass — configuration or data structure for plugin setup

from flyteplugins.sglang import SGLangAppEnvironment

Related Plugins

vLLM

Model Serving

Serve large language models using vLLM with Flyte Apps.

Inference

Model Serving

Serve models natively in Flyte tasks using inference providers like NIM, Ollama, and others.

Dgxc-lepton

Model Serving

A professional Flytekit plugin that enables seamless deployment and management of AI inference endpoints using Lepton AI infrastructure within Flyte workflows.