whylogs
flytekitplugins-whylogs
whylogs is an open source library for logging any kind of data. With whylogs,
pip install flytekitplugins-whylogsQuick Start(example, may need adjustment)
See full examplespip install flytekitplugins-whylogs
from flytekit import task, workflow
from flytekitplugins.whylogs import WhylogsConstraintsRenderer, WhylogsSummaryDriftRenderer, WhylogsDatasetProfileTransformer
config = WhylogsConstraintsRenderer(...)
@task
def my_task() -> None:
...
@workflow
def my_workflow() -> None:
my_task()Available Imports (3)
Creates a whylogs' Constraints report from a `Constraints` object.
extends class — configuration or data structure for plugin setup
from flytekitplugins.whylogs import WhylogsConstraintsRenderer
Creates a whylogs' Summary Drift report from two pandas DataFrames.
extends class — configuration or data structure for plugin setup
from flytekitplugins.whylogs import WhylogsSummaryDriftRenderer
Transforms whylogs Dataset Profile Views to and from a Schema (typed/untyped).
extends TypeTransformer — converts python types to/from flyte-native types
from flytekitplugins.whylogs import WhylogsDatasetProfileTransformer
Related Plugins
Great Expectations
Great Expectations helps enforce data quality. The plugin supports the usage of Great Expectations as task and type.
Pandera
Flytekit python natively supports many data types, including a FlyteSchema type for type-annotating pandas DataFrames. The Flytekit Pandera plugin provides an alternative for defining DataFrame schemas by integrating with Pandera, a runtime data validation tool for pandas DataFrames.
