Metaflow, data science project management?

Metaflow is a human-friendly Python library that helps scientists and engineers build and manage real-life data science projects. Metaflow was originally developed at Netflix to boost productivity of data scientists who work on a wide variety of projects from classical statistics to state-of-the-art deep learning.

I’m quite intrigued by this… I see it as a way to have an API for all parts of a data science pipeline.

Anyone else have any thoughts about it?

How is this different from Airflow? Or dbt?

dbt is purely a DAG of SQL statements.

Airflow is more similar; I sense the difference is in use case emphasis. One of the creators of Metaflow says:

As for comparisons with Airflow, it is an excellent production grade scheduler. Metaflow intends to solve a different problem of providing an excellent development and deployment experience for ML pipelines.