Technology Radar
Published : Apr 26, 2023
NOT ON THE CURRENT EDITION
This blip is not on the current edition of the Radar. If it was on one of the last few editions, it is likely that it is still relevant. If the blip is older, it might no longer be relevant and our assessment might be different today. Unfortunately, we simply don't have the bandwidth to continuously review blips from previous editions of the Radar.
Understand more
Apr 2023
Assess
OpenLineage 是一个开放的数据管道沿袭元数据收集标准，旨在在作业运行时对其进行编整。它使用一致的命名约定定义了运行、作业和数据集实体的通用模型。沿袭模型的核心是可扩展的，可以通过定制切面来增加实体。OpenLineage 解决了生产者和消费者之间互通问题，否则大家不得不想各种办法实现互通。虽然存在它会成为另一个“中间标准”的风险，但作为 Linux 基金会 AI 和数据基金会项目，它获得被广泛采用的机会很大。OpenLineage 目前支持多个平台的数据采集，如 Spark 、Airflow 和 dbt，但用户需要自己配置监听器。OpenLineage 对数据消费者的支持目前较为有限。
