Enable javascript in your browser for better experience. Need to know to enable it? Go here.
Last updated : Apr 13, 2021
Not on the current edition
This blip is not on the current edition of the Radar. If it was on one of the last few editions it is likely that it is still relevant. If the blip is older it might no longer be relevant and our assessment might be different today. Unfortunately, we simply don't have the bandwidth to continuously review blips from previous editions of the Radar Understand more
Apr 2021
Trial ? Worth pursuing. It is important to understand how to build up this capability. Enterprises should try this technology on a project that can handle the risk.

We wrote about Great Expectations in the previous edition of the Radar. We continue to like it and have moved it to Trial in this edition. Great Expectations is a framework that enables you to craft built-in controls that flag anomalies or quality issues in data pipelines. Just as unit tests run in a build pipeline, Great Expectations makes assertions during execution of a data pipeline. We like its simplicity and ease of use — the rules stored in JSON can be modified by our data domain experts without necessarily needing data engineering skills.

Oct 2020
Assess ? Worth exploring with the goal of understanding how it will affect your enterprise.

With the rise of CD4ML, operational aspects of data engineering and data science have received more attention. Automated data governance is one aspect of this development. Great Expectations is a framework that enables you to craft built-in controls that flag anomalies or quality issues in data pipelines. Just as unit tests run in a build pipeline, Great Expectations makes assertions during execution of a data pipeline. This is useful not only for implementing a sort of Andon for data pipelines but also for ensuring that model-based algorithms remain within the operating range determined by their training data. Automated controls like these can help distribute and democratize data access and custodianship. Great Expectations also ships with a profiler tool to help understand the qualities of a particular data set and to set appropriate limits.

Veröffentlicht : Oct 28, 2020
Radar

Download Technology Radar Volume 25

English | Español | Português | 中文

Radar

Stay informed about technology

 

Subscribe now

Visit our archive to read previous volumes