Technology Radar
Published : May 19, 2020
NOT ON THE CURRENT EDITION
This blip is not on the current edition of the Radar. If it was on one of the last few editions, it is likely that it is still relevant. If the blip is older, it might no longer be relevant and our assessment might be different today. Unfortunately, we simply don't have the bandwidth to continuously review blips from previous editions of the Radar.
Understand more
May 2020
Assess
在数据工程中使用良好的软件工程实践，也仍然存在一些工具空白。我们的一个团队尝试在数据管道中的不同步骤之间自动执行数据质量检查时，惊讶地发现，在该领域中只有很少的工具。他们选择了 Deequ ，这是一个用来为数据集编写类似单元测试的库。 Deequ 建立在 Apache Spark 之上，虽然它是由AWS 实验室发布的，但也可以被用在 AWS 以外的环境。
