Published : Apr 26, 2023
Apr 2023
Assess
Worth exploring with the goal of understanding how it will affect your enterprise.
DataFusion is a part of the data community's exploration of Rust's performance, memory safety and concurrency features applied to data processing. It shares similarities with Polars, namely a familiar DataFrame API in Rust (with Python bindings), the use of Apache Arrow under the hood and SQL support. Even though it's primarily designed for single-process execution, distributed processing support is in the works within Ballista. We think the Rust libraries for data processing are an evolving space worth following and exploring, and DataFusion is a part of it.
