Declarative Automation Bundles

Declarative Automation Bundles (formerly known as Databricks Asset Bundles) has evolved into a primary tool for bringing software engineering and CI/CD practices to the Databricks ecosystem. The tool has matured significantly and now enables our teams to manage most platform resources as code, including clusters, ETL pipelines, jobs, machine-learning models and dashboards. Using the databricks bundle plan command, teams can preview changes and apply repeatable deployment practices to Databricks artifacts, similar to how infrastructure is managed with tools such as Terraform. Treating traditionally mutable assets, such as dashboards and ML pipelines, as code allows them to be version-controlled, tested and deployed with the same rigor as traditional microservices. Based on our experience in production environments, Declarative Automation Bundles has become a reliable approach for managing data and ML workflows on Databricks. Teams working extensively in the Databricks ecosystem should consider adopting it to standardize infrastructure management practices.

Databricks Asset Bundles (DABs)，于 2024 年 4 月 实现基本可用，正逐渐成为打包和部署 Databricks 资产的首选工具，帮助我们的数据团队应用软件工程实践。DABs 支持将工作流和任务的配置以及要在这些任务中执行的代码打包成一个 bundle，并通过 CI/CD 管道部署到多个环境中。它提供了常见资产类型的模板，并支持自定义模板，这使得能够为数据工程和机器学习项目创建定制服务模版。我们的团队越来越多地将其作为工程工作流中的关键部分。尽管 DABs 包含了针对 notebooks 的模板，并支持将 notebooks 部署到生产环境中，我们并不推荐 生产化 notebooks ，而是鼓励有意地编写符合生产标准的代码，以支持此类工作负载的可维护性、可靠性和可扩展性需求。

Databricks 最近发布了 Databricks Asset Bundles (DABs) 的 公开预览版，它包含在 Databricks CLI 0.205 及更高版本中，正成为打包 Databricks 资源进行源代码控制、测试和部署的官方推荐方式。DABs 在我们的团队中逐渐取代了 dbx。DABs 支持将工作流、作业和任务的配置以及要在这些任务中执行的代码打包成一个可以部署到多个环境的捆绑包。它附带了常见资产类型的模板，并支持自定义模板。虽然 DABs 包含笔记本模板并支持将它们部署到生产环境，但我们仍然建议不要将 笔记本用于生产环境，而是鼓励有意地编写生产代码，并采用支持此类工作负载的可维护性、弹性和可扩展性的工程实践。