Enable javascript in your browser for better experience. Need to know to enable it? Go here.
radar blip
radar blip

Continuous delivery for machine learning (CD4ML)

Last updated : Apr 13, 2021
NOT ON THE CURRENT EDITION
This blip is not on the current edition of the Radar. If it was on one of the last few editions, it is likely that it is still relevant. If the blip is older, it might no longer be relevant and our assessment might be different today. Unfortunately, we simply don't have the bandwidth to continuously review blips from previous editions of the Radar. Understand more
Apr 2021
Adopt ? We feel strongly that the industry should be adopting these items. We use them when appropriate on our projects.

We see continuous delivery for machine learning (CD4ML) as a good default starting point for any ML solution that is being deployed into production. Many organizations are becoming more reliant on ML solutions for both customer offerings and internal operations so it makes sound business sense to apply the lessons and good practice captured by continuous delivery (CD) to ML solutions.

Oct 2020
Trial ? Worth pursuing. It is important to understand how to build up this capability. Enterprises should try this technology on a project that can handle the risk.

About a decade ago we introduced continuous delivery (CD), our default way to deliver software solutions. Today's solutions increasingly include machine-learning models and we find them no exception in adopting continuous delivery practices. We call this continuous delivery for machine learning (CD4ML). Although the principles of CD remain the same, the practices and tools to implement the end-to-end process of training, testing, deploying and monitoring models require some modifications. For example: version control must not only include code but also the data, the models and its parameters; the testing pyramid extends to include model bias, fairness and data and feature validation; the deployment process must consider how to promote and evaluate the performance of new models against current champion models. While the industry is celebrating the new buzzword of MLOps, we feel CD4ML is our holistic approach to implement an end-to-end process to reliably release and continuously improve machine-learning models, from idea to production.

May 2020
Trial ? Worth pursuing. It is important to understand how to build up this capability. Enterprises should try this technology on a project that can handle the risk.

Applying machine learning to make the business applications and services intelligent is more than just training models and serving them. It requires implementing end-to-end and continuously repeatable cycles of training, testing, deploying, monitoring and operating the models. Continuous delivery for machine learning (CD4ML) is a technique that enables reliable end-to-end cycles of development, deploying and monitoring machine learning models. The underpinning technology stack to enable CD4ML includes tooling for accessing and discovering data, version control of artefacts (such as data, model and code), continuous delivery pipelines, automated environment provisioning for various deployments and experiments, model performance assessment and tracking, and model operational observability. Companies can choose their own tool set depending on their existing tech stack. CD4ML emphasizes automation and removing manual handoffs. CD4ML is our de facto approach for developing ML models.

Nov 2019
Trial ? Worth pursuing. It is important to understand how to build up this capability. Enterprises should try this technology on a project that can handle the risk.

With an increased popularity of ML-based applications, and the technical complexity involved in building them, our teams rely heavily on continuous delivery for machine learning (CD4ML) to deliver such applications safely, quickly and in a sustainable manner. CD4ML is the discipline of bringing CD principles and practices to ML applications. It removes long cycle times between training models and deploying them to production. CD4ML removes manual handoffs between different teams, data engineers, data scientists and ML engineers in the end-to-end process of build and deployment of a model served by an application. Using CD4ML, our teams have successfully implemented the automated versioning, testing and deployment of all components of ML-based applications: data, model and code.

Apr 2019
Trial ? Worth pursuing. It is important to understand how to build up this capability. Enterprises should try this technology on a project that can handle the risk.

Continuous delivery for machine learning (CD4ML) apply continuous delivery practices to developing machine learning models so that they are always ready for production. This technique addresses two main problems of traditional machine learning model development: long cycle time between training models and deploying them to production, which often includes manually converting the model to production-ready code; and using production models that had been trained with stale data.

A continuous delivery pipeline of a machine learning model has two triggers: (1) changes to the structure of the model and (2) changes to the training and test data sets. For this to work we need to both version the data sets and the model's source code. The pipeline often includes steps such as testing the model against the test data set, applying automatic conversion of the model (if necessary) with tools such as H2O, and deploying the model to production to deliver value.

Published : Apr 24, 2019

Download Technology Radar Volume 29

English | Español | Português | 中文

Stay informed about technology

 

Subscribe now

Visit our archive to read previous volumes