ThoughtWorks
  • Contact
  • Español
  • Português
  • Deutsch
  • 中文
Go to overview
  • Engineering Culture, Delivery Mindset

    Embrace a modern approach to software development and deliver value faster

    Intelligence-Driven Decision Making

    Leverage your data assets to unlock new sources of value

  • Frictionless Operating Model

    Improve your organization's ability to respond to change

    Platform Strategy

    Create adaptable technology platforms that move with your business strategy

  • Experience Design and Product Capability

    Rapidly design, deliver and evolve exceptional products and experiences

    Partnerships

    Leveraging our network of trusted partners to amplify the outcomes we deliver for our clients

Go to overview
  • Automotive
  • Cleantech, Energy and Utilities
  • Financial Services and Insurance
  • Healthcare
  • Media and Publishing
  • Not-for-profit
  • Public Sector
  • Retail and E-commerce
  • Travel and Transport
Go to overview

Featured

  • Technology

    An in-depth exploration of enterprise technology and engineering excellence

  • Business

    Keep up to date with the latest business and industry insights for digital leaders

  • Culture

    The place for career-building content and tips, and our view on social justice and inclusivity

Digital Publications and Tools

  • Technology Radar

    An opinionated guide to technology frontiers

  • Perspectives

    A publication for digital leaders

  • Digital Fluency Model

    A model for prioritizing the digital capabilities needed to navigate uncertainty

  • Decoder

    The business execs' A-Z guide to technology

All Insights

  • Articles

    Expert insights to help your business grow

  • Blogs

    Personal perspectives from ThoughtWorkers around the globe

  • Books

    Explore our extensive library

  • Podcasts

    Captivating conversations on the latest in business and tech

Go to overview
  • Application process

    What to expect as you interview with us

  • Grads and career changers

    Start your tech career on the right foot

  • Search jobs

    Find open positions in your region

  • Stay connected

    Sign up for our monthly newsletter

Go to overview
  • Conferences and Events
  • Diversity and Inclusion
  • News
  • Open Source
  • Our Leaders
  • Social Change
  • Español
  • Português
  • Deutsch
  • 中文
ThoughtWorksMenu
  • Close   ✕
  • What we do
  • Who we work with
  • Insights
  • Careers
  • About
  • Contact
  • Back
  • Close   ✕
  • Go to overview
  • Engineering Culture, Delivery Mindset

    Embrace a modern approach to software development and deliver value faster

  • Experience Design and Product Capability

    Rapidly design, deliver and evolve exceptional products and experiences

  • Frictionless Operating Model

    Improve your organization's ability to respond to change

  • Intelligence-Driven Decision Making

    Leverage your data assets to unlock new sources of value

  • Partnerships

    Leveraging our network of trusted partners to amplify the outcomes we deliver for our clients

  • Platform Strategy

    Create adaptable technology platforms that move with your business strategy

  • Back
  • Close   ✕
  • Go to overview
  • Automotive
  • Cleantech, Energy and Utilities
  • Financial Services and Insurance
  • Healthcare
  • Media and Publishing
  • Not-for-profit
  • Public Sector
  • Retail and E-commerce
  • Travel and Transport
  • Back
  • Close   ✕
  • Go to overview
  • Featured

  • Technology

    An in-depth exploration of enterprise technology and engineering excellence

  • Business

    Keep up to date with the latest business and industry insights for digital leaders

  • Culture

    The place for career-building content and tips, and our view on social justice and inclusivity

  • Digital Publications and Tools

  • Technology Radar

    An opinionated guide to technology frontiers

  • Perspectives

    A publication for digital leaders

  • Digital Fluency Model

    A model for prioritizing the digital capabilities needed to navigate uncertainty

  • Decoder

    The business execs' A-Z guide to technology

  • All Insights

  • Articles

    Expert insights to help your business grow

  • Blogs

    Personal perspectives from ThoughtWorkers around the globe

  • Books

    Explore our extensive library

  • Podcasts

    Captivating conversations on the latest in business and tech

  • Back
  • Close   ✕
  • Go to overview
  • Application process

    What to expect as you interview with us

  • Grads and career changers

    Start your tech career on the right foot

  • Search jobs

    Find open positions in your region

  • Stay connected

    Sign up for our monthly newsletter

  • Back
  • Close   ✕
  • Go to overview
  • Conferences and Events
  • Diversity and Inclusion
  • News
  • Open Source
  • Our Leaders
  • Social Change
Blogs
Select a topic
View all topicsClose
Technology 
Agile Project Management Cloud Continuous Delivery  Data Science & Engineering Defending the Free Internet Evolutionary Architecture Experience Design IoT Languages, Tools & Frameworks Legacy Modernization Machine Learning & Artificial Intelligence Microservices Platforms Security Software Testing Technology Strategy 
Business 
Financial Services Global Health Innovation Retail  Transformation 
Careers 
Career Hacks Diversity & Inclusion Social Change 
Blogs

Topics

Choose a topic
  • Technology
    Technology
  • Technology Overview
  • Agile Project Management
  • Cloud
  • Continuous Delivery
  • Data Science & Engineering
  • Defending the Free Internet
  • Evolutionary Architecture
  • Experience Design
  • IoT
  • Languages, Tools & Frameworks
  • Legacy Modernization
  • Machine Learning & Artificial Intelligence
  • Microservices
  • Platforms
  • Security
  • Software Testing
  • Technology Strategy
  • Business
    Business
  • Business Overview
  • Financial Services
  • Global Health
  • Innovation
  • Retail
  • Transformation
  • Careers
    Careers
  • Careers Overview
  • Career Hacks
  • Diversity & Inclusion
  • Social Change
Technology

Why Configuration Management and Provisioning are Different

Carlos Nuñez Carlos Nuñez

Published: Feb 7, 2017

At many organizations I’ve worked at over the last few years, I’ve seen a common anti-pattern: configuration management (CM) tools used incorrectly as provisioning tools. This has been frustrating because using CM tools to provision infrastructure undoubtedly leads to complex code that is unmaintainable and hard to extend.

Let’s take a look at the downfalls of misusing CM tools for provisioning and how provisioning tools are more equipped for this purpose.

What is Configuration Management?

Configuration management tools are used to repeatably and consistently manage the configuration of systems and the services that they provide entirely through code. Many of them achieve this in three ways: an intuitive command line interface, a lightweight and easily-readable domain-specific language (DSL) and a comprehensive REST-based API that lowers the barrier-to-entry for integrations with other tools. Chef, Ansible, Puppet and SaltStack are popular, open-source examples of these tools.

I've seen many companies use these tools to create and modify, or provision, new infrastructure and configure them afterwards. In theory, this seems like a job that these tools are well-suited for given their advantages. However, my experience has shown that what actually happens is that much more code is written in order to take small edge cases into account. If I had to sum up what the resulting code often looks like, it would be something like this:



Let me explain.

Complexity in simplicity

CM code written in a particular tool’s DSL is meant to be simple. Copying and modifying files, installing packages or setting environment variables are examples of configurations that are straightforward to implement in any popular CM tool. However, many of the details that go into provisioning new machines (number of servers, metadata, disks associated with those servers, etc.) increase the complexity quickly.

One of the more attractive features of modern CM tools is their ability to express complex system and application configurations in an easily-readable way. This works well for expressing how applications should be installed on a system in a way that's easy for others to read.

Let’s use an example to see what we mean here.

You are responsible for a team that maintains a web-based application. The web servers in your environment run on nginx. Your team currently configures these servers by hand, and you would like to use a CM tool, like Ansible, to automate this process entirely with code and source control.

Creating an Ansible playbook to configure these nginx instances enables your engineers to do this. The process is straightforward: create a “playbook” that contains “plays” which does things like install nginx from yum or apt, copies configuration to the host and/or starts the nginx service. All of these plays are expressed with a human-readable templating language that makes it easy to see what is happening and the order in which these things happen. An example of this is shown here. 

This simplicity begins to fall apart when one tries to provision the instances on top of which nginx will be running. It usually starts off simple: create a playbook that fetches SSH keys from a backing store and uses them to deploy a static number of instances into a single region within AWS or your private cloud. All of the popular CM tools have modules or resources available for doing things in various cloud providers. But, what happens when you want to deploy onto multiple regions with different IDs and, potentially, different keys? What about when you need to perform modifications to the operating system after the machine is already created? Let’s say you want to use properties from this playbook to deploy a set of database servers and load balancers with them. How do you relate them?

What I've usually seen happen is that custom modules get created to fill in these gaps. These modules might do things like define an Amazon EC2 instance template or OpenStack Cinder volume. This isn't so bad on its own, but the devil in these details is in controlling dependencies between them. Telling Chef, for example, to create a set of volumes with one resource call and then bind them all to one instance isn't impossible, but attempting to express it without writing “real” Ruby is quite challenging.

This may not seem problematic to those who know Ruby (or whichever language in which the CM tool is based) well enough to put this together, but there are hidden costs to this: stricter testing requirements, loss in readability to engineers less familiar with the language, more complexity and more fragility.

Provisioning tools abstract many of these nuances; expressing relationships between disparate infrastructure with something like CloudFormation or Terraform is quite straightforward, and displaying a graphical representation of one’s current stack is similarly easy to do.

Maintaining state is important

There is also a major cost to consider in this approach: maintaining state.

Continuing on with our web application example, let’s say that traffic to your site has spiked up recently. To accommodate this demand, you would like to have three database servers in your environment instead of two. (We will assume that this environment does not currently have auto-scaling built in.) Because configuration management tools, like Chef or Puppet, are designed to modify existing infrastructure, not to keep a history of past configurations, in order to add an additional database server, you would need to write code to calculate the state of your current environment before taking any action against it. Otherwise, your cookbook will deploy three more database servers instead of just the additional one that you needed.

Provisioning tools like Terraform and AWS CloudFormation have logic built into them to prevent this problem from happening without needing to write this code yourself. They can show you what your infrastructure currently looks like and, in the case of Terraform, what it will look like after your desired changes take effect. This way, you can know exactly what is going to happen before it actually happens.

Rollbacks are hard too

Now, imagine that during your database server deployment, your monitoring system alerts you that the disks being allocated to your new servers will risk exceeding your team’s infrastructure budget. To prevent this from happening, you want to cancel the deployment and roll your environment back to its previous state before troubleshooting the problem and re-attempting.

As discussed above, Chef and other configuration management tools do not keep state of their actions. Consequently, without writing additional code to account for scenarios like this, performing this rollback with cookbooks will be impossible. You will need to wait for the cookbook application to fail and then go into your AWS console and delete your failed servers manually.

Provisioning tools like Terraform and CloudFormation provide this rollback for free. What’s more, if the rollback fails for some reason, you can modify the attribute describing the number of servers in your environment and rerun the deployment to purge any failed servers or other infrastructure, such as load balancers and DNS records, from it.

Use the right tools

There are a few tools available that can make provisioning your infrastructure simpler. Terraform, Heat, and CloudFormation are the most popular at the moment. Terraform is a better choice if the majority of your infrastructure is not within AWS or if reducing cloud platform lock-in is a priority. Several of our developers have used it with great success; one example of such can be found here. Both have established enterprise support options available as well as extremely active communities backing them.

There are also a few upstarts in this space to choose from. Cumulus and SparkleFormation share many features with their incumbents, though their communities are smaller and their support options less established.

Good luck!

Ready to shape the future of tech?

Join our team of passionate and bright technologists.

Join us
Related blogs
Cloud

Choosing the Right Tool to Provision AWS Infrastructure

Sneha Somwanshi
Learn more
Cloud

Infrastructure as Code: From the Iron Age to the Cloud Age

Kief Morris
Learn more
Continuous Delivery

Infrastructure as Code: A Reason to Smile

Jafari Sitakange
Learn more
  • What we do
  • Who we work with
  • Insights
  • Careers
  • About
  • Contact

WeChat

×
QR code to ThoughtWorks China WeChat subscription account

Media and analyst relations | Privacy policy | Modern Slavery statement ThoughtWorks| Accessibility | © 2021 ThoughtWorks, Inc.