Continuous delivery for modern architectures

Podcast Transcript

Zhamak Dehghani:

Hello, everyone and welcome to Thoughtworks podcast. My name is Zhamak Dehghani and I'm here today with my co-worker, Sheroy Marker. Welcome, Sheroy.

Sheroy Marker:

Hi Zhamak, thanks for having me. I'm Sheroy Marker. I'm the Head of Technology for Thoughtworks products. We build products that make software engineering teams more efficient.

Zhamak Dehghani:

Great. Great to have you. I'm super excited to have this conversation. It took us a while to actually connect. One of the products that your team builds is GoCD, which implements continuous delivery. I would like to pick your brain today about safe continuous delivery under the new microservices architecture under the continuing rise framework of deployments and operating services, and how that's shifted the developer experience. So maybe we start with how continuously delivery worked before this architecture, before modern distributed architecture.

Sheroy Marker:

So, there's been a shift in architectural thinking while building large scale applications of late. So in the recent past we were used to more larger monolithic systems where the system could be comprised of smaller components. So when you thought of continuous delivery in that context, you would have to think about how these components depended on each other and so if any of these components change, what that really meant for the system on the whole. So in that context, we used to see a lot of complex CD pipelines where a pipeline would consist of a build of a smaller component, and then for the downstream, once an upstream component changed, a further downstream component will get rebuilt, and then you would have a larger application binary or something. And then you have to figure out how the whole application gets deployed together.

Sheroy Marker:

There seems to be a shift in this thinking. So with the more recent microservices based architectures and systems, the focus seems to be to decompose the complexity of these larger systems into smaller independent components that have that own independent life cycles. So what that means is the smaller components now also have fewer dependencies. So the overall CD process or pipeline for these smaller microservices are starting to get simpler and hopefully more standardized. So if you think about our developer experience from that perspective, it's much simpler to now reason about the build process for a smaller microservice as compared to a small component that fits into a larger system. The typical stages in a CD pipeline or a CD process still apply. You still have to build your component, you still have to run verification and tests and stuff like that and you still have to then deploy your application into a test environment. But all of that now starts becoming at a much smaller scale and much, much easier to reason about.

Zhamak Dehghani:

Yeah, that's interesting. So if we simplify the pipeliners, you have your code repo where you have some, hopefully, some culture of continuous integration, that you're integrating your code changes in a smaller iterations more frequently. That triggers a pipeline that is constituted of build... Whatever build means for that pipeline, verification testing, and then deploy, and possibly some monitoring post-deployment for [inaudible 00:04:15] post-deployment. And in that very, very simple 10,000 feet view, you're saying that in the monolithic world, you had all these dependency management team builds, other pipelines that probably were building libraries and they were triggering the build of this bigger monolithic. And that has changed because we are building independent pieces of software, independent quantum of architecture]. I love this expression that Neal Ford used in the book Evolutionary Architecture, the quantum of architecture, which is, if I use it correctly — maybe it was architecture quantum — that's the piece of software that can be built and deployed and used independently. And so now this piece is much smaller. So we don't have this big dependency graphs and pipeline dependencies to other pipelines. Is that the right?

Sheroy Marker:

Yeah, I think so. So there's also a shift in thinking about reuse in general. So with earlier architectures and the earlier thought process around some of this, there was this concept of trying to share everything. Share services in the enterprise, share libraries that were better upstream and stuff like that. And I think there seems to be a shift in thinking towards a more share nothing model. So if you have a bunch of services, maybe hundreds of microservices or whatever, a few microservices, depending on your use case, each microservice could independently choose to use its own libraries, its own technology stacks. Really not share anything with other services in the ecosystem. So with that shift in thinking, you're not really trying to react to say a new version of a library build or a new version of a competent build, you're really just composing your service with the libraries and the versions and whatever it needs at an independent microservice level.

Sheroy Marker:

So if you really think about it, the phases in a CD pipeline really are a CI, which is really the first stage in your pipeline where you react to a code change and rebuild a binary version of your component. And then, the second stage of your pipeline would be you deploy this component somewhere and run a bunch of tests. Obviously, the CI phase takes care of running unit tests and competent level tests. And then the second phase of your CD pipeline, we're test more at an integration or an application level. And then the final stages would deal with yarn deploying to staging environments and production environments and stuff like that. So those stages in the pipeline still exist but they start becoming more standardized. The way you build a component is pretty much standard across the board, across all of your services.

Sheroy Marker:

Hopefully, the way you run the kinds of tests you run on these components are also kind of similar. And the way you do deployments starts becoming very standardized as well. So, if you look at the current state of infrastructure with platforms like Kubernetes, deployments to Kubernetes, especially for various environments, has started becoming much simpler now because you can start using concepts like namespaces to deploy your application in multiple environments on the same Kubernetes cluster. So if you are doing just that your deployments to a Kubernetes cluster could look pretty a standard across the board for various microservices. So that's what I mean by more simpler standardized CD workflows.

Zhamak Dehghani:

Yeah, absolutely. You mentioned the reusability that we were all obsessed with, dry principles and try to reuse libraries and we learned that the more we get reuse out of a component, the less usable it becomes. I remember so many projects that they were just using plain java objects, like POJOs and automatically created source code that's really had no value but we were sharing these codes across client and server side or consumer-provider side of two different services and that created coupling. So that's the moving away from just that level of reusability had helped to build. I guess drilling a little bit into the test phase. So as you said, the paradigms still exist. You've got to validate your code before going to production. But how do you see the testing practices or how the pipelines actually formulate tests and execute tests have shifted in this kind of new or standardized way of building service pipelines?

Sheroy Marker:

So one shift that we are starting to see is testing more at a service level and testing the functionality of a single service. The other thing to note here, is that with microservices the focus seems to be on a single purpose-built service. So these services model a specific business functionality as compared to modeling a subset or functionality in a business scenario as web services would model. So when you have purpose-built services doing one specific function, it's much simpler to also model test suites that actually test that functionality in isolation. The focus seems to be shifting away from composing larger systems and then testing larger system functionality altogether. Because that is something that you could test in production or production-like environments with synthetic user transactions or even just real user testing and seeing how the system's behaving in production. So the verification is more at a smaller subset of the business functionality perspective. But the testing tools or the practices that you use are pretty much the same as you what we used to use earlier.

Zhamak Dehghani:

Yeah, I remember, you're right. We were doing this multi-service, integrated testing that they were one of the painful parts of continuous delivery. Having an integrated environment where everything's deployed to, everything talks to everything else. And we run a large, number of tests that runs through a lot of business scenarios across all of the systems and it was always almost impossible to continue to get through that phase because you have the shared environment, it's always has some sort of an inconsistency in terms of our expectations, of expectations of your tests. And we were moving away from that model and just using consumer contract testing at the service levels. So you don't have to integrate all these services to come together and wanting to create an environment, as you said, run any number of tests that you need to test as an integrated test, as synthetic transactions in the actual production environment. That's a really interesting shift. And I feel like a lot of organizations that are moving to this architectural side, they don't think about that testing as much and still tend to have integration testing and that just makes life much more difficult.

Sheroy Marker:

Yeah, for sure. So if you think about it, when you are releasing a new version of something, you want to make sure that that version works well with whatever else is running in production, right? And there are variety of ways to do that. So first of all, if you are deploying smaller components of the system, as microservices, it's easier to verify the complete production, a state of a microservice with a with a much smaller test suite and an isolation. And eventually when you're rolling out to production, you could even use deployment strategies to verify that the new version of a microservice plays well with the existing production version of other microservices. So that's what blue-green deployments and canary deployments also serve. So if when you're releasing a new version of a service, if you release it with our canary deployment strategy and run some production traffic through the servers, you pretty much get to see how it plays with other current production versions of services pretty quickly and then make a decision to roll forward or hold back on deployment. So the few options when it comes to verifying in production. So, you can do it at deployment time, you could do it with synthetic user transactions at runtime, or if it applies to you in your scenario, you could start considering a stuff like chaos engineering where you also see how resilient your systems are. But the focus is more around verifying production environments and making sure everything's working well in productive.

Zhamak Dehghani:

Yeah, absolutely. And I feel like there is a convergence of testing and monitoring, right? Because when you are deploying to production you're actually using monitoring as a way of continuously validating that your systems are working and where you don't don't see that as this isolated phase of testing in a different environment, it's just how we monitor our application and their liveliness in the production environment.

Zhamak Dehghani:

We've talked about kind of build and tests and now, with deployment, the shift from building a [inaudible 00:14:45] application and putting it on some sort of a deep tech sack of layers of software from your bare metal OS to your application server and all the other dependencies that it has libraries that are deployed to that environment. We've shifted from that to self containers, in a way, and leaving the destiny of that container to the container management system with the old test version system. How do you see that shifts changing the way we now build artifacts and deploy?

Sheroy Marker:

So Docker has now provided a more standard packaging format, which is also very portable. So a Docker container, once built, contains your application and all of its dependencies. And then that's a pretty standard package across any environments that you test on. So it also gives you more reliability in resilience across environments. So it's a pretty standard format these days for built systems. Kubernetes also provides you a good layer of abstraction over physical hardware or even cloud infrastructure. Where you had VMs earlier, you still needed to orchestrate deployments across all of these VMs with some sort of config management tooling like Chef or Puppet. And then make sure that the status of the roll outs were consistent and then keep track of what versions we're running in which environments and stuff like that. So with Kubernetes it gives you an abstraction over all of that stuff.

Sheroy Marker:

So all you do is, you define your deployments in a declarative format and then let Kubernetes handle the roll out of that application, keep your cluster consistent across the board. So containers have played a pretty vital role in introducing a shift in the way we think about infrastructure and application deployments and managing versions of applications across this infrastructure. So from a developer perspective, especially, it's simplified how you think about CD and delivering your applications into production environments. So developers are much more in control of the CD process now and there's a new renewed expectation on also being able to interact with your CD tooling more programmatically and declaratively now. So the same constructs that Kubernetes provides you, you are also starting to expect the same stuff out of any CD tooling use you interact with. It's an interesting shift. It's a definitely a shift words modified dev-centric, DevOps centric sort of approach.

Zhamak Dehghani:

Yeah. Can you talk about some of the technologies and tooling that has changed? For instance, we would have used some sort of an artifact repository to keep up what version WAR file and now we use different artifact repository to keep the document. If you think about that flow that we've been talking about, maybe can we talk about some of the tools that have also changed?

Sheroy Marker:

Yeah, it's what just revisiting some of the concepts we touched upon. So traditionally, you would build an artifact which would have been either a JAR file or a WAR file, some sort of binary, you'd store it in some sort of artifact repository like Artifactory. And then, crafted a deployment that would get this application binary but then also other dependent libraries and then do a deployment to some sort of a deployment environment. Conceptually that process still exists, but what has changed now is instead of building a JAR or a WAR file, you would just build a Docker image. These Docker images are then stored in Docker registries. There are a bunch of options available all the way from Docker registry is provided by cloud providers to stuff you can provision yourself.

Sheroy Marker:

But the interface to these registries is still pretty consistent. So when you think about it from a CD tooling perspective or a CD process perspective, the process now is the first step in the process is build an image. The second process is verify your image. So you could either create an instance of your service with Docker and your CD tooling should allow you to do all of that. And then eventually once you've verified your image you would store it in some sort of a Docker registry. And then the last step in this process would be then defining a deployment descriptor for Kubernetes or whatever your target deployment environment is and then hand off the task of deployment to your deployment environment like Kubernetes or OpenShift or whatever.

Zhamak Dehghani:

That makes sense. And you are playing the role of head of tech for GoCD and other products that thought were produced. Is that right?

Sheroy Marker:

Yeah. Yeah. So we work on a couple of products. So we build a CD product called GoCD. GoCD is an open-source, continuous delivery product. Primarily GoCD gives you the ability to model CD pipelines. It lets you reason about the provenance of artifacts that you generate in your process. It gives you the ability to store and fetch these artifacts from Artifactory repositories like Docker registries, the one type of artifact repository. And finally, it lets you monitor deployments to your target runtime environment. We do have a plugin for Kubernetes specifically, which lets you do all of this with Docker. Docker images, integrations with Docker registries, and deployments to Kubernetes. Once you want to define the CD workflow, the other aspect of this is being able to visualize this end-to-end part of production. So once you have a CD process working for you, it's also important to see how efficient this process is for you. It's important for you to measure a few metrics specifically around throughput and reliability and stuff like that. So how many deployments are you putting out every day or what is your frequency of deployment now, how many deployments fail, for example, and stuff like that. So GoCD lets you visualize this end-to-end part to production and lets you monitor some of these metrics as well.

Zhamak Dehghani:

So developers need to make decisions around their continuous delivery that they're choosing for the project or for the product and I've been on many different projects that we looked at Jenkins, we looked at CircleCI, we look at a bunch of different tools and I'm wondering if you can give a summary of comparison between GoCD and these other continuous delivery continuous integration tools that are out there to help that developer decision making.

Sheroy Marker:

There are variety of tools available in the market and the parameters with which to look at some of these tools are the nature of tool itself. So is it an open source product that you can use just right out of the bat? Is it a commercial product? Some of them are hosted SAAS products, some are installed products, and then there is also just the problem that they solve. Some of these products are just CI tools, some of them purely model deployments. So it really depends on what you want out of your CD tooling. So for example, Jenkins was initially built as a CI product. It's ever since been enhanced with some pipeline capability and stuff like that, but it wasn't built for CD pipelines. You can get pipeline capability with plugins. So it's worth evaluating if stuff like that works for you.

Sheroy Marker:

Spinnaker is an example of a product that was built primarily for deployments and deployment pipelines, not for solving the CI problems. So it interfaces with other products for a CI capability. When we built GoCD, our philosophy was that it was important to model the end-to-end continuous delivery workflow in one single pane of glass. So the GoCD handles everything from CI all the way to deployments as one tool. And the reason you want to do that is, it's important for you to see this end-to-end part of production on something that we call a value stream, which is a concept that's borrowed from lean. So the value stream will show you every part of the process to production and how long each step is taking. So when you see this visualization with the entire process, it's easy for you to spot inefficiencies and then figured out where you need to make improvements. It's also a much simpler to model deployments to multiple environments and stuff like that when you have your NDRs end-to-end CD pipeline in a single pane of glass. So that's where GoCD sits in the space. It's completely open source. It's got a powerful plugin model. So it lets you enhance GoCD capabilities with plugins and we have a pretty rich ecosystem of plugins as well.

Zhamak Dehghani:

Fantastic. If I wanted to add plugins, what language do I need to write that in?

Sheroy Marker:

So you need to write plugins in... So GoCD is primarily a Java-based system, so you'd have to write plugins in Java.

Zhamak Dehghani:

What are some of the popular plugins that community has contributed to GoCD?

Sheroy Marker:

So there are, just before I answer that, there two types of two categories of plugins on our plugin site. So there's one category is community contributed plugins and then that are enterprise plugins that are built by the GoCD team. So for the community contributed plugins, you have a good example would be stuff like the configuration repository plugins that let you define GoCD pipelines and either YAML or JSON format that was a community contribution. A good example of enterprise plugins would be maybe the LDAP plugin that lets you hookup GoCD to an LDAP repository in an enterprise. So that was a very specific enterprise requirement that we had to spend our time building and so it's an enterprise plugin from our side that you have to pay for if you are an enterprise at the scale that needs some of these capabilities.

Zhamak Dehghani:

Great. I better sharpen my Java skills then if I want to contribute to GoCD. What is on the roadmap, for you and for GoCD?

Sheroy Marker:

So our current is on making improvements to the product across the board. So this includes everything from integration with cloud providers and making it easier to you use GoCD elastic agents, which is our facility to have a go CD scale more elastically on cloud environments. We are also actively working on improving the developer experience of GoCD. So this could take the form of better SDKs and improving the config repository experience. Lastly, we are also continuing to improve the user experience of GoCD. We're trying to make it easier for users to understand GoCD concepts and just overall improve the usability across the board.

Zhamak Dehghani:

Great. So we talked about the shift of continuous delivery in terms of some of the underlying tooling and approaches as the architecture has shifted to more distributed to compose microservices architecture. We talked about different toolings that are out there. I wonder if we could leave with, a few universal principles of CD that haven't changed and probably won't change with the change of architecture or with the change of the underlying deployment mechanisms that are out there. And if you could call out a few of your favorite ones, favorite continuous delivery principles, what would they be?

Sheroy Marker:

So some of the companies that very principles like generating application artifacts or code artifacts, binary artifacts that then need to be validated. So then there's a step of validating artifacts itself, promoting good versions of artifacts and having a provenance of what changes are in production in which artifacts actually go to production. So that end-to-end traceability is pretty important. Being able to model a CD workflow with the right levels of parallelism and serial steps and stuff like that. So having that composability of a CD workflow is pretty important. And then, just having great visualizations, that's pretty important as well. Finally, having a good way for developers to interact with your tooling we feel is pretty important and that's something that we're working on pretty actively right now. One of the philosophies that we follow and is something that we appreciate in tools is a product or a tool should really stick to what it does best. So in this case, if you know we are building a CD server, we try to stick to providing CD capability in the tooling and not let any of the features or functionalities of maybe a runtime environment like Kubernetes bleed into the capabilities of the CD tooling so we stick to building CD capability and then stay out of the way of people and let them model the work source that they need.

Zhamak Dehghani:

Absolutely. I really liked the idea of traceability that you mentioned. Be able to trace back the artifact that is in production to the change that [inaudible 00:30:15] produced [inaudible 00:30:17]. Also, you mentioned you know the same artifact that is built, that same artifact would get to validational tests set and the same artifact would get deployed. You mentioned something earlier that I think is important to include in the best practices, somewhat or principles, which was the metrics of measurement. You mentioned that GoCD gives you that ability out of the box to see different metrics and be able to kind of radiate those information outs for your developers to know the efficiency of the pipeline, the change, fail rate of the pipeline, and I think it's important to continuously have those information dashboards and measure the performance that's with our pipelines and provide that visibility to the developers.

Sheroy Marker:

Yeah, for sure. And with GoCD, we give you those metrics by means of a new plugin we've just released. So our metrics plugin will give you a metrics like mean time to recovery and failure rate and deployment rate, and just the overall time it takes to get a change. So cycle time. How long does it take to get a change through the pipeline and into production. So yeah, it's pretty cool. You should check it out.

Zhamak Dehghani:

Perfect. Thank you so much, Sheroy, for joining me.

Sheroy Marker:

Thanks for having me. Thanks Zhamak.

Zhamak Dehghani:

Okay. Bye-bye.