Microservices: The State of the Art

Podcast Transcript

Mike Mason:

Hello and welcome to Thoughtworks Podcast. I'm Mike Mason.

Zhamak Dehghani:

I'm Zhamak Dehghani.

James Lewis:

I'm James Lewis.

Lakshminarasimhan Sudarshan:

I'm Lakshminarasimhan Sudarshan.

Mike Mason:

Today we're going to be talking about microservices in Sydney, Australia, putting together the new edition of the Thoughtworks radar. Whilst kind of doing the usual discussion and arguing about what should go on the radar and what we should say about it, a number of things came up to do with microservices, and we thought we would run a podcast on this, and especially since we have James in the room. James actually with Martin Fowler authored one of the kind of original early articles defining what microservices really are. So James, I thought maybe we could start with you giving us a brief history of how microservices came about?

James Lewis:

Sure. So in order to be brief, we need to get back maybe not quite as far as the beginning. So if we start in 1969 with the birth of Unix. Maybe not. So I guess it was really around 2012, where I started seeing different organizations adopting I guess what we now sort of think of the microservice style. Fred George originally coined the term back around then to describe a particular way of plucking together very, very small bits of functionality, generally attached to some form of message bus.

James Lewis:

I guess at the same time you have organizations like Netflix, in the US, so they were talking about fine-grain service oriented architectures. That was Adrian Cockcroft's term for what they were doing at the time. Then I think it was about 2012 where I attended a workshop called The Software Architecture Workshop in Europe, where over the course of three days, we were sort of talking about things like why is my product so unmaintainable? Why is it so hard to make changes to my thing? How can I apply the strangler pattern to break my thing up because I can't make any changes to it anymore? And so on.

James Lewis:

I guess at the end of that, as a group, we had this sort of revelation around, well maybe we're solving the wrong problem. Maybe it's not about how to maintain all the big things. Maybe the problem is why are we building these things so large in the first place? So we sort of started thinking along the lines of maybe if we broke our systems up into smaller bits, we wouldn't necessarily have these issues to start with. Which is kind of an interesting insight, I think at the time. And I joked about 1969 and Unix, but actually a lot of the thinking behind microservices goes back quite a long way. And seeing as we're here with the TAB putting together the Radar at the moment, if you actually look at some of the themes on the radar predating microservices, so sort of 2009/10/11/12, what we see on the Radar are things like evolutionary architectures.

James Lewis:

We have things like continuous delivery. I think there was one around smaller, simpler application architectures, so things that were sort of very clearly trending in one direction towards this thing that we, I think now talked about, we now call microservices. And then a bit later, Martin and I sort of got together and wrote up what we were seeing and published it, and it's become I guess a pretty popular article by a lot of people. It's pretty well cited

Mike Mason:

I mean it's the top Google hit for, or at least it was for a long time, top Google hit for when you search for microservices. Since everybody has been talking about microservices for a long time, I can see that getting a lot of traction. But I mean it isn't new. I mean you wrote the article a few years ago. Why are we still talking about microservices? Is it still important in the industry?

Zhamak Dehghani:

Yeah, I think so. I think what has happened is the idea of doing microservices happened in 2012, and people start doing it. But the actual practices, how to do it, how to break down your monolith of legacy, and then how to operationalize it, how to run it and monitor it. We had to catch up, and we're still catching up and building ecosystem around it. So I guess that's one aspect of why it's still hot and we're talking about it because we're just evolving an ecosystem to support it.

James Lewis:

Yeah. I think it's the classic hype cycle actually. There's a bunch of people who are doing this already. So the Netflixes and so on. Guardian newspaper Comparethemarket before quote, microservices was a thing. So you've got those sorts of organizations who are narrow, they've pushed on, right? I mean you mentioned things like observability. We're seeing the sort of second and third order effects from this. So we're seeing actually entire ecosystems around things like monitoring tools. We're seeing entire ecosystems around things like secret management to help you integrate services securely. But then at the same time, we're seeing another massive group of organizations who aren't there yet. So they're on the way on this journey. They're just starting to think that maybe this style of architecture might be appropriate for their circumstances as well. So you've got this two speed thing happening with microservices.

Mike Mason:

Because I mean I once heard someone describe it as with a microservice architecture, you're trading kind of individual component complexity for overall system complexity. So each component in a microservice system is simpler because it's kind of smaller and doing a more focused task. But then the fact that you have to run all of these things and wire them all up in production, and I mean you were just talking about monitoring and security and all of that kind of stuff. That operational complexity goes up quite a lot. I think that's one of the things that people get stuck on. It's a surprise. We've got all these nice simple microservices, but as a whole...

James Lewis:

I'm reminded of the origin story of Spiderman that I read to my son every now and again. With great power comes great responsibility, right? I think that's true with microservices. It's quite sharp as a style and you can really hurt yourself. I think my take on why it's interesting is I think it's because it increases the degrees of freedom that you have in the system as a whole. So with a monolith, especially if you've, as we used to build monoliths when they were stateful, it was very difficult to scale them in a kind of, unless you cookie cutter them I guess, but it was very difficult to scale them unless you bought a bigger box.

James Lewis:

You have to scale up vertically. But with microservices, you tend to be able to scale horizontally as well as vertically. But also you can scale on the Z axis of the scale cube. So you can scale by business domain as well. So you've got three degrees of freedom in your system for scaling, for various reasons, whether that's security, whether that's throughput, whether that's availability, et cetera, et cetera. So I think that's why they're interesting. But you're right, it's sharp.

Zhamak Dehghani:

And if you look at the space, what we see is on the point of scale, a series of kind of post startup organizations that they started with a monolith to just see if their business idea actually works, got a large number of user base and now they need to scale out their organizations to a larger number of teams to work in parallel, to respond to the market. So that's one end of the spectrum in terms of people who are going towards microservices. And the other end of spectrum are enterprises who have legacy and large systems. They do have the scale, they have large number of teams and a large number of organizational units, but they're going really, really slowly.

Zhamak Dehghani:

So you either want to speed up a larger organization or you want to scale out a smaller kind of organization that's going through the growth. And in both cases, we see kind of application of microservices.

Lakshminarasimhan Sudarshan:

But I think this is where things get interesting because if you're a large organization and you've got all these big monolith sort of applications, and you're reading the microservices article and saying, great, I want to apply it, now what is happening or what is likely to happen is that those large organizations take in monoliths, try to convert them to microservices style and end up saying, okay, this sounds like a lot harder. It feels like you're actually getting that system complexity. Therefore they perceive microservices as being yet another hype cycle, but there's a fine print under the microservices article that says you need to have a certain level of maturity in terms of your operability and your infrastructure space before you can actually do microservices. This is one of the I guess the classic anti-patterns of microservices, doing it before you have the ecosystem maturity so to speak. If you are finding it difficult to deploy and monitor a monolith, you are not really ready for a microservices set up because you're just going to multiply that complexity by N if you have N different microservices.

Mike Mason:

But there's now a whole bunch of vendors who will sell you a microservices ready platform.

James Lewis:

Yeah. And it'll come with the DevOps.

Mike Mason:

Yeah. You can download and install the DevOps. But seriously, there's some good intent there, right? It's kind of wasteful to have every team in the organization understand exactly precisely how to deploy and run and scale their software. Wouldn't it be nicer to just have teams kind of understanding how to fit into something that is going to do those kind of platform like things for you and do that scaling and manageability component? And so I think the vision is to have a department of the company who is in charge of running the platform and understanding all of that stuff, and then everybody else can be kind of insulated from that. Is that a reasonable approach?

James Lewis:

There's be dragons, Mike. There's be dragons. I mean it's certain you understand the platform team conversation. I mean I really like the client definition from this. I can't remember all the properties, but one of the properties is the idea of on demand self service without requiring human intervention. And if you don't have that characteristic I think as part of a platform, I think you can get into real trouble. In fact, you can replace a thing you've already got which isn't working, usually because infrastructure and deployments are so difficult in large organizations is usually because they require massive amounts of human intervention. You have to wait on tickets being created and then serviced and large queues and the work and so on. If you just replace that with exactly the same thing and call it a platform, you're not going to gain anything.

James Lewis:

If you replace with on demand self service infrastructure, then yeah, you can get a huge amount of benefit I think. But I think, I mean just going back I guess to one of the points, I mean you go about organizations and I think you do see a lot of anti-patterns in real large organizations, but it can also act as an interesting forcing function I think because it can drive out of some of these constraints that you have. So both around how your teams are put together, the organizational boundaries that you have. Because when you look at the organizations that are really successful using the microservice style, they're set up in a very particular way. They're set up with cross functional teams who own their infrastructure or maybe are deploying onto a platform, but crucially are able to get stuff done on their own autonomously without having to hand software over to other departments. So...

Zhamak Dehghani:

I think we are hearing, at least I'm hearing, platform and product almost in the same sentence. As in platform as a product or bringing in product thinking to building the platform, which the main attribute of that, as James said, is being self-served. And you can discover the capability you're looking for to deploy your service or to monitor your service in the self-serve way, use those capabilities.

Mike Mason:

But also when you think about a product, you think about deliberately trying to woo and acquire customers for it. So I think one failure mode that we see is centralized anything frankly being mandated by an organization rather than building a platform that developers want to use and actually kind of attracting them to that even if it's an internal platform. That's kind of a much better model because then you're designing platform features that treat developers as happy customers rather than building some kind of tricky to use piece of infrastructure and then forcing people to use it.

James Lewis:

And on the last edition of the radar, we had Google cloud platform blipped for exactly that reason and others. But one of the reasons we had it on the radar was because it offers a compelling developer experience, and I think that's actually crucial as well. Yeah, I totally agree that one of the crucial characteristics is if you're building a product that's a platform, you have to make it as compelling as any other product you're building for customers. But in this case, your customers were internal rather than external.

Lakshminarasimhan Sudarshan:

Yeah, and I think with regards to keeping this platform team on the conversation with the topic of why are microservices still relevant? Why have they survived the hype cycle? It's because I think there have been significant ecosystem innovations in how to do a good platform. Go back five years, and we really didn't have great advice as to how to do a good platform. Or rather we had advice, but we didn't have great tooling support or great solutions or open source tooling which you could pick up and use. And I think that change over the last few years is really what has allowed microservices to survive the backlash of the hype cycle.

James Lewis:

I do agree with that. I think it's that five years ago, all the tooling was disparate. So what we found and what I saw when we were building microservice style architectures a few years ago is everyone had to do the same thing again and again and again. And I used to talk about this as having a long runway. If you're going to start a microservices project or program in your organization or build an architecture based on microservices, there was quite a long run up before you actually started to get value. So we're going to have to create our platform. We're going to have to create our monitoring infrastructure. We're going to have to create our alerting infrastructure. We have to put the observability characteristics around it, build all our CD infrastructure up.

James Lewis:

Whereas actually I think now we're getting to the point with platforms, compelling platforms, that a lot of this stuff can actually be pushed down near the undifferentiated heavy lifting that's sort of one level above the heavy lifting of installing firewalls and configuring firewall rules. The sort of heavy lifting around monetary alerting. I know Zhamak, you've been doing quite a lot around observability of systems and things and...

Zhamak Dehghani:

Yeah, I think for me the most exciting part of 2017 was service mesh without revealing how boring my life was in 2016. But at least in the software architecture world, I felt like there was something new and exciting again and that was I think the idea of externalizing a lot of cross cutting concerns such as observability, such as security out of process into the infrastructure. But in a way, that still kind of coupled with the service that you are deploying. So having the idea of side cars deployed as a unit with your process and externalizing kind of discovery of the services, termination of TLS endpoints and a whole lot of other things that you had to build in process and in a polyglot environment, in multiple languages. And we're probably going to put links to our Tech Radar for these and service mesh. I think it appeared in the last one in November.

James Lewis:

Yeah, it's interesting. I think it's another example of something that's quite sharp as a tool. You look at where service meshes are coming from and the people adopting them, and they tend to be way further on their journey than most organizations in this space. So they're sort of seeing issues that are third and fourth order removed from things that people would see when you start. So I can give you an example where I saw a sort of sidecar like implementation, but the developers who were working on the project didn't really sort of realize that they had this sidecar and didn't really realize that they could configure it in particular ways and so on, and just sort of assumed middle load of assumptions about their infrastructure and didn't really get any of the benefit of it. But again it's, what do we call that? Stupidity [crosstalk 00:17:10].

Mike Mason:

It's interesting that the discussion has all been around kind of infrastructure and operability so far. Because one of the things I get the impression that people struggle with is not just all those things, those are the things that cause you to fall over completely, but design is a tricky part as well. Is it domain driven design? Are we finding bounded contexts? How do our microservices interoperate with each other? How do I build a report across 25 different microservices? Am I doing that from some downstream system? So I mean from my perspective there's still a lot to be learned about the logical design of these things.

Lakshminarasimhan Sudarshan:

I think that's accurate. A good point, Mike. In my previous project, I remember it was one of these traditional organizations, and we were building a microservices project. And this was going to be the forcing function as James put it, to try and change some of the organizational approach to building software. We, halfway through the project, we realized that the way we were building our services, one of our assumptions we had made, were actually incorrect, and we had to now fix some of it. But given that we were still working in a mode where we are trying to fix discerning organization, fix a lot of the infrastructure beds, the cost we actually paid was significant, which meant that within the context of the project itself, there was a strong feeling, you know what? Maybe we shouldn't have conflated to things which is damaging the project and helping build the ecosystem using the project.

Lakshminarasimhan Sudarshan:

So yeah, going back to your point around what if you get it wrong, yes, I think that why we have some of the rules around bonded context, domain driven design, all of those are still absolutely relevant. But I think the cost of what if you get it wrong, is a very important question here. One or two years ago, I remember a lot of the advice around this was don't start off with microservices, evolve to microservices. And that was get your componentization, get your domain modeling right. Basically build a well-factored monolith and break out as you start to feel you're gaining the confidence. But I think there were a couple of very good points which came up as well. There is all discerning cost of working with microservices.

Lakshminarasimhan Sudarshan:

If you don't plan and build for it, how likely is it that your product owner is going to say, okay, let's invest a few hundred thousand dollars in bringing out the seconds and breaking up your nicely working functionality to two different services. So I think there are both sides of the argument. On one hand, I think yes, we shouldn't jump all in unless you understand the domain really, really well. But on the other hand, I think it's perhaps a little bit too optimistic to say, okay, let's start full on monolith and of course we'll gradually evolve perfectly into a nice microservice architecture.

James Lewis:

Yes. That's an interesting point around that. So I sort of used, actually, I sort of quote borrowed this, I'm going to attribute this to Chad Fowler. He talks about building heritage software, right? It's about building software that we're proud to leave behind for the next set of developers that come into the organization. I think actually microservices can kind of help with that in fact. But you have to make a certain set of assumptions. That is you're building a product, so you're using this style to build long lived products. And the team is able, or teams who are building those products, are able to make decisions internally about how best to build those products and are able to, once they've learnt from experience and experiments, change their mind about what the internal architecture of their systems look like, and have the ability to then implement those changes.

James Lewis:

So I get the iteration blinkers, sometimes you're just told you can't do this. But actually one of the foundational characteristics was around products, not projects. So it's more about long lived products. So I love this idea of building heritage systems or building systems that are the sort of heritage rather than legacy. I like this story, it's an old sort of philosophical, it's an old bit of philosophy from the ancient Greeks called the ship of Theseus, if you've come across this story. So the ship of Theseus was, Theseus was a king who had a ship built, and to paraphrase... Had a ship built, and he went off on a massively long multi-year voyage. Over the course of the voyage, bad things happen. So the mast fell off in a storm and the hull was breached. And so they had to replace the mast and they had to replace parts of the hull.

James Lewis:

By the time they got back to the origin point where they started, they'd replaced every single thing on the ship was new in some way. The paradox is the ship of Theseus, is it still the same ship? Is it still the same thing? I think actually that's the thing with microservices. Terry Pratchett talks about my father's ax. Yes, of course, I've had this ax for 800 years and of course I've changed the head and of course I changed the handle. But is this still not the same ax?

James Lewis:

I think that's how we should look at microservices. From the outsides, things should evolve as products evolve. So functionality should evolve, behavior should evolve, but internally, we should be able to replace all the individual parts without necessarily having any external influence. And we should be free to do that. If we're not really free to replace all those small internal moving parts, then you're going to run into these difficulties I think around maintainability. We shouldn't have to get designed right upfront. That's like... Yes, we need an idea of what a domain looks like, ideas about what our main contacts look like, what our business capabilities look like, but we shouldn't be afraid to change them as well.

Lakshminarasimhan Sudarshan:

I think the key underlying principle there though is the cost of reversing or addition or making a change, which is that if it genuinely is low cost to change your design, if you got your microservices wrong, to move APIs around, to conflate functionality, to split them apart, if that's easy, you're spot on. I mean that's exactly what you should do. I think the problem comes if the surrounding organization context is such that the cost of change is high for whatever reason. It could be because the infrastructure is complex, you don't understand the system around the team shape is not right, whatever it is. So if that cost of change is high, then you're probably better off I think deferring the microservices decision and just going for the well factor in monolith. So I guess essentially what I'm saying is depending on your ability to make poor decisions and back out of it or how costly that is, so depending on it, start off either aggressive or pessimistic I guess in this case.

Zhamak Dehghani:

I think that a lot of organizations start with a monolith and the question of like, how do I break it down in a way that the impact of radius of change is small so when I change one small piece it's not going to make all 50 other services fall apart? It's tricky. So how do you evolve from the monolith to microservices? And yes, we can go on a whiteboard or with stickies do a great domain driven design, but the reality of the engineering principles you need to put in place to break monolith down, it's hard. And we don't talk enough about it I think. I think there are great examples of organizations doing this, but there aren't many patterns that have emerged. We have good patterns around how to not build a tightly coupled distributed system. Don't put your configuration in one service, don't put your reference data in one service, don't reuse libraries so aggressively across services. But the patterns of migration, especially migration of data, because the migration of services is easy. The migration of data is a whole other...

James Lewis:

Yeah, I think it comes down to what are microservices actually, right? What do we mean when we say... I like Stephen Tilcoff's description of them. He goes through four different types of microservice architecture, and one of which I especially like because it's the just use your own head style of microservice architecture, think for yourself. I'll give you an example of that. Say you've got an application, it's a website, it's serving a bunch of web pages, but it's going to integrate off to somewhere else in able to get some comment data or some other form of data. Now, I mean, it's long been good practice to put a facade over that external data source. So we put this facade somewhere, so we'll learn the API. We can control the rate of change of that and limit change to only that data.

James Lewis:

We'll run that out of process somewhere. Is that a microservice architecture? Kind of. It kind of is. But it's also just stuff we would have done 10 years ago because it's a sensible way of controlling interactions with the outside world. So I don't know. It's not a binary I think monolith or microservice style of architecture. It's kind of what requirements do I have? How do I apply a set of patterns to those requirements, that we've learned back to the start of the podcast since 1969 and before? How do we set patterns? And sometimes that comes up looking like a distributed system and a microservice architecture.

Mike Mason:

So on that note, I'd like to thank all of the guests here, James, Zhamak and Sudarshan. I'm Mike Mason. Thanks for listening.