Why the tech industry needs "Expert Generalists"

Episode transcript

Prem Chandrasekaran: Welcome to yet another exciting episode of the Thoughtworks Technology Podcast. My name is Prem. I'm one of the regular hosts on the podcast. I'm joined by my co-host, Lilly Ryan. Lilly?

Lilly Ryan: Hi. I'm Lilly Ryan. I'm a principal cybersecurity engineer here at ThoughtWorks and another one of the regular hosts here today.

Prem: Today we've got two very illustrious guests, Martin Fowler, you obviously know Martin, I guess, and Unmesh Joshi, who is one of our luminaries out of the India office. Do you want to quickly introduce yourselves?

Unmesh Joshi: Yes. I can go first, yes. My name is Unmesh, and I'm a principal consultant at Thoughtworks. I've been with Thoughtworks for quite some time, but I like to call myself working for Office of Developer Happiness. That's what I call it.

Martin Fowler: I'm Martin Fowler, a loudmouth. I've been around at Thoughtworks a long time. As I've said in my recent bio, I'm past it, and I'm living off the intellectual blood of my younger colleagues, and that blood is very tasty.

Prem: Wonderful. Today we are going to be talking about this topic and blog post that you folks wrote on martinfowler.com, which is on this thing called the expert generalist. Let's get right into it and clarify what you mean by expert generalist.

Unmesh: I can go a little back in time. A while back, I was working with Martin on writing about patterns of distributed systems, and we published a book a couple of years back. While working on that, we figured out that there are a lot of commonalities under the hood of systems which look very different on the face of it. We saw in many organizations, essentially, very siloed verticals getting built around tools like data engineers, infra, and DevOps, and full-stack developers who generally just work on microservices and React front-ends.

I thought that if there are commonalities in this pattern's world, and if we can probably show it to everyone, people who are siloed into their verticals, it will be easier to cross-pollinate people. You want to form a common language across. I was experimenting with a few workshops, which I did essentially for these infra, DevOps, and application developers, and data engineers. That worked pretty well, and people seemed to resonate with it.

I thought, naturally, I could relate it to how we work at Thoughtworks and a lot of colleagues who I admire I've seen them working in this fashion, trying to understand the first principle patterns and not limiting themselves to any particular vertical or any particular technology, or even in any particular role. I've seen people moving from BAs to developers to infra and stuff like that. I thought we need to acknowledge these expertise. People, if you just call them generalists, there is a perception that they will have a shallow understanding of everything, and they're jack of all trades. That's the word people use. This is not that.

This is people who understand systems at good depths, and their approach always is to understand the first principle patterns. That helps them to navigate through many such siloed verticals. That's the reason I like calling it expert, but a generalist. Martin, you want to add anything? It seemed to resonate well with Martin when I was discussing this with Martin.

Martin: Yes, it resonated very strongly with me because the heart of my thinking about software development has always been that it's the people that matter most. I very much like to reinforce the very first line of the Agile manifesto, well, first value sorry of the Agile manifesto, that says that individuals and interactions are more important than processes and tools.

My point is getting the right, getting good people who work well together, interact well together. If you've got that, then everything else is secondary. Then the question is, how do you identify those individuals and interactions that you want to pull together like that? Certainly, one of my other touch points is that the particular tools and technologies du jour are terribly temporary things. I've been in the industry now for a long time, but stuff that was stuff you had to know when I started is all forgotten now. What's important is your ability to learn things. I've always felt it's far more important to be able to learn than it is what you know.

That's been part of my long-term value system. I've been getting increasingly frustrated in recent years by seeing an industry that's always tried to go down the 'we want very particular specialties' it seems to have gotten much, much worse. I've seen it in a couple of different dimensions. One dimension of it is in Thoughtworks business. We see clients increasingly saying, "I want three and a half years Databricks experience. You sent me somebody with only three years Databricks experience," or "You sent me with five years of Snowflake experience. That's no good."

Then I see it in the broader sense, talking to other colleagues who are going through recruitment. They may have 20 years' experience and they're going into interviews where they're being asked questions about specific tools that they haven't had experience with and they're being asked questions that they know are not relevant to the job that they're being asked to do. What we're seeing is enormous frustration due to that. I've been struggling with how to deal with that frustration. One of the things I was experimenting more in my conversations with folks was to say, well, again, "It's the people that matter, the individual's interactions. How do you get to the right individuals?"

Trying to talk about what are the real characteristics of the people that really matter? Then when Unmesh talked to me about what he was doing, it just really clicked together because I think what Unmesh did was two things that seem incredibly obvious in hindsight, which is all the best ideas seem bloody obvious in hindsight. One is that the skill of being a really good generalist is an explicit skill that we should actually talk out and treat the same way as we do when we talk about skills in Java programming or whatever. There is a skill of being of a generalist, and that should be a first-class skill that we relate as much as anything else.

The second thing is we can teach this, and what he was doing with his workshops was actually doing a step towards teaching people this skill. That really resonated with me. I think it was back in last fall you first talked to me about this, Unmesh. It was quite a while ago. I said, "Unmesh, you've got to write about this." Of course, Unmesh was busy with a gazillion other things, and it took a long time to actually get it to hatch. I really felt we wanted to make something obvious, because I felt there's a real problem in the industry, and we have to fight against it.

Lilly: We have a plethora of words that approach this concept. There are things like the T-shaped person, systems thinker, that concept tends to describe a little bit of what you're approaching with the expert generalist. What was it about that terminology, the existing terminology, that failed to address what you were talking about? Why did you feel the need to say an expert generalist is its own thing? We need a definition for this concept that we're not seeing expressed anywhere else?

Unmesh: A couple of things in there, particularly the T-shaped when we talk about people. Our industry is pretty tool-oriented, and more so in the recent cloud era. People can very easily build a perception that T is I build expertise in one tool, like the Databricks example that Martin gave or ReactJS and then everything else is just a shallow. Maybe I will just do some work in that. The people that we have observed and the qualities that we have seen, it's actually ability to build multiple verticals, not just one of the T. Martin and I, when we were discussing Martin used the term called comm shift, where you have this multiple not just one vertical bar of the T. The other aspect is that what you focus on.

When you say you're building a specialty, it's like going to this first principle patterns. We have used the term sympathy for domains. While you build that specialty or special sympathy for the expertise that you built in at the first principles level and understand the patterns, you also have sympathy towards other things. If you are working on, let's say, you build expertise in distributed data systems, let's say, but you still have sympathy towards UI and how people access that data. You will see that in the end, any data solution is built by certain access patterns in mind. Having that understanding that nothing is isolated and nothing is designed in isolation is very, very critical, I thought. That's why calling this expert generalist as a first-class term, I thought was useful.

Prem: Okay, let me play devil's advocate. This is what I tend to do on these podcasts. One of the things, one of the criticisms about the use of the term, whether you call it a T-shaped person or an expert generalist or whatever else related to that, the concern is that these folks join the team and then they actually have a negative effect on the productivity because they're not at the same expertise level as the others on the team who are using said technology, whether it be the Snowflake or Databricks or what have you. How do you counter that?

Unmesh: Two things to consider. One is when we call someone as expert generalist, I mentioned that they are expert in first principles. When your focus is first principles and patterns and the other is you own end-to-end things. It's not like you are limiting yourself to just one aspect of your entire project. This focus on, I'll say, getting things done end-to-end, that's a game changer. I'll tell you from a couple of my experiences, when I started figuring out what distributed systems are, it's a pretty old topic and a lot of history in academia.

Unless you write, try to see, let's say, how HCD code is written and why it's written that way, and if I need to write something like that, what I need to consider, you can't understand things beyond a point. This approach of getting things done and owning things. I've seen one of the great experiences I had was working with this institute called Ayuka in Pune. This is inter-university center for astrophysics. We were working on some consulting assignment for setting up data centers for their astronomical data.

People whom we were talking to, in the initial meetings, we thought that they are the infrastructure experts who have good experience in setting up data centers. It turned out that they were renowned astrophysicists. They just learned setting up the data center stuff just because they had to manage their own data. That was very, very inspiring. I've seen many such incidences in the past. Once you have people with right mindset and right focus, I think what you're saying doesn't happen.

Martin: This also gets back to the comment about is this just a T-shaped person? The thing is that the skills blend that we imply from phrases like T-shaped or cone-shaped or paint drip or whatever, that's one aspect of what we're talking about. There are other important characteristics as well. That's why we highlight six characteristics in the article. Other characteristics that are really important are collaborativeness, the fact that these people are very good at collaborating with people outside of their domain and working with them. The point here is that when you've got those kinds of skills, you can very quickly come up to speed in a team, even when they're working with unfamiliar technology.

We've seen it many times where we bring staff folks onto a project and you may have one or two who know the technology fairly well, but most of them don't. It actually doesn't really matter because if they've got those key characteristics, if they're collaborative, if they're focused on the customer problem, if they understand some of the fundamental technologies, they can learn very rapidly from the deep people on the team.

You only actually need a few deep people on that team who can be there answering questions and providing the guidance, and the other people will accelerate rapidly. Because they have these other characteristics, in particular the customer focus, the getting things done element, they will actually be more productive than adding yet more specialists would give to the team. This is where I think it's really important that this expert generalist skill is a skill that is always valuable in this situation because it blends with other skills and it's the blending that matters.

Unmesh: I'll add to what Martin said. This collaborativeness is crucial. In the internet age, I've seen people reaching out to anyone on the planet. You don't hold yourself back saying, I need to undergo this particular training and only then I will probably start contributing. You don't find these people doing this thing. They reach out to anyone on the planet practically.

Lilly: You mentioned earlier that it's very easy to say in a job description, we need somebody with three and a half years of TypeScript experience or whatever it might be. You can get three and a half years of TypeScript experience in three and a half years. Can you be an expert generalist in three and a half years?

Unmesh: Well, I think you are an expert generalist forever. I'm still learning, after 25 years, so you can never say that I'm now a true expert generalist. I think the qualities that you imbibe and focus on and the three Cs that Martin talked about, that helps you get there quickly.

Lilly: What I am getting at there, I suppose, is that could somebody who is relatively new to the industry, to the workforce in general, be considered an expert generalist fairly quickly? The word expert does imply level of seniority. If that's not the case, then how do we grow somebody to that point? Some of what you're discussing tends to come only with experience and can't really be taught. It has to be learned as one goes. What do you feel is crucial for growing this in someone? How do you know when someone is an expert generalist?

Unmesh: Right. I have some very good experiences with very junior folks. We run this program called STEP where people who are undergraduate or who are doing their undergraduate courses, we get them and teach them through a bootcamp. I think I have had very good experiences with those folks, people who are just fresh out of their college or even before that. Sometimes I prefer those people on the team as opposed to some senior folks. It's because the bootcamps they go through at Thoughtworks, what you encourage is allow them to ask questions or encourage them to ask questions. Encourage them to ask for help. Encourage them to focus on, as I said, the first principles and patterns and not on the syntax.

Encourage them to not limit themselves to very siloed verticals. Organizationally, if your career growth essentially is not marked by a specific path that you have to be like, if you become a UI developer, then you will become a senior UI developer and then a UI architect. Only then you will get promotions and salary hikes. If those things are not tied like that, then people are free and fearless to experiment. I think that's very crucial. If you see at Thoughtworks, a lot of boot camps that we run for freshers or training programs like universities, I think these things I've seen we focus on. That helps anyone, even new to industry, having these capabilities.

Martin: Yes, I think like any skill, it takes time to develop. What you can do, even with fairly junior people, is assess the potential. I think that's one of the things that we have historically done very well, is we've looked for people who have those. Again, the core characteristics that we talk about, curiosity, collaborativeness, a customer focus, a respect of fundamental knowledge, these things are all things that can be the potential you can look at. I think if you have those characteristics, you can accelerate then that path towards the expert generalist skill. What's key is, again, as Unmesh says, you've got to have a program in mind that leads you in that direction, that doesn't force you into a specialty all the time.

Prem: It looks like what we are saying is these so-called expert generalists favor fundamental knowledge, meaning instead of being a Ruby programmer or a Java programmer, focus on being a really good programmer. Then now when you switch from Ruby to Java to C# to whatever the next new thing that arrives, you will be in a much better position to be able to cope with the change in ecosystem. That's what we seem to be saying. Let's talk about the flip side of this. Are we saying then that there is no place for specialists on teams? I know we talked about a mix but are we going to penalize people who choose to specialize in, let's say, I want to be a UI specialist or I want to be a data specialist and so on and so forth. Are those kinds of people at a disadvantage now?

Unmesh: The words that you said, like UI or data, I consider them as broader domains. I think that's important because you need to have broader domain expertise and you need focused efforts to gain expertise in that. What's problematic is when you tie your specialty to a particular product or framework or programming language and stuff like that. There is definitely a need for specialists and deeper knowledge in a particular technology domain, I would say. Sometimes, I must say that even in certain products like Databricks or AWS Cloud, you need people who are aware of some deeper knowledge of that ecosystem. The focus should be on broader domains is what is important.

Lilly: The way I'm hearing it when you're speaking sounds an awful lot like the concept of a general practitioner in medicine, somebody who you can go to with a wide variety of problems where they might then also be able to say, well, it sounds like you probably need a nephrologist. It sounds like you probably actually need to speak to a dermatologist, for example, someone who specializes in a different domain.

What is really interesting, I think, in this context and something I'd love for you to unpack for us is how when you are in this expert generalist role you know when it is appropriate to hand over to somebody who does have that deep specialty and that deep expertise and where your boundaries are. Is there any guidance you would give to people who are looking at this role in terms of where it's appropriate to hand off?

Unmesh: I would say, at least in my experience, any good professional will start asking for help right from start. Generally, you won't go in isolation and study something and only, maybe after a week, when you figure out that you can't solve it, you reach out to someone. It's not like that. It happens from the beginning. I don't know if that answers your question, but that collaborativeness is crucial, I think.

Lilly: Yes. I suppose it does require an understanding of your own limits. One of the concepts that you've mentioned in the article is that collaborativeness and the curiosity, the humility that you have to have as well as being part of a team and being quite self-aware of where your own boundaries are.

Unmesh: No, absolutely. Absolutely. That humility is so important, particularly when you become experienced and you yourself become expert in certain domains. People feel awkward asking for basic help. That humility tied to the collaborativeness, I think that's very crucial. You should not be shy asking for basic help because there are so many things and you can't learn everything. That's one of the great learnings I myself had after so many years. I don't know a lot of things even after 25 years in the industry. I don't feel awkward about it. I ask for help and sometimes even to very junior folks. A lot of times, people who are fresh out of college they know certain things more than me and I enjoy learning from them.

Prem: Okay, are there specific tools, technologies, techniques, patterns that these folks can use? Let's say I'm a developer or a IT professional who is straight out of college. How do I get to becoming an expert generalist quickly? Is there no shortcut to this?

Unmesh: One thing that has helped me is talking to like-minded people because, and when I started coding early '90s, it was, and I was talking to Martin about it, it was all object-oriented coding and user interfaces. That was the key. A lot of senior folks asked me to learn UML and a specific framework like Microsoft foundation classes. That was not probably what was a long lasting underlying technical trend. Figuring out, beneath all the noise that goes on in our industry after every few years because of the buzzwords and different tools which become popular, you need to be constantly talking to like-minded people and figure out what this underlying technical trends are. That helps a lot.

Martin: It's one of these things where we don't have much experience of trying to develop this as a named expertise because it's something we don't really pay conscious first-class attention to. In terms of advice to people who are early on in their career, what I would say is find people who do seem to have this ability, who do have this spread and treat them as mentors and try to learn from them. Also, be very wary of getting pigeonholed into specific tools or specific technologies. Always try to look at least a step beyond and pay attention to the other things that you need to collaborate with.

If you're having to work with something that crosses a boundary, go and peek around the other side of that boundary a bit and get some sense of what's going out there. This development of sympathy related domains, as we talk about in the article, is a very valuable skill and you can reach out and try to get there yourself. Usually what encourages this, of course, is your natural curiosity. That's why curiosity is such a useful characteristic of these folks.

Prem: In the words of Steve Jobs, be curious, be foolish. Is that the mentality that we need to adopt? Great, yes. Look, we are in 2025, and it's the world of AI. It's the world of generative AI these days. Is there a place for these kinds of folks in this new world where everybody talks about getting things done really quickly? Do we need expert generalists in this era?

Unmesh: Oh, yes, I think you need more of those in the LLM era. I enjoy LLM. There are so many things piled up over the last several years that I wanted to try, just couldn't try it out. I'm trying so many of those things with all these LLM tools that we have. That entry barrier to trying out anything new has significantly gone down with LLMs. I wanted to learn some UI stuff and I could do that. I wanted to learn Jetson test suite and how that's done, I could do that. The entry barrier to different programming languages has gone down. I think you will need more and more expert generalist skills if you want to really use LLMs well.

Lilly: We've spoken a fair bit about what it looks like if you yourself are trying to become an expert generalist or find expert generalists to learn from. What about for the folks who are actually putting together these teams or putting out these advertisements, asking for people with X number of years of expertise in something? What team composition do you think would ideally work best in terms of the balance between people who do have deep expertise and we want to make sure that we do learn from them and expert generalists?

I can see cases where a team of expert generalists may not succeed, but with the guidance of a specialist or two could absolutely elevate everything more than a team of dedicated specialists would. Where do you think the balance should be? How can people assure that whatever they're doing, they're getting that balance in the teams that they're putting together?

Unmesh: There are two aspects to it. I think a lot of times why people or clients ask for specific tools and expertise in specific tools is it's not easy to figure out how to assess a particular expertise. In the industry where tools and frameworks, they become limelight and buzzwords. For example, I'll give you an example of things like Kubernetes and data. If you have to assess people on their expertise in distributed systems, it's hard. A lot of people don't even know that, okay, they need to assess on expertise for distributed systems. That's the first problem. The simple route is that you know that we are going to use this tool or framework and it's easy to then just ask for specific thing.

The other is generally the qualities of expert generalists and the people that we have talked about. If you have majority of them in the team with just enough, maybe a couple of experts in specific tools and frameworks, that combination works pretty well. I have seen that even from plans perspective, that provides great flexibility. If you want to staff a team of, say, 20 people, you don't need 20 people expert in specific tool or technology. You maybe need a few of them and then you look for these other attributes in other people that you get on the team.

Lilly: Apart from tools and technologies, when we're talking about focuses or practices, for example, being a security specialist myself, if you were going to look at security from that point of view, and you want to make sure that the people who are around you are going to pick up on those sets of ideas, but it's not necessarily a tool or a technology but more of a mindset or an approach to a task, how would you look for that input from your expert generalists and supplement your team specialists with people who can help?

Unmesh: You're asking about how to assess people on those skills or what?

Lilly: Yes, how to craft a team that has the right balance of specialists and expert generalists for the task at hand when you are in that position of looking around trying to get some people in the door.

Unmesh: I think that the key aspect there is owning the problem and getting things done. The astrophysicists example that I gave, it was not their job to set up a data center. When you treat something as the problem that you own, it's very different. I talk to a lot of people and tell them that when you are signing up for a certain job, imagine that it's your own startup. The mindset then shifts. You don't expect that someone else will help me in this particular aspect and someone else will help me in that particular aspect. You need to own things end to end. That mindset shift is crucial. When you encourage people and ask people to go in that direction, I think that helps a lot. Then you have one or two specialists on the team, that combination works perfectly well.

Martin: I think it's tricky, though, to talk about how to design team composition in the abstract like this, because it's a very difficult exercise to go through. One advantage of expert generalists is because they are fairly broad, and because they have this more collaborative style or thing, they have a sense of the skills that are missing and can therefore reach out and say, oh, we need some deeper expertise in these topics.

That being said, there is still the biggest danger, which is the skills you don't know that you don't have. I don't know a way to spot that because it's looking for the invisible stuff. You can really only get that by having enough people that may have collided enough to realize, oh, we need this to bring this skill in. Of course, security is often a great example of the thing that people don't know that they need to know about, because it's the kind of thing that doesn't crop up until something goes wrong, and by then it's too late.

Prem: Are there specific anti-patterns then, in terms of how we approach this, whether we choose to staff teams, whether we add these kinds of people to specific contexts or not add them to specific contexts? Are there things that we need to avoid when talking about people in these terms?

Unmesh: Yes, one of the things is that there are unknown unknowns, and you need people who are aware of these unknown unknowns. A lot of times I call these as blind spots that you have while you are driving. When you are aware of this, if you're not aware of this, then as Martin said, the security things, classic also in distributed systems that you won't know about something unless things go wrong. Being mindful of this is crucial.

I'm just thinking about it on the fly now, but all the agile practices that we had been talking about for the last two decades, having stand-ups, sitting on the same table, talking to your customers, having safety nets around what you do. All of that, I think they increase the possibilities of finding these unknown unknowns early on and knowing your blind spots. I think you can create, and you have to give importance to these things. They look like small things, but these are crucial things.

Prem: It looks like you're saying that we do need a mix, so we can't have a team full of expert generalists. We need a mix and hopefully everybody is learning from the other person on the team, other people on the team, and thereby increasing the amount of value that we can add without having to specifically depend on these individuals who are experts at, let's say, I don't want to pick on security, but I guess I'm going to. Now everybody else is learning from the expert and becoming experts themselves, not at the same level maybe, but definitely enough to get the job done.

Unmesh: No, absolutely. I think that whole team aspect of extreme programming, I think that's crucial. One of the problems that you can go into is if you designate someone as a security expert on the team, then the natural tendency is to go to that person for every security problem. That shouldn't happen. If you have more expert generalists on the team, that won't happen. If that happens, then you don't have right people on the team. That's the thing to note.

Prem: Any final parting thoughts for our listeners on this topic?

Unmesh: Yes, I do read the article and try to reflect on how you work and the work that you do. I think all these qualities that we have talked about in the article, I think with AI and LLM, people are going to enjoy more and more if they have these qualities.

Martin: Yes, my conclusion, the point here is that the point of what we're trying to do with this article is to shine a light on a vital skill that we've tended to ignore because of the fact that it's a generalist skill. It's never really been treated as a thing in its own right that we have to put a lot of attention to. It's weird because even though it's been around for ages, we're having to treat this like something relatively new because we can't answer many of these questions that we've been talking about on this podcast because of the fact we've just not really looked at this skill as a first class skill yet. When we put together six core characteristics in our article, is that the only six that you could put out?

Is it the best six? Almost certainly not. It's just the six that, for us, made sense. I think we need to look more at this skill and then more at how do we identify it? How do we identify people with a potential in it? How do we grow it? The workshops that Unmesh has been doing to try and deliberately grow this skill, I think it's something we have to look at more and develop more. To some extent, it's what I know some people in academia have said that they try to want to do, but they want to teach people general skills in computer science as opposed to specific technologies.

I don't think we can leave it entirely to the academic world, because one of the problems is that there's not enough back flow in between the professional world and the academic world. We within our profession have to do more to develop and grow this and talk more about this and think about what this looks like. I would say that what we've got at the moment is sort of fuzzily looking at this picture and trying to get it into focus. It may take a while before we get that into focus, but at least we can make the attempt to get there.

Prem: Thank you very much, Unmesh and Martin, and of course, Lilly. This was a really invigorating conversation. I really enjoyed it and I hope our listeners enjoy it just as much as we did recording this. Until next time, thank you very much.