Democratizing programming

Podcast Transcript

Mike Mason:

Hello and welcome to the Thoughtworks Technology Podcast. I'm your host, Mike Mason and I'm here with my cohost, Neal Ford. Hi Neal.

Neal Ford:

Hello and welcome everybody. My name is Neal Ford. I'm a Director and Meme Wrangler and a Software Architect at Thoughtworks. And we're joined today by two of our Doppler members. I will let Brandon introduce himself.

Brandon Byron:

Hi folks, Brandon Byron. I'm a Technology Director at Thoughtworks.

Scott Shaw:

I'm Scott Shaw and I'm the Head of Technology for Thoughtworks in Australia. Despite my American accent, I have lived in Australia for many years.

Neal Ford:

And today, we are talking about one of the themes on the Thoughtworks Technology Radar Volume 23. Every Radar includes a few themes, which are sort of the culmination of conversations that we've had during the face-to-face meeting. And one of the themes that we're going to talk about today is called democratizing programming. Now, we don't assume that everyone who's listening to this has read the Radar. So I will go ahead and read the theme to you and then we will discuss various aspects of it. So the theme from our Radar says, several of our discussions revolve around tools and techniques that promote the democratization of programming allowing non-programmers the ability to perform tasks that previously only programmers could do. For example, solutions such as IFTTT, which is if this, then that. And Zapier have long been popular in this space. We've observed an increasing use of tools such as Amazon Honeycode, a low-code environment for creating simple business applications.

Although tools such as these provide fit to purpose programming environments, challenges arise when moving them to production scale environments. Developers and spreadsheet wizards have long managed to find compromise between domain-specific and traditional coding environments. The advent of more modern tools renews that discussion across wider domains with many of the same positive and negative trade-off. So the overall discussion here, number of our blips had to do with this increasing trend that we see of building tools that allow non-programmers to do program or sort of stuff. And we called out spreadsheets here, which is the classic example of that. But it turns out this theme has all sorts of interesting positive and negative connotations, which is what we're here today to talk about.

Mike Mason:

Interestingly enough, just earlier this week, we did a sneak peek webinar for the Radar. And one of the things that we got many, many questions on was the topic of low-code, which is extremely popular in the industry right now and is absolutely a form of democratizing programming. And so I pointed people from the webinar to the podcast. So hopefully some have... Really we have some repeat listeners for that.

Brandon Byron:

Yeah, I think that's an interesting space where you might be able to just to make a little bit of a cut: Excel, IFTT. These are MAC automated. These are built for the citizen who needs some degree of optimization, citizen program or so to speak. There's a whole other class of tools. I think Honeycode probably falls in that space that are advertised as solving problems for the enterprise. And I think that's where the conversation whirls up quite a bit of passion.

Neal Ford:

Well, not only things like Honeycode, but a lot of the integration and enterprise architecture space is now dominated by these low-code drag and drop kind of environments. We've had several clients who have mandated at the integration architecture level that only use low-code environments, which has a lot of real issues when you're talking about something as nuanced as integration architecture because the lines and boxes are pretty and they demo great. But then when you try to put them in real environments, they start suffering from the hairball effect, which is, wow, this visualization is so complex that it makes no sense anymore. And I can't understand it. And so this brings up some of the problems that inherent in low-code environment.

So one of the things particularly... So Brandon makes a really good point here that purely democratizing programming and putting tools in the hands of non developers frees up developers to do more critical stuff. But when you try to extend that into places that really should do development, one of the arguments I have against low-code environments at something like for, and I'm going to say this a little bit controversially, for someone who should know better than use a low-code environment. One of the ways that I push against that and say, "Okay. Well, that's a trade-off to use a low-code environment versus using actual code."

And so one of the ways I pushed back against that is, well, for code, I have version control. I can diff it. I've got really brilliant IDEs with code inside and refactoring support. I can unit, functional and user acceptance test that code. Can I do all of those exact same things in this low-code environment? And very often these low-code environment support none of those things, particularly that diffing and versioning and these BPL tools that are drag and drop. When you create that artifact and six months later, what's different between this artifact and the other one? I don't know. Can you do a visual diff in your brain between these two spiderwebs to see where the spiders live now? So that's one of the big issues with people that should know better using low-code environment. But it's very compelling because, hey, you don't have to write real code. You can just drag and drop or write a few little things and you're done.

Scott Shaw:

I got to admit I'm really conflicted about this because all that stuff Neal talked about, I hold really important because I'm an engineer. I've been an engineer my whole life. And the idea of engineering rigor and discipline is really important. And I think that's the value that we bring to our customers a lot of times at Thoughtworks. But on the other hand, I worry that we're creating a priesthood of programmers. I worry that we're creating this cognizanti and only we are allowed to touch the code. And in a world where software penetrates every aspect of our lives, everything we do, everything we're doing right now, especially in these times is completely controlled by software. I feel like everybody ought to have a way to manipulate it. Everybody ought to have control over their own software environment. And I don't know that that engineering discipline is necessary for everybody to learn. And it certainly creates a barrier. It creates this huge barrier for people to get over if they want to interact with their technology environment.

Brandon Byron:

And the trick is of course finding the right context for that. So on the Radar, we call it the bounded low-code platform and even the term bounded was quite controversial just amongst us. Even with the priesthood, how many CNN headlines do you see of security breaches? And now you're asking people who don't have that background to come in and drag and drop a few boxes, piece them together and say that they develop some key business functionality. I'm incredibly skeptical that could work. And I'm actually even more skeptical to Neal's point, when they start to add some claim that they can support version control or they start to add some claims they can support testing because they do it in a way generally, maybe it's XML. You can't really diff XML if in loops from one version to the next.

I think I've always been a fan of, I believe it was Gregor Hohpe who said this that when the vendors would come in and demo their wares, look how productive you can be. They'd let them get through the demo and then say, "Okay, do me a favor technical sales representative. Stepped out of the room for two minutes. I'm going to just change something in the underlying XML. And I don't want you to come back in and show me how you debug it." And I think those tools oftentimes ignore the life cycle of software. And I think that's the problem is that you don't just write something once, it's an evolution. That's why it's soft. It's not putting a building together and then walking away.

Neal Ford:

Well, I think the building analogy is really act here because one of the things that when people say, well, you're an agilist. Don't agilist, say you don't need architecture? It was like, well, it depends on how big a thing you're building. If I'm building a dog house, then I'll go to the hardware store and buy lumber and put together a dog house. If I have to build a 50 story building, I have to do some planning. I have to worry about structural analysis and security and all those things that you have to worry about with serious pieces of software. And I worry that we don't acknowledge that spectrum difference in this world that all democratizing programming environments are basically the same.

And so one of the things that... So am very much one of the gray hairs on the Doppler boards. I've been writing software for a while. And one of the things I remember, a lot of the fourth generation languages that were all the rage in the late 1990s, early 2000, which were their version of low-code environments. But when you build a Microsoft Access application, you're basically putting together a doghouse. But security concerns and scalability. I mean, we wrote a lot of Access applications that when you try to put them live, wouldn't scale and you're stuck because there's no way past that problem.

So that leads me to the other negative thing that we called out in the Building Evolution Architecture book that many of these environments have is what we refer to as the last 10% trap. So when you look at something like Microsoft Access. I worked for years in a company that built Client Applications of Microsoft Access. And we ended up shutting down that part of the company because we realized every Access project started as a booming success and ended in total failure. We wanted to understand the dynamic of why that kept happening. And the reason it kept happening was you'd start building something in Access and you could get really a lot done in almost no time at all. It was amazing.

But then you'd get 80% done, but then the next 10% is possible, but difficult. But you could pack your way to it, but that last 10%, you could never get to. And users always want 100% of what they want. And that's the problem with a lot of these environments, including 4GLs. And when I look at Amazon Honeycode, it looks to me like access and a browser. I mean, I have all of these horror. I get nightmares when I see Honeycode because I've seen so many applications that start down that road and spend three months building something and then realize we have to throw all this away because it will not do this fundamental thing that we want to do here. I think we have a hard time identifying where that line is.

Scott Shaw:

There are hundreds, thousands of applications that don't require that. That are only used by one person. That you just need the power of a programming environment to do some analysis or something like that. And why should you have to write tests and have version control and everything on that? It's convenient. I do it because I know how to use those tools and I have them at my disposal, but I don't really need to do that. And you're building analogy. I want to be able to do home repairs without having to call an architect and an engineer to approve it. I don't want to have to get a permit to fix a home, my dry wall.

Neal Ford:

But what happens if you tear out a load bearing wall?

Scott Shaw:

I'm not going to. You need to be able... I mean, this is what everything is converging on. Is there need for governance? Is there need to have some... You need to have some guidelines as to when things cross the line because I think a lot of these tools are really handy, but I've seen, just like all of you, I have seen them grow completely out of control. I've seen these BPM diagrams where you couldn't actually... It was just where the lines, the connected boxes were blue, and you bring up the screen and it's just one solid blue screen because there's so many lines and boxes on that screen.

And obviously, it can grow completely out of control. And you need to have some discipline. But where are the people who are building these tools? This is the sad thing is that we, the programmers, the software professionals are the ones who are building these tools and providing these tools to people to use. And I think it's our responsibility to provide better tools. I think it's our responsibility to provide tools that have built-in safeguards, that have a governor built-in to them that prevent you from doing things that are too complex or from doing things that it's trying to support too many users at once or doing things that are too potentially insecure or vulnerable in other ways.

Mike Mason:

But I mean, Scott, your sensible middle of the road pragmatic suggestion is not going to get the kind of VC funding and market buzz that it really needs. I mean, part of the problem here as well is I think there's a real need which all of us are... I mean, Scott's advocating for it. But I think we're a little bit in danger of being programmer cognizanti priesthood folks and sort of poo-pooing the idea that IT is failing to deliver and people are frustrated. I just want to get some simple software out the door. Why can't I get some simple software? And the story, yes, we can help you build simple software with this platform.

And actually this platform can deploy to your cloud and it can do the scaling stuff that you've wanted it to do. And actually, if you need to, you can drop down into code. You can actually get one of your programmers to spend some of their time and drop down into code. Do you know what I mean though? There's a lot of compelling storyline there as to why this kind of thing is both needed and ought to be plausible. And I think the big problem is people getting in over their heads and then not realizing that they're in over their heads. And that they, I think we've previously talked about kind of programmer sensibilities. If you don't really understand what an abstraction is, how are you going to know that you need one when you're in one of these environments. And that actually copy pasting was a problem.

Brandon Byron:

And that's where I really liked the governor metaphor that Scott described. A lot of those platforms do exactly what you just described, Mike, is they let you drag and drop your boxes and your connection points and then let you dip into code when you think you need it. But who's you in that context. It's a low-code programmer or a non-programmer who doesn't necessarily understand the value of abstraction in lifecycle evolution. I'd love to see us invert. I'd love to see us say, we have a coding environment that supports our enterprise, but this space right here can require and can benefit from some self-service evolution without programmer interaction.

Let's own this invert who owns that decision because I think then we can actually provide much better governance structure. One of the big concerns I have with these low-code platforms, when they were adopted at the enterprise level as Neal described. I'm not morally opposed to vendor lock-in. I'm okay going to AWS and using some AWS services because I can either pay now or I can pay later to get off of it, but I'm still going to have to pay. And I have some intentionality walking into that relationship. And when I have all of my code in MuleSoft or Pega or some of these BPM tools, you're at the mercy of them remaining commercially viable for years and years and years. And if not, what's your fallback. That's a very different situation than if you're in Java or Python. Those have a continuity independent of the organization that sponsors.

Scott Shaw:

I know of a high-frequency trading house that their traders are not computer professionals at all. They're financial people. They're really smart though. And they like to write scripts and they like to write scripts to adjust parameters and things then. And what we built for them is a sandbox environment. So we built a containerized environment that they could write scripts in and that would run it in and not consume too much resources. And if it ran out of control, it would be automatically killed. And that's the kind of thing that I'm talking about. We need to provide safe environments for people to be able to play in and experiment.

Neal Ford:

Well, that's an interesting case because that's the thing that really replaced the 4GLs were domain-specific languages. And one of the advantages there to Scott's point is in a domain-specific language, you can dip down into the underlying language if you want if it's written as an internal DSL and provide some of that governance. And I think that's actually the key to building effective environments like this is built environments that you can't make mistakes in and that people fully understand the limitations of. So one of the things that has made the remote Radar creation possible, which we never thought was possible before, part of it was this magnificent remote Google spreadsheet that was created that allows us to do real live voting. And it was really... That was actually one of the things that brought up the low-code discussion within the TAB meeting was like, look at how much we built on top of the spreadsheet. It's really amazing.

The real-time voting and collaboration was really nice. But we inherently understood the limitations of that because while we could get real time number of votes for each row, we could not build intermediate results into the spreadsheet because that capability's just not in the spreadsheet. But we understand the spreadsheet metaphor so well that we understand the limitations that come along with it and we accept those limitations. So I think it's a mistake to build low-code environments that pause it, that, hey, you can build anything in software in this environment. And then frustrate people when they bump into limitations versus creating a sandbox that says, you can build this kind of thing here, but you can't really go outside of that. But within that sandbox, you can build some really amazing things.

Mike Mason:

The other contrast to domain-specific languages is that they are designed very specifically as part of probably a larger system whereas with a lot of the low-code stuff, we're talking about something that way you could potentially build anything at all and the box around it is extremely fuzzy.

Neal Ford:

But see, I've always maintained that Excel was actually a domain-specific language for Matrix Junkies. And it just turns out that accountants are Matrix Junkies.

Brandon Byron:

Well, that's a great example that sort of proves the conversation because I think we've spent the last few decades trying to replace Excel and IT and largely failing because it allows so much contained isolated automation for the business to run that it's proven its worth over and over again. And anytime we try to supplant it with something that has some centralized controls, those controls are why the business goes to Excel in the first place so they can bypass it.

Mike Mason:

Well and I mean, we built our own Google spreadsheet to run the Radar off of. We didn't seek to engage internal programming resources and getting them to build something for us. And I just called people resources. I hate when someone does that. But I meant the capability. We didn't try to get programmers to build this for us, we just hack something together in a spreadsheet because well, first time around because we need something fast because the in-person meeting had been canceled. And the second time around because we'd already made a bit of an investment. We knew the value that we had gotten from it. I mean, that's the thing here. We got huge amount of value from what ultimately was limited and where we were going to run into the edges. But we wanted to make that trade-off because of the value we could get.

Scott Shaw:

I always imagine Charlton Heston screaming, "It's people!" in Soylent Green when people use that resources. I think it's an old fashioned idea maybe. But I think if we're going to talk about democratizing programming, it's incumbent on us to have an informed citizenry. If we are going to democratize program, we need to have part of our basic education. And given, like I said, that the degree that software controls our lives, we really ought to be giving people better computer literacy. Computer literacy isn't how do you use Excel and Word, it's what are the risks inherent in using computers and how do you control your own computing environments so that you can stay safe and that you can use it to your greatest advantage.

Brandon Byron:

And I think we also need better language rather than just one monolithic term programming. We're not democratizing surgery, but there are some nutritionists, chiropractors without a medical degree. There are people that you can get legitimate health advice from that didn't go to medical school. And I think when we just use one term for it and call it programming role, you're running your enterprise on low-code tools, that's a mistake. PM's down. I'm controversial or not. In my opinion, that's just a mistake. But running certain bits of automation that you need that are isolated, that don't have security implications, that have limited lifecycle implications, limited team collaboration requirements. Yeah, and we should find some of those and we should provide the appropriate level of education for the users of those, the programmers of those systems so that they can be effective in the context with which they're providing the tool in.

Mike Mason:

Yeah, I think that's really important because something that I'm very worried about right now is that you have IT are having a history of not delivering very much and disappointing the business and business being frustrated that they want more software. And these new tools being touted as a silver bullet. And those of us in the programming community saying, well, those are bad because of these 10 reasons. But then ending up talking past the folks who have the need and potentially either us in the programming community being written off and those tools being used anyhow. If it's a tool in the toolbox, I think we should know how and when to apply it so that if we're in an organization that feels a need for that kind of stuff, that they could take it on safely and understand some of the boundaries and lightweight governance that you might need and stuff like that.

Neal Ford:

Well, towards Scott's example, I think this is exactly right. What we're lacking in this world is when should you stop doing this? Because the examples we've given before Brandon just talking about medicine. Well, if I'm trying to get rid of a sinus infection, I'll go to a website. But if my leg falls off, I'm going to a doctor. If you're renovating your house and you need to knock out a doorframe, that's okay. But if there's a wall that you think may be load bearing, you better call an engineer. So we have some sense in the real world of when do we need to escalate this to an expert where as in programming, I mean, as in our prep call for this week, we were talking about and this is true throughout the world including within Thoughtworks.

Many of the very critical functions within most companies in the world are run through spreadsheet Macros that are not documented at all and they live in one person's brain. And when that person leaves, it is sometimes a forensic exercise to reverse engineer. Exactly, what is this? Because it's not an attractive programming language, these Excel formulas that grew into... And again, if you use proper extraction and create name cells as variables and that kind of... But users like that typically don't do extraction. So I think that's part of the problem of really democratizing programming is finding the place where people understand intuitively that you should stop doing this and go to a professional. And it doesn't seem anybody has that good sense in the programming world.

Scott Shaw:

Those people are probably some of the best paid people in IT. I've seen and I think there's one guide that, and it's usually a guy that has the spreadsheet that runs a business and nobody else understands it. They got to keep them around.

Neal Ford:

That's the famous Dilbert joke. They bumped into the guy just like, "Oh, wow. You're the guy who wrote the spaghetti accounting code 25 years ago." And he said, "Yeah, job security. Hey, I like you guys. You get a little bump in your pay next week." That's the priesthood that we were talking about before.

Scott Shaw:

There is this other aspect of needing to democratize programming that we should mention and that is that us software engineer professionals all look pretty much like the four of us, four white guys here. And something like 2/3 of professional programmers are exactly that. And as a result, we're probably writing software for that audience. And if we really need to expand the pool of people that we have to draw on to write programs. And so perhaps a different set of tools, a different way of approaching it is what's needed to bring more people in, to bring a more diverse set of people into the world of programming.

Brandon Byron:

Yeah, that's a really interesting thought. I had a conversation earlier in the week where as you survey the landscape of first-class programming languages, they all have just a natural, strong English bias, if and while and for. Whereas when you're dragging and dropping boxes, you can localize that description and the user interface that you're working on and you can get something that's a bit more natural to you. So I think that is one of the lessons that we needed to pull back into our own field rather than trying to push it off into these user interfaces.

Scott Shaw:

I do think those tools lack expressiveness though. I do. I think that's one of the problems with those low-code tools is it's difficult to express intent in any language in them because they're just a bunch of generic operations strung together. There's this weird notion that somehow if you draw a picture, it's going to be easier for people to understand. But we've known for many, many years that text-based instructions for the computer is the clearest way to explain the business intent of what you're talking to do.

Neal Ford:

So maybe the democratizing was a good name because I'll quote Winston Churchill that, "Democracy is the worst form of government except for all the others." And so maybe this is the worst form of programming except for all the others. So now maybe the question in front of us is we've touched on a little bit of this, but what can we do to actually make this better? Make it so that we can empower citizen developers. What kind of governance can we put in place? What kind of safeguards? How can we indicate to people when you've gone too far and you're about to tear down a load-bearing wall and you're going to create a security problem for our enterprise? And how can we build those kinds of checks into either our ecosystem or is this something that's contingent on the tool vendors to build in? Or is this something we can externalize from those tools?

Brandon Byron:

Well, the tool vendors will never have a commercial incentive to build it in because they'll get the most market adoption through the more Turing complete they make their tool. So I think it's going to have to come through education and industry awareness. There's a pretty clear line in my head that the sort of enterprise capabilities that a lot of IT organizations spend the most of their money on are better served with first-class tools, text-based tools that Scott was describing.

But I think some of the self-service capabilities where, no, you don't get to integrate this into our website, because we haven't vetted the security and we don't know that we can maintain it. We don't know that we can collaborate on this as a team. That's a pretty clear dividing line. So that's fine. You can do this for your internal analytics amongst your marketing group that you're working with and providing some value for that. You're not bottlenecked by IT to get that, the information, the insights you're looking for. But anything that integrates into our transactional flow of running a business, that's a different story. We need to put some symmetry governments on that.

Mike Mason:

I mean, I would say, I think the education piece is really interesting that Scott was talking about trying to teach more people at least to have a passing familiarity with programming concepts. I'm not sure where this needs to end up, but with... Back to the house building analogy. One of the biggest things that you need to know when doing a programming like activity is when do I need an abstraction? When have I... Okay, I've done the same thing three times, how do I stop repeating myself and use an abstraction to encapsulate that. And I think we all as programmers kind of understand that through years of pain of initially writing and then probably inheriting bad code that didn't do that kind of thing.

With the load bearing wall thing, s a 20 year old, I wouldn't have thought about a load bearing wall. That's not a concept that I would have understood. But eventually, someone older and wiser than you, as you're thinking about that kind of thing, points out what a load bearing wall is or you watch a home renovation show and someone has a disaster and you get to understand what a load bearing wall is. And now they concept it because it's physical. You're like, okay, I get that. I feel we need some amount of that for allowing programming to be a bit more accessible to people so that they at least get that feeling. If they've got an entire screen full of blue because all the boxes are joined up to all the other boxes, they should have a gut instinct that something has gone wrong here. I should be able to do this in a better way somehow. And to me, the education approach, whatever it is would somehow convey that to people.

Neal Ford:

Yeah, that's actually one of my biggest complaints to gets things like rules engines is that you hand over the rules engines to a bunch of business users to go, okay, problem solved. We don't have to get developers involved. But if you have a rule that's very slightly different than another rule, well, most business users don't understand. And rules engine won't let you create an abstraction so you can make that common. So you just copy and paste and change the little things. So after years of rules engines, you see the same rules replicated over and over with slightly different steps and you can't understand it anymore. So exactly Mike’s point.

But the problem with that is every time my accountant tries to explain complex accounting stuff to me, my eyes glaze over and I just, I have a hard time paying attention to him because I'm just not that interested in accounting stuff. I need to be. It's a responsible adult thing, but ah, it's just so hard to get interested in that. And that's what most people when I try talking about programming, they get that same look in their eyes about abstraction. Even getting them to understand a few simple things, it seems like this big elaborate thing that I don't want to be a programmer. I just want to get some stuff done. And it's like, we got to find a way to convey that in a very easy to understand chunk so that they understand what the danger is, but don't get into the distinctions between Lisp and Scala.

Mike Mason:

I actually think one potential positive avenue might be, if you are using a low-code platform to have programmers working a bit more closely with the business folks who are using it so that they can pick up on those kinds of things because explaining it to someone in the context of the problem that they're trying to solve, that, hey, an abstraction or we're using something might be a better approach than copy paste. In context, that's much more likely to stick as a lesson than trying to do it in the abstract.

Neal Ford:

Absolutely, that's also true for design patterns. You can read the design patterns book until your eyes bleed. But when you actually see it happen, that's when you actually understand them. Same for those kinds of abstractions that Mike is talking about.

Scott Shaw:

Yeah, we did get a question that I didn't get a chance to answer it in the webinar when we discussed this about multidisciplinary teams and how does low-code or what is the role of low-code or what implications does it have from... And I think that having a multidisciplinary team is a really good way to get around this problem because if you've got somebody that has deep knowledge of the engineering discipline needed to write code. But then you have other people that had to have more of a design sense and other people that have more business knowledge and you have those people working together, I think you come up with a stronger outcome every time.

Neal Ford:

Yeah, we need the equivalent of, I'm about to tear down a wall. I need to go check with an engineer and an architect. Or I'm about to amputate one of my toes. I should check with a doctor. We need the program or equivalent of that for someone who's using a low-code environment.

Mike Mason:

Okay, well, that's about all we have time for this week. Thank you very much for listening and thank you to Neal for co-hosting and Brandon and Scott, our guests on this week's podcast.