Macro Trends in the Technology Industry

We publish our Technology Radar twice a year: it’s a snapshot of current and future technologies that we think are important and relevant across the industry. We assemble about 20 of our most senior technologists from around the world to create the Radar and it’s a great opportunity to compare trends and directions using a global view. We summarize major trends in the Radar itself, but there is enough going on that a longer article is appropriate too. In this piece, we highlight some of these macro trends that can’t be covered directly in the Radar.

Empowered Teams and Businesses

You need more than just great software to build a business. There’s an explosion of open-source being released, including some companies giving away software you might think is proprietary or a competitive advantage. Examples include Google’s TensorFlow Deep Learning toolkit and Netflix’s entire production stack. At first blush this seems like a bad idea: competitors could use this code to gain a leg-up and to compete. But the real “secret sauce” for Google isn’t the software, it’s the data that sits underneath it. For Netflix their cloud scaling isn’t (any longer) their competitive advantage, it’s their original content, licensing deals, and brand power. These are high profile examples where although software and tech powers the business, it’s not the real reason that business succeeds. We’re seeing many successful players focus on Product Thinking to build great experiences, while attracting and retaining excellent staff to build those products. In these cases open-source is used more as a recruiting and credentialization tool and companies can give software away for free and still be market leaders.

Autonomous teams gain whole stack control. Over the last few years we’ve seen the rise of “you build it, you run it” agile teams: the same developers create software, release it, and support it into production. We’re big fans of this approach, not least because doing production support makes you a much better developer. This trend is expanding and we’re seeing teams gain even deeper control, choosing the PaaS on which to deploy and run, their (usually cloud-based) load and performance test tool, monitoring tools, and security tooling. This is a good trend, but at the end of the day some level of standardization is usually appropriate in the enterprise context. We like Spotify’s approach of allowing teams to innovate and then promoting the best solutions—it’s a good balance between innovation and efficiency.

Open-source as a virtuous byproduct. Innovation in open-source is happening faster than ever before. Earlier we mentioned Google and Netflix releasing open-source libraries, but type “open source” and a company name into a search engine and you’ll likely find a page dedicated to that company’s contributions to OSS. There are two main reasons companies are investing in open-source. The first is talent acquisition: with business and society increasingly powered by technology, finding the best talent is a critical success factor. But as a technologist, figuring out whether you really want to work for a particular company is difficult—there’s a world of difference between an interview process and actually working somewhere. Open-source can help a candidate can gain deep insight into an organization by looking at the quality of the software it produces, and can help a company demonstrate its technical culture to potential recruits. The second reason for creating open-source is as a cultural statement. Many technologists believe in the power of sharing and reuse, and contributing to open-source is a way companies can demonstrate their alignment with this philosophy.

Platforms and Ecosystems Enhance Productivity

Developers are communicating primarily through code. The best way to communicate about a technical problem is often in code. One of the Agile Manifesto principles is that “working software” trumps theory or opinion and we see this reflected in open-source projects. Rather than trying to explain all the nuance of a technical problem as prose, developers will often write some (or all) of the code, using GitHub’s “social coding” features to easily discuss and accept or reject suggestions. Communicating directly through code often allows much more efficient discussion and forward progress, and we’re starting to see these techniques adopted in the enterprise context as well. Companies are adopting an “internal open-source” model where source code is readily shared and used for collaboration. The enterprise should ignore this trend at its peril.

Everyone should adopt a PaaS, but today’s choices won’t be an end-state. The industry evolution from racks of servers hosted in a data centre to virtualized infrastructure available on demand has been exciting to be a part of, and ThoughtWorks has long been at the leading edge of movements such as DevOps and Continuous Delivery. We’re often asked by clients to help them with a cloud strategy and our advice today is likely to be “you need to adopt PaaS.” Many organizations have embraced IaaS and created tangled “chuppet”-based domain-specific PaaS, often held together with duct tape and shell scripts. Whilst these home-grown platforms have served their purpose we think the PaaS products available today are mature enough to be adopted. The range of PaaS options is significant, ranging from highly structured offerings such as Cloud Foundry and OpenShift to unstructured container-centric offerings such as Mesos and Kubernetes. We don’t think these platforms are a panacea; there are likely times that you’ll need to dip down to the underlying IaaS to achieve an outcome (such as configuring complex networking) or they reduce flexibility (such as making it harder to use a specific service locator library). But a PaaS, implemented well, can bring huge productivity gains to most Enterprise development shops. We’ve seen a 2x productivity boost simply by standardizing on a platform and gaining efficiencies deploying and running apps. PaaS is absolutely game-changing over asking 20 teams to figure out a deployment strategy themselves.

Docker, Docker, Docker. An industry overview would be incomplete if it missed Docker, the containerization phenomenon that has captured the imagination of everyone from developers to testers to production operations specialists. Docker makes it easy to wrap an application or component into a container, then manage that container through various lifecycles. Docker is great for development scenarios such as spinning up multiple microservices on a laptop, all the way to managing giant production loads across a datacentre with Docker images as the unit of scaling and management. What we think is most interesting here is the high-energy ecosystem surrounding Docker. There are competing tools, but the mindshare and investment in Docker is significant, and it’s likely to be the basis of architectural designs, development strategies, and production PaaS platforms for the foreseeable future.

The Swift ecosystem picks up steam. We discussed a number of Swift tools, libraries and frameworks when creating this edition of the Radar, although not all of them made it into blips. As with many platforms over the years—Microsoft’s .NET, Ruby on Rails, Scala and Clojure—developers are recreating their favourite software using the Swift language. This is a good sign that the Swift platform is reaching maturity and is a good choice for iOS development. In particular, we’d question any new project choosing Objective-C as a starting point.

Underlying “Compute Fabric” Innovation Continues Apace

Foundational technologies—IoT, VR and blockchain—give us radical new building blocks.
Tweet this

When evaluating tech trends it can be instructional to look at the underlying engineering and “physics” that power them. The Internet of Things (IoT), for example, has been enabled by underlying engineering advancements including better batteries, lower power chipsets with good compute performance and ubiquitous network connectivity. These smart, connected devices will bring many benefits, first in industrial and commercial settings where efficiency and power savings can be significant. Consumer IoT remains shallow and gimmicky, with major security and privacy concerns to be resolved over the next couple of years. Virtual reality is becoming widely available with platforms including the Oculus Rift and HTC Vive. It’s likely we’ll see VR beyond gaming become a growth area as we adapt to this new building block. For blockchain we expect slower progress in the next 6-18 months, but beneath the hype distributed trust and indelible ledgers are a fundamental new capability on which we can build businesses, economies and even society.

It’s clearly an exciting time to work with technology. Over the next six months we expect to see these trends continue to bring us increased productivity, empowered teams and businesses, and entirely new ideas and models that weren’t previously possible. We’d like to hear from you, too: tell us below what you think are the most important industry trends, and the things we’ll see happening in the future.

Many thanks to the following people for their contributions, comments, suggestions and corrections: Ian Cartwright, Erik Dörnenburg, Suzi Edwards-Alexander, Neal Ford, and Jonny LeRoy.