Themes for this edition
We created this Radar entirely remotely, which we never thought was possible. Has work (as we know it) forever changed?
Increasingly we see almost all aspects of software development and delivery managed as code. But while software might be eating the world, practitioners too frequently overlook proven engineering practices.
As the industry thinks more diligently about how data should work in modern systems, we’re encouraged to see the growing maturity in the techniques and tools surrounding analytical data and machine learning.
As Kubernetes continues to dominate, we see a rich ecosystem growing and expanding to take advantage of its strengths and address its weaknesses.
The Elephant in the Zoom
"Necessity is the mother of invention" – proverb
Many companies have experimented with the idea of remote working as the technology to enable it has slowly matured. But suddenly, a global pandemic has forced companies all over the world to rapidly and fundamentally change their way of working to preserve some productivity. As many have observed, "working from home" is starkly different from "being forced to work from home during a pandemic", and we think there will be a journey ahead to become fully productive in this new context.
We've never believed that creating a Radar remotely was possible, and yet here we are — this is the first Radar we've ever produced without meeting in person. Many of the proposed blips spoke to the pressing need to enable first-class remote collaboration. We didn't want to ignore the elephant in the room and not comment on the crisis, but doing a good job of remote-first collaboration is a deep and nuanced subject and certainly not all of our advice would fit in the Radar format. So alongside this edition you'll find a podcast where we discuss our experiences in creating the Radar remote-first, a written experience report including advice on remote-first productivity, a webinar covering tech strategies in a crisis and links to other ThoughtWorks materials, including our remote working playbook. We hope that these materials, together with other internet resources, will help organizations that attempt to navigate these unknown waters.
X is Software Too
We often encourage other parts of the software delivery ecosystem to adopt beneficial engineering practices pioneered by agile software development teams; we return to this topic so often because we keep finding niches where we see slow progress on this advice. For this Radar, we decided to call out again infrastructure as code as well as pipelines as code, and we also had a number of conversations about infrastructure configurations, ML pipelines and other related areas. We find that the teams who commonly own these areas do not embrace enduring engineering practices such as applying software design principles, automation, continuous integration, testing, and so on. We understand that many factors hamper fast movement for some engineering practices: complexity (both essential and accidental), lack of knowledge, political impediments, lack of suitable tooling and many others. However, the benefits to organizations that embrace agile software delivery practices are clear and worth some effort to achieve.
Data Perspectives Maturing and Expanding
A theme that spanned many blips and quadrants in this edition concerned maturity in data, particularly techniques and tools surrounding analytical data and machine learning. We note many continuing innovations in the natural language processing (NLP) space. We also welcome both the emergence and continuing maturity of full-lifecycle machine learning tool suites, combining enduring engineering practices with combinations of tools that work well in an iterative manner, showing that "machine learning is software too." Finally, for distributed architectures such as microservices, we see great interest in data mesh as a way to effectively serve and use analytical data at scale in distributed systems. As the industry thinks more diligently about how data should work in modern systems, we're encouraged by the general direction and opening perspectives in this arena and expect to see exciting innovations in the near future.
Kubernetes & Co. Cambrian Explosion
As Kubernetes continues to consolidate its market dominance, the inevitable supporting ecosystem thrives. We discussed a number of blips surrounding Kubernetes in the tools, platforms and techniques quadrants, showing just how pervasive this subject has become. For example, Lens and k9s simplify cluster management, kind helps with local testing and Gloo offers an alternative API Gateway. Hydra is an OAuth server optimized to run on Kubernetes, and Argo CD uses Kuberenetes’ native desired-state management to implement a CD server. These developments indicate Kubernetes is perfectly poised to create a supporting ecosystem; it offers critical capabilities but with abstractions that are often too low level or advanced for most users. Thus, the complexity void fills with tooling to either ease the configuration and the use of Kubernetes or supply something missing from the core functionality. As Kubernetes continues to dominate, we see a rich ecosystem growing and expanding to take advantage of its strengths and address its weaknesses. As this ecosystem matures, we expect it to evolve toward a new set of higher-level abstractions offering the benefits of Kubernetes without the bewildering range of options.
Contributors
The Technology Radar is prepared by the ThoughtWorks Technology Advisory Board, comprised of:
Rebecca Parsons (CTO)| Martin Fowler (Chief Scientist)| Bharani Subramaniam| Birgitta Böckeler| Camilla Falconi Crispim| Erik Doernenburg| Evan Bottcher| Fausto de la Torre| Hao Xu| Ian Cartwright| James Lewis| Jonny LeRoy| Lakshminarasimhan Sudarshan| Mike Mason| Neal Ford| Ni Wang| Rachel Laycock| Scott Shaw| Shangqi Liu| Zhamak Dehghani|