Tools
Adopt
-
Conan is an open-source dependency management tool for C/C++ applications. It provides an intuitive interface for defining, fetching and managing dependencies which makes it easy for developers to integrate third-party libraries into their projects. Conan works across all major operating systems and can target a variety of platforms, including servers and desktop, mobile and embedded devices. It can also be used for building and publishing C/C++ libraries and packages. The packages can be shared across teams via JFrog Artifactory servers. By leveraging prebuilt binaries, it significantly reduces build times, especially for hefty dependencies. Conan integrates with popular build systems like CMake and also has a Python SDK for extending the build system for tasks like signing. In our experience, Conan translates to improved build reproducibility across environments and faster development cycles. The resulting codebases are cleaner and easier to maintain, a major win for large-scale projects. If you're wrestling with dependency management in your C or C++ projects, Conan is a must-consider tool to boost your development efficiency.
-
We added Kaniko to the Radar in October 2022, shortly after Kubernetes moved on from supporting Docker, highlighting at the time the trend away from Docker as the default to build container images within container-based pipelines. Since then, we've expanded our experience with Kaniko across different pipelines' tooling and configurations. Our teams appreciate its flexibility and performance which is why we’re moving it to Adopt, highlighting Kaniko as the default tool in this space.
-
One of the fundamental capabilities of Kubernetes is horizontal autoscaling: its ability to launch new pods when additional capacity is needed and shut them down when loads decrease. However, this only works if the nodes needed to host the pods already exist. Cluster Autoscaler can do some rudimentary cluster expansion triggered by pod failures, but it has limited flexibility; Karpenter, however, is a smarter, open-source Kubernetes Operator node autoscaler: it analyzes current workloads and pod scheduling constraints, selects an appropriate instance type and then starts or stops it as needed. Karpenter is an operator in the spirit of tools such as Crossplane that can provision cloud resources outside the cluster. Even though Karpenter was originally developed by AWS for EKS, it’s becoming the default node autoprovisioner across all cloud Kubernetes service providers, and Azure recently started supporting Karpenter with AKS Karpenter Provider.
Trial
-
42Crunch API Conformance Scan is a dynamic testing tool designed to identify discrepancies between your API's documented behavior and its actual implementation. This tool takes your API's spec definition in OpenAPI format, which outlines the expected functionalities and responses, and compares it to the API's actual behavior. By generating real traffic and interacting with live endpoints, the tool can identify any discrepancies between what the API promises and what it delivers. This translates into several benefits for development teams. For example, it catches inconsistencies early in development, saving time and preventing issues from reaching production. The tool also helps improve API quality and security by identifying potential vulnerabilities arising from deviations from the documented behavior. Overall, API Scan helps you assess the security posture of your APIs by identifying problems such as weak authentication protocols, insecure data handling practices and insufficient input validation. It provides detailed reports highlighting any issues found, along with recommendations for remediation.
-
actions-runner-controller is a Kubernetes controller that operates self-hosted runners for GitHub Actions. Self-hosted runners are helpful in scenarios where the job that GitHub Actions runs needs to access resources that are either not accessible to GitHub cloud runners or have specific operating system and environmental requirements that are different from what GitHub provides. In such scenarios where the team uses Kubernetes clusters, actions-runner-controller orchestrates and scales these runners. Our teams like its ability to scale runners based on the number of workflows running in a given repository, organization, enterprise or Kubernetes cluster, as well as its ability to handle both Linux and Windows runners.
-
Android Emulator Containers streamline Android app testing by eliminating the complexities arising from OS compatibility issues and system dependencies as well as from setting up emulators for multiple Android versions. Traditionally, this complexity led to extra effort or teams foregoing automated testing completely, which, in turn, resulted in slower development and testing cycles. Android Emulator Containers simplify this process, allowing seamless integration into CI pipelines for automated testing. Our teams utilize these containers primarily for instrumented tests, which are automatically executed with each commit to provide instantaneous feedback to developers. Additionally, we leverage Android Emulator Containers for running nightly end-to-end tests as well.
-
Our advice has always been to monitor costs as a fitness function. Cloud providers offer a variety of services to monitor cloud spend such as AWS Cost Explorer or Google Cloud FinOps Hub. In the AWS ecosystem, our teams use the CUDOS (Cost and Usage Dashboards Operations Solution) dashboards to monitor AWS Marketplace spend segregated by the business departments or legal entities in a large parent organization. This dashboard provides comprehensive cost and usage details, with resource-level granularity that helps optimize costs, track usage goals and achieve operational excellence.
-
aws-nuke is an open-source tool that tackles the common challenge of unused resources accumulating in development and sandbox AWS accounts that can lead to cost inefficiencies. The tool identifies and removes all deletable resources within an AWS account or region with the exception of default or AWS-managed resources, essentially resetting the environment to a Day One state. It also offers customizable exclusion policies to ensure critical resources remain protected. We’ve used this tool for the default use case of cost optimization as well as for disaster recovery (DR) contexts with good results. By automating cleanup in development and sandbox environments, aws-nuke helps minimize unnecessary resource expenditure. It also facilitates efficient teardown of temporary DR infrastructure after drills or exercises. Although stable, aws-nuke is a very destructive tool and is not intended to be used in production environments. Always perform a dry run to confirm that essential resources won’t be deleted.
-
Bruno is an open-source desktop alternative to Postman and Insomnia for API testing, development and debugging. It stores your collections locally on the filesystem so you can use Git or a version control of your choice to collaborate. Several Thoughtworks teams are using Bruno and like its simple offline-only design.
-
Develocity (previously Gradle Enterprise) addresses the pain point of lengthy build and test cycles in large-scale software projects. It employs performance improvements such as build caching and predictive test selection to shorten developer feedback loops in both local and CI/CD environments. Our platform teams have found it useful for speeding up builds and tests, analyzing commands to determine what part of the workflow still needs to be optimized, identifying and troubleshooting flaky tests and performing analysis on the hardware used to run them.
-
While the AI coding assistance market is getting busier and busier, GitHub Copilot remains our default choice and is used by many of our teams. Since we last wrote about GitHub Copilot, the most interesting improvements came in the chat feature. For instance, it's no longer necessary to clutter the code with comments as prompts; instead, an inline chat helps you prompt without writing a comment. The inline chat can also change code, not just write new lines. You can now also significantly expand the context of the chat when asking questions about your code by using the
@workspacetag. This allows you to ask questions about the entire codebase, not just the open files. You can expand this context even further with the Copilot Enterprise version, which pulls in context from all repositories you host on GitHub. Finally, GitHub has started routing some chat requests to a more powerful GPT-4–based model, and availability of the chat in the popular Jetbrains IDEs is imminent (although still in private beta at the time of writing). These releases show that the pace of improvements in the space has not slowed down. If you tried a coding assistant last year and dismissed it, we recommend that you keep monitoring the features being released and give it another try. -
Gradio is an open-source Python library that facilitates the creation of interactive web-based interfaces for machine learning (ML) models. A graphical user interface on top of ML models provides a better understanding of the inputs, constraints and outputs by nontechnical audiences. Gradio has gained a lot of traction in the generative AI space, as it is one of the tools that makes generative models so accessible to experiment with. Usually, we put technologies into the Trial ring when we’ve seen them used in production at least once. Gradio's purpose and strength are experimentation and prototyping, and we’ve used it for that purpose many times. Recently, one of our teams even used it to help a client with live demonstrations at big events. We’re very happy with Gradio's capabilities for those use cases, and therefore move it into the Trial ring.
-
Gradle version catalog is a useful feature of the Gradle build tool that allows you to manage dependencies centrally in the build file. Our teams have found it especially useful with Android multi-module projects. Instead of hardcoding dependency names and versions in individual build files and managing upgrades, you can create a central version catalog of these dependencies and then reference it in a type-safe way with Android Studio assistance.
-
Maestro is extremely useful when testing complex flows in mobile applications. Our teams have found it easy to learn, easy to understand and easy to integrate into our development workflow. Maestro supports a range of mobile platforms including iOS, Android, React Native and Flutter apps. Its declarative YAML syntax simplifies the automation of complex mobile UI interactions. Based on the tool's evolution, marked by enhanced features like comprehensive iOS support and the introduction of tools like Maestro Studio and Maestro Cloud, we encourage anyone seeking to optimize their mobile application testing processes to give it a try.
-
Microsoft SBOM tool is an open-source tool to generate SPDX-compatible Software Bill of Materials (SBOM). We have blipped about the need for SBOM previously, and this tool makes it easier to get started. SBOM tool supports a variety of popular package managers (including npm, pip and Gradle), making it compatible with a wide range of projects. It’s very easy to use and can be integrated into existing development workflows, including integration with CI/CD pipelines. By leveraging SBOM generated with this tool, developers gain multiple advantages. Improved software security is a key benefit, as a clear view of components allows for easier vulnerability identification and risk management. License compliance is also enhanced, as developers can ensure adherence to all relevant agreements. Furthermore, SBOM promotes transparency within the software supply chain, aiding dependency tracking and mitigating potential risks. If you're looking to streamline SBOM generation, improve software security and gain control over your software supply chain, you should give Microsoft SBOM tool a try.
-
Open Policy Agent (OPA) is a uniform framework and language for declaring, enforcing and controlling policies. For our teams, it has become a favored way of defining policies for distributed systems, particularly where we need to implement compliance at the point of change. OPA allows teams to implement various platform engineering patterns, such as controlling what is deployed to Kubernetes clusters, enforcing access control across services in a service mesh and implementing fine-grained security policy as code for accessing application resources. While there is some complexity associated with OPA implementations, it has proven to be a highly valuable tool for ensuring compliance in a DevOps culture. We’re also continuing to keep an eye on the extension and maturity of OPA beyond operational systems to (big) data-centric solutions.
-
While GitHub Actions runners cover a wide range of the most common run times and are quickest to start with, teams sometimes need to manage self-hosted runners, such as when organizational policy only allows deployments to a privately hosted infrastructure from within the organization's own security perimeter. In such cases, teams can use Philips's self-hosted GitHub runner, a Terraform module that spins up custom runners on AWS EC2 spot instances. The module also creates a set of Lambdas that handles lifecycle management (scaling up and down) for these runners. In our experience, this tool greatly simplifies the provisioning and management of self-hosted GitHub Actions runners. An alternative for teams that use Kubernetes is actions-runner-controller.
-
Pair programming continues to be an essential technique for us, because it helps improve code quality and spread knowledge within a team. Although it’s best done in person, our distributed teams have explored tools to make remote pairing as pleasant and effective as possible, such as Tuple, Visual Studio Live Share, Code With Me and general-purpose chat and conferencing tools. The latest tool in the space that's caught our attention is Pop (formerly Screen). Coming from the founders of Screenhero, it supports multi-person screen sharing, annotations and high-quality audio/video. Some of our teams have used it extensively for pair programming and remote working sessions and report positively on their experience.
-
Automatically monitoring and updating dependencies as part of the software build process has become standard practice across the industry. It takes the guesswork out of staying current with security updates to open-source packages as they're released. For many years, Dependabot has been the standard tool for this practice, but Renovate has become the preferred tool for many of our teams. They find that Renovate is more suitable to the modern software development environment where a deployable system relies not just on code and libraries but encompasses run-time tools, infrastructure and third-party services. Renovate covers dependencies on these ancillary artifacts in addition to code. Our teams also found that Renovate offers more flexibility through configuration and customization options. Although Dependabot remains a safe default choice and is conveniently integrated with GitHub, we'd recommend evaluating Renovate to see if it can further reduce the manual burden on developers to keep their application ecosystems safe and secure.
-
Terrascan is a static code analyzer for infrastructure as code (IaC) designed to detect security vulnerabilities and compliance issues before provisioning cloud-native infrastructure. It supports scanning for Terraform, Kubernetes (JSON/YAML), Helm, AWS CloudFormation, Azure Resource Manager, Dockerfiles and GitHub. The default policy pack covers all the major cloud providers, GitHub, Docker and Kubernetes. Our teams use Terrascan locally as a pre-commit hook and integrate it into CI pipelines to detect IaC vulnerabilities and violations.
-
Velero is an open-source tool for backing up and restoring Kubernetes resources and persistent volumes. It simplifies disaster recovery and cluster migrations by enabling on-demand and scheduled backups. Velero also allows finer-grained controls over which resources get backed up as well as over the backup/restore workflow (with hooks). Our teams appreciate its ease of use and its reliance on Kubernetes APIs instead of lower-level layers like etcd.
Assess
-
aider is an open-source AI coding assistant. Like many open-source tools in this space, aider doesn’t have direct IDE integration but is started as a CLI in the terminal. aider is interesting because it provides a chat interface with write access to the codebase across multiple files, whereas many of the coding assistant products today either only read the code or can change only one file at a time. This allows aider to help you implement concepts that stretch over multiple files (e.g., "add locators to my HTML and also use those in my functional test") and to create new files and folder structures in the codebase (e.g., "create a new component similar to the one in folder X"). As aider is open source and not a hosted product, you have to bring your own OpenAI or Azure OpenAI API key to use it. On the one hand, this can be great for occasional use because you only have to pay per use. On the other hand, aider does seem to be quite chatty in its interactions with the AI API, so keep an eye on request costs and rate limits when using it.
-
Akvorado is an open-source network monitoring and analysis tool. It captures the network flows, Netflow/IPFIX and sFlow, enriches them with interface names and geo information and then saves the updated flows in ClickHouse for future analysis. Although OpenTelemetry is gaining adoption for observing application-level traffic, we often come across challenges in the network layer that can be difficult to spot and troubleshoot. Tools like Akvorado are quite handy in such situations as they help you analyze the network flows across various devices in the network topology.
-
Baichuan 2 is part of a new generation of open-source large language models. It was trained on a high-quality corpus with 2.6 trillion tokens, achieving quite good performance for its size on Chinese, English and multi-language benchmarks. Baichuan has been trained on several domain-specific corpora, including healthcare and law data sets, which is why we prefer using it in these and related fields.
-
The efficiency and performance of Rust make it a good fit for serverless computing. Another advantage is that Rust functions don’t require a run time, which results in fast startup times. However, the developer experience for writing the functions in Rust wasn’t great. That changed with Cargo Lambda. As a cargo subcommand, it integrates with the typical Rust workflow and allows you to run and test your AWS Lambda functions on the developer machine without needing Docker, VMs or other tools. Using a Zig toolchain, Cargo Lambda can cross-compile the functions on several operating systems for the Linux sandboxes used by AWS Lambda, and both Arm and Intel are supported as target architectures.
-
In the busy emerging space of AI coding assistants, some products, instead of competing with the strong incumbents, take a more focused approach. Codium AI is focused on test generation with AI. It works for all languages but has advanced support for common stacks, such as JavaScript and Python. We particularly like that the tool, rather than taking developers straight to the test code, offers a list of scenario descriptions in natural language for review. This makes it easier for developers to reason about the scenarios and decide which ones to turn into test code. To further improve the test generation for a particular codebase and use case, users can provide example tests and general instructions to enhance the information used by the AI to generate the tests.
-
Continue is an open-source autopilot for VS Code and JetBrains IDEs. We quite like it because it eliminates the pain of copying/pasting from a chat-based interface to large language models (LLMs) with a direct integration in the IDE. It supports several commercial and open-source models and makes it easy to try different LLM providers, including self-hosted LLMs. You can even run Continue without an internet connection.
-
One hallmark of widely used REST APIs is that their contracts are thoroughly documented. Developers are more likely to adopt and use APIs whose behavior and syntax are described accurately in a structured, organized way. Keeping this documentation up to date as the contract evolves can be time-consuming and is a task that is easily overlooked. Fern Docs helps with this by reducing the toil involved in writing and maintaining API documentation. Fern automatically generates a website with attractive, usable documentation from a specification file that can be versioned alongside the API code. While our initial impressions of this product are positive, Fern does require you to maintain API information in a proprietary configuration file. While it provides a way to convert OpenAPI specs into its own configuration format, we'd prefer a tool that generates docs directly from annotated source code.
-
Given how common multi-account strategy is in organizations' AWS environments, engineers frequently have to switch between multiple accounts within a short period of time. Granted, a command-line tool that simplifies the opening of multiple accounts in the browser simultaneously, streamlines account switching. It leverages each browser's native features to isolate multiple identities, Firefox's Multi-Account Containers and Chromium's Profiles. If a specific service (such as S3) is specified as an argument, Granted will open the service's landing page. Granted currently only supports AWS. Notably, it stores AWS SSO's temporary credentials safely in the keychain rather than as plain text on the disk.
-
LinearB is a platform designed to empower engineering leaders with data-driven insights for continuous improvement. It tackles three key areas: benchmarking, workflow automation and investment. Our experience with LinearB's metrics tooling highlights its potential to support a culture of continuous improvement. One of our teams leveraged the platform to track engineering metrics, identify and discuss improvement opportunities and define actionable steps based on data, leading to measurable progress. We're happy to see that this aligns with LinearB's core value proposition: benchmark, automate and improve. LinearB integrates with GitHub, GitLab, Bitbucket and Jira. It offers a comprehensive suite of preconfigured engineering metrics, with a strong focus on DORA metrics (deployment frequency, lead time, change failure rate and time to restore). As strong advocates of the four key metrics as defined by the DORA research, we appreciate LinearB's emphasis on measuring what truly matters for software delivery performance. Historically, gathering DORA-specific metrics has been a challenge. Teams have resorted to complex CD pipeline instrumentation, custom-built dashboards or rely on manual processes. Although our experience is limited to one team, LinearB seems to be a compelling alternative for gathering and tracking engineering metrics as well as fostering a data-driven approach to continuous improvement.
-
LLaVA (Large Language and Vision Assistant) is an open-source, large multimodal model that connects a vision encoder and LLM for general-purpose visual and language understanding. LLaVA's strong proficiency in instruction-following positions it as a highly competitive contender among multimodal AI models. The latest version, LLaVA-NeXT, allows for improved question answering. Among the open-source models for language and vision assistance, LLaVA is a promising option when compared to GPT-4 Vision. Our teams have been experimenting with it for visual question answering.
-
Marimo offers a fresh take on Python notebooks by prioritizing reproducibility and interactivity. It addresses challenges with hidden state in traditional notebooks (like Jupyter) which can lead to unexpected behavior and hinder reproducibility. It does that by storing notebooks as plain Python files with no hidden state and using a deterministic execution order based on dependencies (when a variable changes, all affected cells are automatically run). Marimo also comes with interactive UI elements that similarly propagate value changes to cells that depend on them. As it can be deployed as a web app, it’s also a useful tool for demos and prototyping purposes. Although we’re excited for the potential of Marimo, in particular in terms of reproducibility for data exploration and analysis purposes, we continue to caution against productionizing notebooks.
-
Mixtral is part of the family of open-weight large language models Mistral released, that utilizes the sparse Mixture of Experts architecture. The family of models are available both in raw pretrained and fine-tuned forms in 7B and 8x7B parameter sizes. Its sizes, open-weight nature, performance in benchmarks and context length of 32,000 tokens make it a compelling option for self-hosted LLMs. Note that these open-weight models are not tuned for safety out of the box, and users need to refine moderation based on their own use cases. We have experience with this family of models in developing Aalap, a fine-tuned Mistral 7B model trained on data related to specific Indian legal tasks, which has performed reasonably well on an affordable cost basis.
-
NeMo Guardrails is an easy-to-use open-source toolkit from NVIDIA that empowers developers to implement guardrails for large language models (LLMs) used in conversational applications. Although LLMs hold immense potential in building interactive experiences, their inherent limitations around factual accuracy, bias and potential misuse necessitate safeguards. Guardrails offer a promising approach to ensure responsible and trustworthy LLMs. Although you have a choice when it comes to LLM guardrails, our teams have found NeMo Guardrails particularly useful because it supports programmable rules and run-time integration and can be applied to existing LLM applications without extensive code modifications.
-
Ollama is an open-source tool for running and managing large language models (LLMs) on your local machine. Previously, we talked about the benefits of self-hosted LLMs, and we’re pleased to see the ecosystem mature with tools like Ollama. It supports several popular models — including LLaMA-2, CodeLLaMA, Falcon and Mistral — that you can download and run locally. Once downloaded, you can use the CLI, API or SDK to interact with the model and execute your tasks. We're evaluating Ollama and are seeing early success as it improves the developer experience in working with LLMs locally.
-
OpenTofu is a fork of Terraform made in response to a recent ambiguous license change by HashiCorp. It's open source and has been accepted by the Linux Foundation. It's backed by several organizations, including third-party vendors. The current version is compatible with the last open-source version of Terraform. Version 1.7 adds client-side encryption. The future of the OpenTofu project is unclear in terms of how closely it will support compatibility with future versions of Terraform. There are also questions around the long-term support by its current backers. We recommend keeping an eye on the project but remain cautious around usage, except for teams that have the capability to manage risks that may include being able to contribute to the codebase.
-
Large language models (LLMs) and retrieval-augmented generation (RAG) techniques have greatly improved our ability to synthesize and extract information. We’re seeing emerging tools taking advantage of this, and QAnything is one of them. QAnything is a knowledge management engine with a question-and-answer interface that can summarize and extract information from a wide range of file formats, including PDF, DOCX, PPTX, XLSX and MD files, among others. For data security concerns, QAnything also supports offline installation. Some of our teams use QAnything to build their team knowledge base. In GenAI scenarios with more industry depth (such as generating abstracts for investment reports), we also try to use this tool for proofs of concept before building real products and showing the potential of LLMs and RAG.
-
Little has emerged in recent years to challenge the dominance of Terraform as an infrastructure coding tool. Although alternatives such as Pulumi, CDK and, more recently, Wing have emerged, Terraform's modular, declarative paradigm has proven to be the most enduring. Indeed, all of these approaches share the common goal of modular code creating monolithic infrastructure. System Initiative is a new, experimental tool that represents a radical new direction for DevOps work. One way to view System Initiative is as a digital twin for your infrastructure. Interactive changes to the System Initiative state result in corresponding change sets that can be applied to the infrastructure itself. Likewise, changes to the infrastructure are reflected in the System Initiative state. One of the great advantages of this approach is the collaborative environment it creates for things like application deployment and observability. Engineers interact with System Initiative through a user interface that has a graphical representation of the entire environment. In addition to managing the cloud infrastructure, you can also use the tool to manage containers, scripts, tools and more. Although we're generally skeptical of these kinds of GUI tools, System Initiative can be extended to handle new assets or enforce policy via TypeScript code. We really like the creative thinking that has gone into this tool and hope it will encourage others to break with the status quo of infrastructure-as-code approaches. System Initiative is free and open source under an Apache 2.0 license and is currently in open beta. The maintainers themselves do not recommend the tool for production use yet, but we think it’s worth checking out in its current state to experience a completely different approach to DevOps tooling.
-
Tetragon is an open-source eBPF-based security observability and run-time enforcement tool. We mentioned Falco for detecting security threats a while back in the Radar. Tetragon goes beyond threat detection by leveraging eBPF to enforce security policies at run time in the Linux kernel. You can use Tetragon either as a standalone tool on bare metal or inside the Kubernetes environment.
-
We’re seeing a lot of movement in the infrastructure-as-code (IaC) space with tools like Winglang emerging. Winglang takes a different approach to defining infrastructure and run-time behavior. It provides high-level abstractions over platform specifics provided by tools such as CloudFormation, Terraform, Pulumi and Kubernetes. With Winglang, you write code that runs at compile time to generate infrastructure configuration and then code that executes at run time for application behavior. It provides a simulation mode to run locally and has an integrated test framework. We’re keeping an eye on this interesting tool; it’s a potential preview of the future direction of IaC.
Hold
- New
- Moved in/out
- No change
Unable to find something you expected to see?
Each edition of the Radar features blips reflecting what we came across during the previous six months. We might have covered what you are looking for on a previous Radar already. We sometimes cull things just because there are too many to talk about. A blip might also be missing because the Radar reflects our experience, it is not based on a comprehensive market analysis.
Unable to find something you expected to see?
Each edition of the Radar features blips reflecting what we came across during the previous six months. We might have covered what you are looking for on a previous Radar already. We sometimes cull things just because there are too many to talk about. A blip might also be missing because the Radar reflects our experience, it is not based on a comprehensive market analysis.
