Azure Container Service Engine (acs-engine) is an Azure Resource Manager (ARM) template generator. The required configurations of the cluster are defined in a JSON file; acs-engine reads these cluster definitions and generates a number of files that can be consumed by ARM. The tool also provides flexibility to choose different orchestrators—including Kubernetes, DC/OS, OpenShift, Swarm mode and Swarm—and to configure features and agents of the cluster. We’ve been using acs-engine in a number of projects and would recommend it for managing clusters in Azure Container Service.
We're seeing significant advances in security tooling integration with modern software delivery processes. Archery is an open source tool with an active community that's doing a good job of pulling together a collection of other tools, including Zap. Designed primarily for web applications, Archery makes it easy to integrate security tooling into your build and deployment systems. Its dashboards also let you track vulnerabilities as well as application and network scans.
ArchUnit is a Java testing library for checking architecture characteristics such as package and class dependencies, annotation verification and even layer consistency. We like that it runs as unit tests within your existing test setup, even though it supports only Java-based architectures. The ArchUnit test suite can be incorporated into a CI environment or a deployment pipeline, making it easier to implement fitness functions in an evolutionary architecture way.
Running end-to-end tests can present challenges, such as the long duration of the running process, the flakiness of some tests and the challenges of fixing failures in CI when running tests in headless mode. Our teams have had very good experiences with Cypress by solving common issues such as lack of performance and long wait time for responses and resources to load. Cypress is a useful tool that helps developers build end-to-end tests and records all test steps as a video in an MP4 file to make it easier to identify errors.
Security continues to be paramount, and inadvertently checking credentials and other secrets into source control is a major attack vector. git-secrets is a simple tool that prevents you from committing passwords and other sensitive information to a git repository. It can also scan all historical revisions before making a repository public, if you want to ensure you've never accidentally checked in a credential. git-secrets comes with built-in support for common AWS keys and credentials and can be set up quickly for other providers too.
One of the challenges of using cloud services is being able to develop and test locally using those services. LocalStack solves this problem for AWS by providing local test double implementations of a wide range of AWS services, including S3, Kinesis, DynamoDB and Lambda. It builds on top of existing best-of-breed tools such as Kinesalite, Dynalite and Moto and adds isolated processes and error injection functionality. LocalStack is very easy to use and ships with a simple JUnit runner and a JUnit 5 extension. We're using it in a few of our projects and have been impressed with it.
Mermaid lets you generate diagrams from a markdown-like markup language. Born out of need to simplify documentation, Mermaid has grown into a larger ecosystem with plugins for Confluence, Visual Studio Code and Jekyll to name a few. To see how it works, you can use the Live Editor on GitHub. Mermaid also has a convenient command line interface that lets you generate SVG, PNG and PDF files as output from definition files. We've been using Mermaid in many projects and we like the simplicity of describing graphs and flowcharts with markdown and checking in the definition files with the code repository.
We've covered Visual Studio Code in the Radar since 2015, but it isn't the only cross-platform .NET Core IDE kid on the block anymore. Recently, Rider, which is part of the IDEA platform developed by JetBrains, has gained adoption, especially by those used to the speed and dexterity provided by ReSharper, which drives the refactoring in Rider. Rider, however, does more than ReSharper to bring the full IDEA platform to .NET and increase developer productivity. Regardless of your preferred platform, it's worth exploring Rider as it currently has the productivity edge on Visual Studio Code. It's also great to see the ecosystem alive and well, as competition ensures these tools continue to improve.
Snyk helps you find, fix and monitor known vulnerabilities in npm, Ruby, Python, Scala, Golang, .NET, PHP, Java and Docker dependency trees. When added to your build pipeline, Snyk continuously monitors and tests the library dependency tree against a hosted vulnerability database and suggests the minimal direct dependency version upgrade needed for remediation.
As more and more teams embrace DesignOps, practices and tooling in this space mature, too. Many of our teams now work with what could be called UI dev environments, which provide a comprehensive environment for quickly iterating on UI components, focusing on collaboration between user experience designers and developers. We now have a few options in this space: Storybook, react-styleguidist, Compositor and MDX. You can use these tools standalone in component library or design system development as well as embedded in a web application project. Rather than spinning up the app, plus a BFF, plus services simply to add a feature to a component, you can start up the Storybook dev server instead.
Visual Studio Code is Microsoft's free IDE editor, available across platforms. We've had good experience using this for front-end development using React and TypeScript, and back-end languages such as GoLang, without having to switch between different editors. The tooling, language support and extensions for Visual Studio Code continue to soar and get better. We'd particularly like to call out VS Live Share for real-time collaboration and remote pairing. While complex projects in statically typed languages, such as Java, .NET or C++, will likely find better support from the more mature IDEs from Microsoft or Jetbrains, we find that Visual Studio Code is increasingly becoming a tool of choice among infrastructure and front-end development teams.
VS Live Share is a suite of extensions for Visual Studio Code and Visual Studio. The real-time collaboration for editing and debugging of code, voice calls, sharing a terminal and exposing local ports have reduced some of the obstacles we'd otherwise encounter when pairing remotely. In particular, we like that Live Share allows developers to collaborate with each other, while continuing to use their preconfigured editor, which includes themes, key maps and extensions.
Building, testing and deploying mobile applications entails a number of complex steps, especially when we consider a pipeline from source code repositories to app stores. All these steps can be automated with scripts and build pipelines in generic CI/CD tools. However, for teams that focus on mobile development, and have little or no requirement to integrate with build pipelines for back-end systems, a domain-specific tool can reduce the complexity and maintenance overhead. Bitrise is easy to set up and provides a comprehensive set of prebuilt steps for most mobile development needs.
Codefresh is a hosted CI server similar to CircleCI or Buildkite. It's container-centric, making Dockerfiles and container-hosting clusters first-class entities. We like that the tool encourages a pipelined delivery approach and supports branching and merging. Early reports from our teams are positive, but we've yet to see how it works for larger projects and complex pipelines.
We're continually on the lookout for tools and techniques that allow delivery teams to work independently from the rest of a larger organization while staying within its security and risk guardrails. Grafeas is such a tool. It lets organizations publish authoritative metadata about software artifacts—Docker images, libraries, packages—that is then accessible from build scripts or other automated compliance controls. The access control mechanisms allow for a separation of responsibility between the teams that publish approvals or vulnerabilities and the teams that build and deploy software. Although several organizations, including Google and JFrog, use Grafeas in their workflows, note that the tool is still in alpha.
Heptio Ark is a tool for managing disaster recovery for Kubernetes clusters and persistent volumes. Ark is easy to use and configure and lets you back up and restore your clusters through a series of checkpoints. With Ark you can significantly reduce recovery time in case of an infrastructure failure, easily migrate Kubernetes resources from one cluster to another and replicate the production environment for testing and troubleshooting. Ark supports key backup storage providers (including AWS, Azure and Google Cloud) and, as of version 0.6.0, a plugin system that adds compatibility for additional backup and volume storage platforms. Managed Kubernetes environments, such as GKE, provide these services out of the box. However, if you're operating Kubernetes either on premise or in the cloud, take a closer look at Heptio Ark for disaster recovery.
Jaeger is an open source distributed tracing system. Similar to Zipkin, it's been inspired by the Google Dapper paper and complies with OpenTracing. Jaeger is a younger open source project than Zipkin, but it's gained popularity quickly due to a larger number of supported languages for the client libraries and easy installation on Kubernetes. We've used Jaeger successfully with Istio, integrating application traces with Envoy on Kubernetes, and like its UI. With Jaeger joining CNCF, we anticipate a larger community engagement effort and deeper integration with other CNCF projects.
kube-bench is an example of an infrastructure configuration scanner that automates checking your Kubernetes configuration against the CIS benchmark for K8s. It covers user authentication, permissions and secure data among other areas. Our teams have found kube-bench valuable in the identification of vulnerable configurations.
Ocelot is a .NET API gateway. After three years of development, Ocelot has built a relatively complete feature set and an active community. Although there is no dearth of excellent API gateways (e.g., Kong), the .NET community appears to prefer Ocelot when building microservices. Part of the reason is that Ocelot integrates well with the .NET ecosystem (e.g., with IdentityServer). Another reason may be that the .NET community has extended Ocelot to support communication protocols such as gRPC, Orleans and WebSocket.
UX research demands data collection and analysis to make better decisions about the products we need to build. Optimal Workshop is a suite of tools that helps to do this digitally. Features such as first-click or card sorting help to both validate prototypes and improve website navigation and information display. For distributed teams, in particular, benefit from Optimal Workshop as it lets them conduct remote research.
We have more and more projects that require unstructured data processing. To extract meaningful business information from text data is a key technique. Stanford CoreNLP is a Java-based set of natural language processing tools. It supports named-entity recognition, relationship extraction, sentiment analysis and text classification as well as multiple languages, including English, Chinese and Arabic. We also find tools usable to label corpus and training models for our scenario. With Stanford CoreNLP, we were able to use the latest research in the field of NLP to solve various business problems.
We widely use Terraform as code to configure a cloud infrastructure. Terragrunt is a thin wrapper for Terraform that implements the practices advocated by the Terraform: Up and Running book. We've found Terragrunt helpful as it encourages versioned modules and reusability for different environments with some handy features, including recursive code execution in subdirectories. We'd like to see the tool evolve to support CD practices natively, where all code can be packaged, versioned and reused across different environments on CD pipelines. Our team achieves this today with workarounds.
Traefik is an open-source reverse proxy and load balancer. If you're looking for an edge proxy that provides simple routing without all the features of NGINX and HAProxy, Traefik is a good choice. The router provides a reload-less reconfiguration, metrics, monitoring and circuit breakers that are essential when running microservices. It also integrates nicely with Let's Encrypt to provide SSL termination. When compared to Traefik, tools such as NGINX and HAProxy may require additional tooling to templatize configuration in response to scaling, adding or removing microservices and may, at times, require a restart which can be annoying in production environments.