We've featured Appium in the Radar in the past. It's one of the most popular mobile test automation frameworks. As we scale our test suite, being able to run our tests in parallel against an array of devices is key in having short feedback loops. Appium Test Distribution solves this problem very effectively with its ability to run tests in parallel as well as run the same tests on multiple devices. Among other things, it distinguishes itself by its ability to add and remove devices in which tests run without any manual setup required and with its ability to run tests on remote devices. We've used it in a few projects at ThoughtWorks over the last couple of years and it worked very well for us.
We've been enjoying BackstopJS for visual regression testing of web applications. The configurable viewports and ability to adjust tolerances are particularly useful, as is the visual comparison tool, which makes it easier to spot minor variations. It has good scriptability and the option to run in Headless Chrome, PhantomJS and SlimerJS. We find it particularly helpful when running it against living component style guides.
Our teams very much like the hosted CI/CD tool Buildkite for its simplicity and quick setup. With Buildkite, you provide your own machines to execute builds — on premise or in the cloud — and install a lightweight agent application to connect the build agent to the hosted service. In many cases, having this level of control over the configuration of your build agents is a plus when compared to using hosted agents.
CircleCI is a continuous integration engine offered as SaaS and on premise. CircleCI has been the go-to SaaS CI tool for many of our development teams, who needed a low-friction and easy-to-setup build and deployment pipeline. CircleCI version 2.0 supports workflows of build jobs, with fan-in and fan-out flows and manual gates, as well as mobile development. It allows developers to run the pipelines locally and easily integrates with Slack and other notification and alerting systems. We recommend you take a closer look at the security practices of CircleCI, just as you would with any other SaaS product that hosts your company’s assets.
It’s surprising how many problems can be expressed as mathematical optimization problems and often convex problems that can be efficiently solved. CVXPY is an open source Python-embedded modeling language for convex optimization problems. It’s maintained by academics at Stanford University and offers a batteries-included install for several open source and commercial solvers. The documentation includes many examples which should inspire developers to use it. It’s particularly useful for prototyping solutions even though commercially licensed solvers, such Gurobi or IBM CPLEX, may be required. In most cases though, it suffices by itself. However, the same group has written many extension packages such as DCCP and related software such as CVXOPT based on recent advances in optimization.
gopass is a password management solution for teams, built on GPG and Git. It's a descendant of pass and adds features such as: support for recipient management and multiple password stores in a single tree; an interactive search functionality; time-based one-time password (TOTP) support; and storage of binary data. Migration of your pass store is fairly straightforward, because gopass is largely compatible with the format pass uses. This also means integration into provisioning workflows can be achieved with a single call to a stored secret.
Helm is a package manager for Kubernetes. The set of Kubernetes resources that together define an application is packaged as charts. These charts can describe a single resource, such as a Redis pod, or a full stack of a web application: HTTP servers, databases and caches. Helm, by default, comes with a repository of curated Kubernetes applications that are maintained in the official charts repository. It’s also easy to set up a private chart repository for internal usage. Helm has two components: a command line utility called Helm and a cluster component called Tiller. Securing a Kubernetes cluster is a wide and nuanced topic, but we highly recommend setting up Tiller in a role-based access control (RBAC) environment. We’ve used Helm in a number of client projects and it’s dependency management, templating and hook mechanism has greatly simplified the application lifecycle management in Kubernetes.
Over the last couple of years, we've noticed a steady rise in the popularity of analytics notebooks. These are Mathematica-inspired applications that combine text, visualization and code in a living, computational document. Increased interest in machine learning — along with the emergence of Python as the programming language of choice for practitioners in this field — has focused particular attention on Python notebooks, of which Jupyter seems to be gaining the most traction among ThoughtWorks teams. People seem to keep finding creative uses for Jupyter beyond a simple analytics tool. For example, see Jupyter for automated testing.
Kong is an open source API gateway which also comes as an enterprise product integrating with proprietary API analytics and a developer portal. Kong can be deployed, in a variety of configurations, as an edge API gateway, as an internal API proxy, or even as a sidecar in a service mesh configuration. OpenResty, through its Nginx modules, provides a strong and performant foundation, with Lua plugins for extensions. Kong can either use PostgreSQL for single-region deployments or Cassandra for multiregion configurations. Our developers have enjoyed Kong's high performance, its API-first approach (which enables automation of its configuration) and its ease of deployment as a container. Kong API Gateway, unlike overambitious API gateways, has a smaller set of features but it implements the essential set of API gateway capabilities such as traffic control, security, logging, monitoring and authentication.
kops is a command line tool for creating and managing high-availability production Kubernetes clusters. kops has become our go-to tool to self-manage Kubernetes clusters on AWS, not the least because of its rapidly growing open source community. It also supports installing, upgrading and managing Kubernetes clusters on Google Cloud. Our experience with kops on Google, however, is very limited because of our preference for GKE, the managed Kubernetes offering. We recommend using kops in reusable scripts to create infrastructure as code. We're interested to see how kops continues to evolve to support managed Kubernetes clusters such as EKS, Amazon's own managed Kubernetes service.
Patroni is a template for PostgreSQL high availability. Born out of the need to provide automatic failure for PostgreSQL, Patroni is a Python-based PostgreSQL controller that leverages a distributed configuration store (such as etcd, ZooKeeper, or Consul) to manage the state of the PostgreSQL cluster. Patroni supports both streaming and synchronous replication models and provides a rich set of REST APIs for dynamic configuration of the PostgreSQL cluster. If you want to achieve high availability in a distributed PostgreSQL setup, you have to consider many edge cases, and we like the fact that Patroni provides a template to achieve most of the common use cases.
A key driver for architectures based on microservices is independent evolvability of services. For example, when two services depend on each other, the testing process for one usually involves stubs and mocks for the other one. These can be written by hand, but as with mocking in unit tests, a framework helps developers focus on the actual test scenario. We have known of WireMock for a while but we’ve preferred running tests with mountebank. Over the past year, though, WireMock has really caught up and we now recommend it as a good alternative.
Apex is a tool to build, deploy and manage AWS Lambda functions with ease. With Apex, you can write functions in languages that are not yet natively supported in AWS, including Golang, Rust and others. This is made possible by a Node.js shim, which creates a child process and processes events through stdin and stdout. Apex has a lot of nice features that improve the developer experience, and we particularly like the ability to test functions locally and perform a dry run of the changes before they're applied to AWS resources.
ArchUnit is a Java testing library for checking architecture characteristics such as package and class dependencies, annotation verification and even layer consistency. The fact that it runs as unit tests, within your existing test setup, pleases us, even though it's available for Java architectures only. The ArchUnit test suite can be incorporated into a CI environment or a deployment pipeline, making it easier to implement fitness functions in an evolutionary architecture way.
The cloud and continuous delivery had a dramatic effect on infrastructure security. When following infrastructure as code, the entire infrastructure — which includes networks, firewalls and accounts — is defined in scripts and configuration files, and with Phoenix Servers and Environments, the infrastructure is recreated in each deployment, often many times a day. In such a scenario, testing the infrastructure after it's created is neither sufficient nor feasible. A tool that helps address this problem is cfn_nag. It scans the CloudFormation templates used with AWS for patterns that may indicate insecure infrastructure, and it does so before the infrastructure is created. Running a tool such as cfn_nag in a build pipeline is fast and it can detect a number of problems before they even reach a cloud environment.
Conduit is a lightweight service mesh for Kubernetes. Conduit embraces the out-of-process architecture with data plane proxy written in Rust and a control plane in Go. The data plane proxy runs as a sidecar for all TCP traffic in the Kubernetes cluster and the control plane runs in a separate namespace in Kubernetes exposing REST APIs to control the behavior of the data plane proxy. By proxying all requests, Conduit provides a wealth of metrics for monitoring and observability of interactions in the service mesh for HTTP, HTTP/2 and gRPC traffic. Even though Conduit is relatively new to this space, we recommend it because it’s simple to install and operate.
Fixing end-to-end test failures in CI can be a painful experience, especially in headless mode. Cypress is a useful tool that helps developers build end-to-end tests easily and records all test steps as a video in an MP4 file. Instead of reproducing the issue in headless mode, developers can watch the testing video in order to fix it. Cypress is a powerful platform, not only a testing framework. Currently, we've integrated its CLI with headless CI in our projects.
When developing front-end applications, we've mentioned Headless Chrome as a better alternative to PhantomJS for front-end testing in a previous edition of the Radar. Now we suggest assessing Headless Firefox as a viable option in this area. In the same way as Headless Chrome, Firefox in a headless mode runs the browser without the visible UI components, executing the UI tests suite much faster.
nsp is a command line tool to identify known vulnerabilities in Node.js applications. By running the check command on the root of a Node.js project, nsp generates the vulnerabilities report by checking against the published advisories. nsp provides a way to customize the check command to hide all vulnerabilities below the given CVSS score or exit with an error code if at least one finding has a CVSS score above the given value. Once the advisories are saved through the gather command, nsp can also be used in offline mode.
Parcel is a web application bundler similar to Webpack or Browserify. We’ve featured Webpack previously in our Radar and it continues to be a great tool. Parcel distinguishes itself from its rivals through developer experience and speed. It has all the standard bundling features and provides true zero-configuration experience, making it really easy to get started with and use. It has fast bundle times and beats its competitors in many benchmarks. Parcel has gained a lot of community interest and is worth keeping an eye on.
Scout2 is a security auditing tool for AWS environments. Instead of manually navigating through web pages, you can rely on Scout2 to fetch all the configuration data of an AWS environment for you; it even generates an attack surface report. Scout2 ships with preconfigured rules and can be easily extended to support more services and test cases. Since Scout2 only performs AWS API calls to fetch configuration data and identify security gaps, it is not necessary to complete and submit the AWS Vulnerability / Penetration Testing Request Form.
Sentry is an error-tracking tool that helps monitor and fix errors in real time. Error tracking and management tools such as Sentry distinguish themselves from traditional logging solutions such as the ELK Stack in their focus on discovering, investigating and fixing errors. Sentry has been around for some time and is quite popular — error-tracking tools are increasingly useful with the current focus on "mean time to recovery". Sentry — with its integration options with Github, Hipchat, Heroku, Slack, among other platforms — enables us to keep a close eye on our apps. It can provide error notifications following a release, enable us to track whether new commits actually fix the issue and alert us if an issue comes back due to a regression.
Sonobuoy is a diagnostic tool for running end-to-end conformance tests on any Kubernetes cluster in a nondestructive way. The team at Heptio, which was founded by two creators of the Kubernetes projects, built this tool to ensure that the wide array of Kubernetes distributions and configurations conform to the best practices, while following the open source standardization for interoperability of clusters. We're experimenting with Sonobuoy to run as part of our infrastructure as code build pipeline, as well as continuous monitoring of our Kubernetes installations, to validate the behavior and health of the whole cluster.
In the current state of technology services, exposing RESTFul APIs is increasingly adopted and API documentation is very important for consumers. In this space, Swagger has been largely used across teams and we would like to highlight Swashbuckle for .NET Core. Swashbuckle for .NET Core is a tool that generates living API documentation in Swagger, based on the code for .NET Core projects. When using it, you can also explore and test operations of APIs through its UI.