Web application developers have it easy when it comes to simplifying and automating diverse application workflows; they can choose from a variety of solutions to help automate release processes. When developing for mobile, however, we're dealing with two operating systems with two different ways of building, testing, distribution, generating screenshots, signing and distributing applications. To help ease the pain, our teams have adopted fastlane as the go-to tool to automate the release process for iOS and Android applications. Using simple configurations and multiple pipelines, they can achieve continuous delivery for mobile development.
When combining modern techniques and architecture styles, such as microservices, DevOps and QA in production, development teams need increasingly sophisticated monitoring. Simply looking a graphs of disk usage and CPU utilization is not sufficient anymore, and many teams collect application and business-specific metrics using tools such a Graphite and Kibana. Grafana makes it easy to create useful and elegant dashboards for data from a number of sources. A particularly useful feature allows timescales of different graphs to be synchronized, which helps with spotting correlations in the underlying data. The templating system that is being added shows a lot promise and will likely make managing sets of similar services even easier. Based on its strengths, Grafana has become our default choice in this category.
Airflow is a tool to programmatically create, schedule and monitor data pipelines. By treating Directed Acyclic Graphs (DAGs) as code, it encourages maintainable, versionable and testable data pipelines. We've leveraged this configuration in our projects to create dynamic pipelines that resulted in lean and explicit data workflows. Airflow makes it easy to define your operators and executors and to extend the library so that it fits the level of abstraction that suits your environment.
MSBuild has been the primary build system in the .NET ecosystem since its introduction in 2005; however, it suffers from many of the same weaknesses we've previously called out in Maven. The .NET community has started to develop alternatives to MSBuild which are easier to maintain and more flexible, and evolve more fluidly as a project grows. Two of these alternatives are Cake and Fake. Cake uses a DSL built in C#, while Fake uses F#. Each has seen significant growth over the last year and has proven to be a viable alternative to MSBuild for orchestrating common build tasks in .NET projects.
Testing that layout and styling of responsive websites is working as expected across various form factors can be a slow and often manual process. Galen helps ease this problem by providing a simple language, running on top of Selenium, that allows you to specify expectations for the appearance of your website in various screen sizes. Although Galen suffers from the typical brittleness and speed issues of any end-to-end testing approach, we have found benefit in the early feedback on design issues.
Having a way to securely manage secrets is increasingly becoming a huge project issue. The old practice of keeping secrets in a file or in environment variables is becoming hard to manage, especially in environments with multiple applications and large numbers of microservices. HashiCorp Vault addresses the problem by providing mechanisms for securely accessing secrets through a unified interface. It has served us well on a number of projects, and our teams liked how easy it was to integrate Vault with their services. Storing and updating secrets is a bit cumbersome, because it relies on a command-line tool and a fair amount of discipline from the team.
Pa11y is an automatic accessibility tester that can run from the command line and be embedded into a build pipeline. Our teams have had success using Pa11y on a highly dynamic site by first creating a static HTML version, then running the accessibility tests against that. For many systems—especially government websites—accessibility testing is a requirement, and Pa11y makes it all a lot easier.
Scikit-learn is not a new tool (it is approaching its tenth birthday); what is new is the rate of adoption of machine-learning tools and techniques outside of academia and major tech companies. Providing a robust set of models and a rich set of functionality, Scikit-learn plays an important role in making machine-learning concepts and capabilities more accessible to a broader (and often non-expert) audience.
The popular Serverless Framework provides tooling for scaffolding and deployment of serverless applications, primarily using AWS Lambda and other AWS offerings. Serverless Framework provides template support for JavaScript, Python, Java and C#, and has an active community that contributes plugins that extend the framework. The framework also supports the Apache incubator project OpenWhisk as an alternative to AWS Lambda.
With the maturity of tools such as Vault, there is no longer an excuse for storing secrets in code repositories, particularly since this often ends up being the soft underbelly of important systems. We’ve previously mentioned repository-scanning tools such as Gitrob, but we are now pushing proactive tools such as (the ThoughtWorks-created) Talisman, which is a prepush hook for Git that scans commits for secrets matching predefined patterns.
With Terraform, you can manage cloud infrastructure by writing declarative definitions. The configuration of the servers instantiated by Terraform is usually left to tools like Puppet, Chef or Ansible. We like Terraform because the syntax of its files is quite readable and because it supports a number of cloud providers while making no attempt to provide an artificial abstraction across those providers. Following our first, more cautious, mention of Terraform almost two years ago, it has seen continued development and has evolved into a stable product that has proven its value in our projects. The issue with state file management can now be sidestepped by using what Terraform calls a "remote state backend." We’ve successfully used Consul for that purpose.
Amazon Rekognition is one of the cloud-based image comprehension tools we've mentioned elsewhere in this Radar. What we like about it is that Amazon has taken a somewhat novel approach to making faces anonymous (using GUIDs) from AWS to accommodate some of the privacy concerns that come with facial recognition.
Android-x86 is a port of the Android open source project to x86 platforms. The project started by hosting various patches from the community for x86 support but then later created its own codebase to provide support for different x86 platforms. We have seen significant time savings by utilizing Android-x86 in our CI servers instead of emulators for hermetic UI testing. However, for UI-specific tests targeting a particular device resolution—simulating low memory, bandwidth and battery—it is better to stick with emulators.
With the growth of interest in streaming data architectures and the downstream data lakes they feed, we have seen an increased reliance on "change data capture" tooling to connect transactional data stores to stream-processing systems. Bottled Water is a welcome addition to this field, converting changes in PostgreSQL’s write-ahead log into Kafka events. One downside of this approach, however, is that you are tied to low-level database events rather than the higher-level business events we recommend as the foundation for an event-oriented architecture.
The combination of AWS Lambda with Amazon API Gateway has had a big impact on how we deploy services and APIs. However, even in this serverless configuration, the amount of configuration required to wire things together is not trivial. Claudia is a tool which automates deployment of AWS Lambda functions written in JavaScript and associated API Gateway configurations. It provides reasonable defaults, and our teams have found it allows them to get started quickly with Lambda-based microservices.
One of those perpetual developer debates involves language typing: How much is just right? Clojure, the dynamically typed functional Lisp on the JVM, added a new entry into this discussion that blurs the lines. Clojure.spec is a new facility built into Clojure that allows developers to wrap type and other verification criteria around data structures, such as allowable value ranges. Once they are established, Clojure uses these specifications to provide a slew of benefits: generated tests, validation, destructuring of data structures and others. Clojure.spec is a promising way to have the benefits of types and ranges where developers need them but not everywhere.
How does a business hand autonomy to delivery teams while still making sure their deployed solutions are safe and compliant? How do you ensure that servers, once deployed, remain secure and compliant over their operational lifetime? These are the problems that InSpec tries to address. InSpec is an infrastructure testing tool inspired by Serverspec, but with modifications that make the tool more useful for security professionals who need to ensure compliance across thousands of servers. Individual tests can be combined into complete security profiles and run remotely from a command line. InSpec is useful for developers but extends to testing deployed production infrastructure continuously, moving toward QA in production.
Molecule is designed to aid in the development and testing of Ansible roles. By building the scaffolding for running Ansible role tests on a virtual machine or container of choice, we don't have to setup our testing environment manually. Molecule leverages Vagrant, Docker, and OpenStack to manage virtual machines or containers, and supports Serverspec, Testinfra, or Goss to run the tests. The default steps in the sequence facility model include: virtual machine management, Ansible linting, idempotence testing and convergence testing. Although it is a fairly young project, we see a great potential for its usage.
As any Emacs fan will tell you, Emacs is more than a text editor; it is a platform for character-mapped applications. Over the past few years, there has been an explosion of new developments on this platform, but we think Spacemacs deserves particular attention. Spacemacs provides an introduction to the Emacs platform, with a new keyboard user-interface, simplified customization layers, and a curated distribution of Emacs packages. One of the project's aims is to be the best of worlds by combining the Vim UI with the internal reprogrammability of Emacs. We consider developer productivity tools to be a vital part of effective software development, and if you haven’t considered Emacs for a while, we suggest you take a look at how Spacemacs rethinks this classic development platform.
spaCy is a Natural Language Processing (NLP) library written in Python. It is a high-performance library, intended for use by developers in production, and applies NLP models suited for processing text that often mixes in emoticons and inconsistent punctuation marks. Unlike other NLP frameworks, spaCy is a pluggable library and not a platform; it is aimed at production applications rather than model training for research. It plays well with TensorFlow and the rest of the Python AI ecosystem. We've used spaCy in the enterprise context to build a search engine that takes human language queries and helps users make business decisions.
Netflix has open sourced Spinnaker, its microservices continuous delivery (CD) platform. Compared to other CI/CD platforms, Spinnaker implements cluster management and deployment of baked images to the cloud as first-class features. It supports out-of-the-box deployment and cluster management for multiple cloud providers such as Google Cloud Platform, AWS and Pivotal Cloud Foundry. You can integrate Spinnaker with Jenkins to run a Jenkins job build. We like Spinnaker's opinionated approach for deploying microservices to the cloud—with the exception that Spinnaker's pipelines are created via a user interface (UI) and cannot be configured as code.
Given the wide use of infrastructure tools today, it should come as no surprise that infrastructure as code has increased in current projects. With this tendency comes the need for testing this code. With Testinfra you can test the actual state of your servers configured manually or by tools such as Ansible, Puppet and Docker. Testinfra aims to be a Serverspec equivalent in Python and is written as a plugin to the Pytest test engine.
Yarn is a new package manager that replaces the existing workflow for the npm client while remaining compatible with the npm registry. With the npm client, we may end up with a different tree structure under node_modules based on the order that dependencies are installed. This nondeterministic nature can cause "works on my machine" problems. By breaking the installation steps into resolution, fetching and linking, Yarn avoids these issues using deterministic algorithms and lockfiles and thus guarantees repeatable installations. We've also seen significantly faster builds in our continuous integration (CI) environment because of Yarn caching all the packages it downloads.
Unable to find something you expected to see? Your item may have been on a previous radar
我们不仅仅是一家软件公司，同时也是一个社区：我们汇集满怀激情的软件精英，通过技术和客户共同应对最艰巨的挑战。同时我们寻求IT行业的革新，并致力于对社会产生积极的影响力。
