Headless content management systems (CMSes) are becoming a common component of digital platforms. Contentful is a modern headless CMS that our teams have successfully integrated into their development workflows. We particularly like its API-first approach and implementation of CMS as code. It supports powerful content modeling primitives as code and content model evolution scripts, which allow it to be treated like other data store schemas and enable evolutionary database design practices to be applied to CMS development. Its robustness and a stream of new features, including a sandbox environment, have impressed our teams further and made Contentful our default choice in this space.
AWS Fargate, the docker-as-a-service option on AWS, is now widely available across regions. It's a great solution for situations in which teams want to run Docker containers, because AWS Lambda functions aren't powerful enough, without having to manage EC2 instances or Kubernetes clusters. Our teams report generally positive experiences with Fargate; however, the convenience of this managed service can come at a cost, in financial terms.
Ethereum Virtual Machine (EVM) was originally designed for the Ethereum main network. Nowadays, however, most teams no longer want to reinvent blockchain from scratch; instead, they'd like to take EVM beyond Ethereum. We've seen a lot of blockchain teams choose to fork Ethereum (e.g., Quorum) or implement the EVM spec (e.g., Burrow, Pantheon), adding their own designs. The intention is to not only reuse the Ethereum design but also leverage its ecosystem and developer community. To many developers, the concept of "smart contract" is almost equivalent to a smart contract written in Solidity. Although Ethereum itself has some constraints, the technology around the EVM ecosystem is booming.
Time series databases (TSDBs) have been around for some time now. But increasingly they're becoming more mainstream as more use cases naturally fit the time series model. InfluxDB continues to remain a good choice for TSDBs with monitoring being one of its key use cases. TICK Stack is an example of a monitoring solution that has InfluxDB at its heart. Influx 2.0 alpha recently introduced Flux — a scripting language for querying and processing time series data. It's still early days for Flux and the jury's out on its broader adoption beyond InfluxDB, but it promises to be more powerful and expressive than InfluxQL and enables pushing time series analytic workloads to the database. However, clustering support for InfluxDB is only available with the enterprise version which has limited its adoption on some of our projects.
Istio is becoming the de facto infrastructure to operationalize a microservices ecosystem. Its out-of-the-box implementation of cross-cutting concerns — such as service discovery, service-to-service and origin-to-service security, observability (including telemetry and distributed tracing), rolling releases and resiliency — has been bootstrapping our microservices implementations very quickly. It's the main implementation of the service mesh technique we've been using. We've been enjoying its monthly releases and its continuous improvements with seamless upgrades. We use Istio to bootstrap our projects, starting with observability (tracing and telemetry) and service-to-service security. We're closely watching its improvements to service-to-service authentication everywhere in and outside of the mesh. We'd also like to see Istio establish best practices for configuration files to strike a balance between giving autonomy to service developers and control to the service mesh operators.
Kafka Streams is a lightweight library to build streaming applications. It supports basic streaming APIs such as join, filter, map and aggregate as well as local storage for common use cases such as windowing and sessions. Unlike other stream-processing platforms such as Apache Spark and Alpakka Kafka, Kafka Streams has been a good fit for scenarios that don't require large-scale distribution and parallel processing; hence we could get away without yet another piece of infrastructure such as cluster schedulers. Naturally, Kafka Streams has been a good choice when operating in the Kafka ecosystem. Kafka Streams is particularly useful when we have to process data strictly in order and exactly once. One particular use case of Kafka Streams is to build a change data capture (CDC) platform.
HashiCorp continues to release interesting software. We've featured HashiCorp Vault in March 2017, and tools related to Terraform are all over this edition of the Radar. We've moved Nomad to Trial because we've had positive experiences using it. While Kubernetes continues to gain traction, we like Nomad's general applicability. It's not just limited to running containerized workloads but can be used to schedule just about anything. Java and Golang are supported natively as well as batch and distributed cron jobs. We like its focus on multi- and hybrid-cloud operations, something likely to become more important to avoid sticky clouds and the fact that it does scheduling well.
Outside the function code itself, applications written as serverless functions are tightly coupled to the cloud platform on which they're hosted. Although events are a common FaaS-triggering mechanism, and every cloud provider supports them in some form, the current proprietary specifications prevent interoperability across clouds. The CloudEvents specification is a burgeoning standard that has been accepted into the CNCF Sandbox. The standard is still in active development but several language bindings exist and Microsoft has announced first-class support in Azure. We're hoping other cloud providers will follow suit.
The GraphQL ecosystem and community keep growing. Hot Chocolate is a GraphQL server for .NET (core and classic). It lets you build and host schemas and then serve queries against them. The team behind Hot Chocolate has recently added schema stitching which allows for a single entry point to query across multiple schemas aggregated from different locations. Although there are plenty of ways to misuse this approach, it's worth assessing whether to add it to your toolkit.
The serverless architecture has popularized a FaaS style of programming among developers; it helps developers focus on solving core business problems with independently built and deployed functions that react to an event, run a business process, produce other events in the process and scale down to zero. Historically, proprietary serverless platforms such as AWS Lambda or Microsoft Azure Functions have enabled this programming paradigm. Knative is an open-source Kubernetes-based platform to run FaaS workloads. There are few things that stand out about Knative: it's open source and provider agnostic; it implements the serverless workflow as described in the CNCF Serverless Working Group whitepaper; it ensures cross-service interoperability by implementing its eventing interface consistent with CNCF CloudEvents specification; and, most importantly, it addresses a common challenge of operating a harmonized and yet hybrid FaaS and long-running container-based architecture. It easily integrates with both Istio and Kubernetes. For example, developers can take advantage of roll-out strategies that Istio implements by traffic splitting between different revisions of the functions. Developers can take benefit of Istio-provided observability not only for long-running container services but also for FaaS programs in the same Kubernetes environment. We anticipate that Knative open-source eventing interface will continue to enable new underlying source and destination event integrations.
Object storage is a popular choice for storing unstructured data and in a few cases structured data in the cloud. We do discourage the use of generic cloud but if you want to minimize the risk of cloud stickiness for object storage, we've found MinIO quite helpful. With an S3-compatible API layer, MinIO abstracts object storage across cloud providers, including AWS, Azure and Google Cloud Platform (GCP), and we've used it successfully in products with flexible target infrastructures from data centers to cloud providers.
Even in the era of deep learning, statistical models still play a role in business decision support. Time series models are widely used to forecast inventories, demand, customer traffic, and so on. Hand-crafting these models so that they're robust and flexible has typically been the role of either specialized statisticians or large commercial software vendors. Prophet is an open-source alternative to commercial forecasting packages that can be programmed in R or Python. Facebook claims to use Prophet internally for business forecasting at scale and has made it available as an open-source package for anyone to use. We like that Prophet removes some of the tedium of model construction, maintenance and data manipulation so that human analysts and subject matter experts can focus on doing what they do best.
Quorum is "an enterprise-focused version of Ethereum" that aims to provide network permissioning and transaction privacy as well as higher performance. One of our teams has worked deeply with Quorum; however, their experience so far hasn't been great. Some challenges result from complex smart contract programming and some come from Quorum itself. For example, it doesn't work well with load balancers and only has partial database support, which will lead to significant deployment burden. We faced some stability and compatibility issues especially on private transactions. Quorum recently attracted a lot of attention because of JPM Coin. However, from a tech perspective, we recommend being cautious when implementing Quorum while keeping an eye on its development.
SPIFFE standardization of service identity has been an important step in enabling turnkey solutions for end-to-end encryption and mutual authentication between services. The SPIFFE standards are backed by the OSS SPIFFE Runtime Environment (SPIRE), which automatically delivers cryptographically provable identities to services. Istio also uses SPIFFE by default. SPIFFE enables many use cases, including identity translation, OAuth client authentication, mTLS "encryption everywhere" and workload observability. ThoughtWorks is actively working with the Istio and SPIFFE communities to bridge the gap between legacy service identity providers and SPIFFE-based identities so that mTLS can be used everywhere between services, inside a service mesh and outside.
Byzantine fault tolerance (BFT) is one of the fundamental problems in cryptocurrency and blockchain systems. It requires overall system agreement on a single data value in the presence of a number of arbitrary faulty processes, which includes malicious fraud. Tendermint is a BFT state machine replication engine that lets you implement your own blockchain systems. The consensus engine, Tendermint Core, takes over the peer-to-peer communication and consensus part, you just need to implement the rest of the application (e.g., construct transaction and verify cryptographic signature) and communicate with Tendermint Core through ABCI. Some blockchain implementations have already chosen Tendermint as their consensus engine.
In previous Radars we've discussed PostgreSQL for NoSQL. PostgreSQL's maturity and extensibility have led to a steady stream of innovative persistence stores built on the Postgres engine. One that caught our attention is TimescaleDB, a database that allows fast writes and optimized queries over time-series data. Albeit not (yet) as full-featured as InfluxDB, TimescaleDB offers an alternative data model and querying capability. You should evaluate TimescaleDB if you have modest scalability needs, prefer to use SQL and appreciate the stability and familiar administrative interface that PostgreSQL offers.