Enable javascript in your browser for better experience. Need to know to enable it? Go here.

Platforms

Adopt ?

Trial ?

  • As the Azure DevOps ecosystem keeps growing, our teams are using it more with success. These services contain a set of managed services, including hosted Git repos, build and deployment pipelines, automated testing tooling, backlog management tooling and artifact repository. We've seen our teams gaining experience in using this platform with good results, which means Azure DevOps is maturing. We particularly like its flexibility; it allows you to use the services you want even if they're from different providers. For instance, you could use an external Git repository while still using the Azure DevOps pipeline services. Our teams are especially excited about Azure DevOps Pipelines. As the ecosystem matures, we're seeing an uptick in onboarding teams that are already on the Azure stack as it easily integrates with the rest of the Microsoft world.

  • Azure Pipeline templates allow you to remove duplication in your Azure Pipeline definition through two mechanisms. With "includes" templates, you can reference a template such that it will expand inline like a parameterized C++ macro, allowing a simple way of factoring out common configuration across stages, jobs and steps. With "extends" templates, you can define an outer shell with common pipeline configuration, and with the required template approval, you can fail the build if the pipeline doesn't extend certain templates, preventing malicious attacks against the pipeline configuration itself. Along with CircleCI Orbs and the newer GitHub Actions Reusable Workflows, Azure Pipeline templates are part of the trend of creating modularity in pipeline design across multiple platforms, and several of our teams have been happy using them.

  • Many of our teams choose CircleCI for their continuous integration needs, and they appreciate its ability to run complex pipelines efficiently. The CircleCI developers continue to add new features with CircleCI, now in version 3.0. Orbs and executors were called out by our teams as being particularly useful. Orbs are reusable snippets of code that automate repeated processes, speed up project setup and make it easy to integrate with third-party tools. The wide variety of executor types provides flexibility to set up jobs in Docker, Linux, macOS or Windows VMs.

  • When we originally blipped Couchbase in 2013, it was seen primarily as a persistent cache that evolved from a merger of Membase and CouchDB. Since then, it has undergone steady improvement and an ecosystem of related tools and commercial offerings has grown up around it. Among the additions to the product suite are Couchbase Mobile and the Couchbase Sync Gateway. These features work together to keep persistent data on edge devices up-to-date even when the device is offline for periods of time due to intermittent connectivity. As these devices proliferate, we see increasing need for embedded persistence that continues to work whether or not the device happens to be connected. Recently, one of our teams evaluated Couchbase for its offline sync capability and found that this off-the-shelf capability saved them considerable effort that they otherwise would have had to invest themselves.

  • For several years now, the Linux kernel has included the extended Berkeley Packet Filter (eBPF), a virtual machine that provides the ability to attach filters to particular sockets. But eBPF goes far beyond packet filtering and allows custom scripts to be triggered at various points within the kernel with very little overhead. Although this technology isn't new, it's now coming into its own with the increasing use of microservices deployed as orchestrated containers. Kubernetes and service mesh technology such as Istio are commonly used, and they employ sidecars to implement control functionality. With new tools — Bumblebee in particular makes building, running and distributing eBPF programs much easier — eBPF can be seen as an alternative to the traditional sidecar. A maintainer of Cilium, a tool in this space, has even proclaimed the demise of the sidecar. An approach based on eBPF reduces some overhead in performance and operation that comes with sidecars, but it doesn't support common features such as SSL termination.

  • GitHub Actions has grown considerably last year. It has proven that it can take on more complex workflows and call other actions in composite actions among other things. It still has some shortcomings, though, such as its inability to re-trigger a single job of a workflow. Although the ecosystem in the GitHub Marketplace has its obvious advantages, giving third-party GitHub Actions access to your build pipeline risks sharing secrets in insecure ways (we recommend following GitHub's advice on security hardening). However, the convenience of creating your build workflow directly in GitHub next to your source code combined with the ability to run GitHub Actions locally using open-source tools such as act is a compelling option that has facilitated setup and onboarding of our teams.

  • If you're using GitLab to manage your software delivery, you should also look at GitLab CI/CD for your continuous integration and continuous delivery needs. We've found it especially useful when used with on-premise GitLab and self-hosted runners, as this combination gets around authorization headaches often caused by using a cloud-based solution. Self-hosted runners can be fully configured for your purposes with the right OS and dependencies installed, and as a result pipelines can run much faster than using a cloud-provisioned runner that needs to be configured each time.

    Apart from the basic build, test and deploy pipeline, GitLab's product supports Services, Auto Devops and ChatOps among other advanced features. Services are useful in running Docker services such as Postgres or Testcontainer linked to a job for integration and end-to-end testing. Auto Devops creates pipelines with zero configuration which is very useful for teams that are new to continuous delivery or for organizations with many repositories that would otherwise need to create many pipelines manually.

  • Since we last blipped about Google BigQuery ML, more sophisticated models such as Deep Neural Networks and AutoML Tables have been added by connecting BigQuery ML with TensorFlow and Vertex AI as its backend. BigQuery has also introduced support for time series forecasting. One of our concerns previously was explainability. Earlier this year, BigQuery Explainable AI was announced for general availability, taking a step in addressing this. We can also export BigQuery ML models to Cloud Storage as a Tensorflow SavedModel and use them for online prediction. There remain trade-offs like ease of "continuous delivery for machine learning" but with its low barrier to entry, BigQuery ML remains an attractive option, particularly when the data already resides in BigQuery.

  • Google Cloud Dataflow is a cloud-based data-processing service for both batch and real-time data-streaming applications. Our teams are using Dataflow to create processing pipelines for integrating, preparing and analyzing large data sets, with Apache Beam's unified programming model on top to ease manageability. We first featured Dataflow in 2018, and its stability, performance and rich feature set make us confident to move it to Trial in this edition of the Radar.

  • We've seen increased interest in GitHub Actions since we first blipped it two Radars ago. With the release of reusable workflows, GitHub continues to evolve the product in a way that addresses some of its early shortcomings. Reusable workflows in Github Actions bring modularity to pipeline design, allowing parameterized reuse even across repositories (as long as the workflow repository is public). They support explicit passing of confidential values as secrets and can pass outputs to the calling job. With a few lines of YAML, GitHub Actions now gives you the type of flexibility you see with CircleCI Orbs or Azure Pipeline Templates, but without having to leave GitHub as a platform.

  • Kubernetes natively supports a key-value object known as a secret. However, by default, Kubernetes secrets aren't really secret. They're handled separately from other key-value data so that precautions or access control can be applied separately. There is support for encrypting secrets before they are stored in etcd, but the secrets start out as plain text fields in configuration files. Sealed Secrets is a combination operator and command-line utility that uses asymmetric keys to encrypt secrets so that they can only be decrypted by the controller in the cluster. This process ensures that the secrets won't be compromised while they sit in the configuration files that define a Kubernetes deployment. Once encrypted, these files can be safely shared or stored alongside other deployment artifacts.

  • VerneMQ is an open-source, high-performance, distributed MQTT broker. We've blipped other MQTT brokers in the past like Mosquitto and EMQ. Like EMQ and RabbitMQ, VerneMQ is also based on Erlang/OTP which makes it highly scalable. It scales horizontally and vertically on commodity hardware to support a high number of concurrent publishers and consumers while maintaining low latency and fault tolerance. In our internal benchmarks, we've been able to achieve a few million concurrent connections in a single cluster. While it's not new, we've used it in production for some time now, and it has worked well for us.

Assess ?

  • actions-runner-controller is a Kubernetes controller that operates self-hosted runners for GitHub Actions on your Kubernetes cluster. With this tool you create a runner resource on Kubernetes, and it will run and operate the self-hosted runner. Self-hosted runners are helpful in scenarios where the job that your GitHub Actions runs needs to access resources that are either not accessible to GitHub cloud runners or have specific operating system and environmental requirements that are different from what GitHub provides. In those cases where you have a Kubernetes cluster, you can run your self-hosted runners as a Kubernetes pod, with the ability to scale up or down hooking into GitHub webhook events. actions-controller-runner is lightweight and scalable.

  • Apache Iceberg is an open table format for very large analytic data sets. Iceberg supports modern analytical data operations such as record-level insert, update, delete, time-travel queries, ACID transactions, hidden partitioning and full schema evolution. It supports multiple underlying file storage formats such as Apache Parquet, Apache ORC and Apache Avro. Many data-processing engines support Apache Iceberg, including SQL engines such as Dremio and Trino as well as (structured) streaming engines such as Apache Spark and Apache Flink.

    Apache Iceberg falls in the same category as Delta Lake and Apache Hudi. They all more or less support similar features, but each differs in the underlying implementations and detailed feature lists. Iceberg is an independent format and is not native to any specific processing engine, hence it's supported by an increasing number of platforms, including AWS Athena and Snowflake. For the same reason, Apache Iceberg, unlike native formats such as Delta Lake, may not benefit from optimizations when used with Spark.

  • Blueboat is a multitenant platform for serverless web applications. It leverages the popular V8 JavaScript engine and implements commonly used web application libraries natively in Rust for security and performance. You can think of Blueboat as an alternative to CloudFlare Workers or Deno Deploy but with an important distinction — you have to operate and manage the underlying infrastructure. That said, we recommend you carefully assess Blueboat for your on-prem serverless needs.

  • When Cloudflare Workers was released, we highlighted it as an early function as a service (FaaS) for edge computing with an interesting implementation. The release of Cloudflare Pages last April didn't feel as noteworthy, because Pages is just one in a class of many Git-backed site-hosting solutions. It did have continuous previews, a useful feature not found in most alternatives. Now, though, Cloudflare has more tightly integrated Workers and Pages, creating a fully integrated Jamstack solution running on the CDN. The inclusion of a key-value store and a strongly consistent coordination primitive further enhance the attractiveness of the new version of Cloudflare Pages.

  • Colima is becoming a popular open alternative to Docker for Desktop. It provisions the Docker container runtime in a Lima VM, configures the Docker CLI on macOS and handles port-forwarding and volume mounts. Colima uses containerd as runtime, which is also the runtime on most managed Kubernetes services (thus improved dev-prod parity). With Colima you can easily use and test the latest features of containerd, such as lazy loading for container images. With its good performance, we're watching Colima as a strong potential for the open-source choice alternative to Docker for Desktop.

  • In the increasingly crowded space that is the enterprise data catalog market, our teams have enjoyed working with Collibra. They liked the deployment flexibility of either a SaaS or self-hosted instance, the wide range of functionality included out of the box, including data governance, lineage, quality and observability. Users also have the option to use a smaller subset of capabilities required by a more decentralized approach such as a data mesh. The real feather in its cap has been their often overlooked customer support, which our people have found to be collaborative and supportive. Of course, there's a tension between simple data catalogs and more full featured enterprise platforms, but so far the teams using it are happy with how Collibra has supported their needs.

  • CycloneDX is a standard for describing a machine-readable Software Bill of Materials (SBOM). As software and compute fabrics increase in complexity, software becomes harder to define. Originating with OWASP, CycloneDX improves on the older SPDX standard with a broader definition that extends beyond the local machine dependencies to include runtime service dependencies. You'll also find implementations in several languages, an ecosystem of supporting integrations and a CLI tool that lets you analyze and change SBOMs with appropriate signing and verification.

  • Embeddinghub is a vector database for machine-learning embeddings, and quite similar to Milvus. However, with out-of-the-box support for approximate nearest neighbor operations, partitioning, versioning and access control, we recommend you assess Embeddinghub for your embedding vector use cases.

  • Temporal is a platform for developing long-running workflows, particularly for microservice architectures. A fork of Uber’s previous OSS Cadence project, it has an event-sourcing model for long-running workflows so they can survive process/machine crashes. Although we don’t recommend using distributed transactions in microservice architectures, if you do need to implement them or long-running Sagas, you may want to look at Temporal.

Hold ?

 
  • platforms quadrant with radar rings Adopt Trial Assess Hold Adopt Trial Assess Hold
  • New
  • Moved in/out
  • No change

Unable to find something you expected to see?

 

Each edition of the Radar features blips reflecting what we came across during the previous six months. We might have covered what you are looking for on a previous Radar already. We sometimes cull things just because there are too many to talk about. A blip might also be missing because the Radar reflects our experience, it is not based on a comprehensive market analysis.

Unable to find something you expected to see?

 

Each edition of the Radar features blips reflecting what we came across during the previous six months. We might have covered what you are looking for on a previous Radar already. We sometimes cull things just because there are too many to talk about. A blip might also be missing because the Radar reflects our experience, it is not based on a comprehensive market analysis.

Radar

Download Technology Radar Volume 26

English | Español | Português | 中文

Radar

Stay informed about technology

 

Subscribe now

Visit our archive to read previous volumes