Python 3 introduced many useful features that are not backward compatible with Python 2.x. It also removed numerous Python 2.x features that were maintained for backward compatibility, making Python 3 easier to learn and use and more consistent with the rest of the language. Our experience using Python 3 in domains such as machine learning and web application development shows that both the language and most of its supporting libraries have matured for adoption. We were able to fork and patch minor issues of existing libraries or avoided using incompatible Python 2.x libraries that had been abandoned. If you are developing in Python we strongly encourage you to use Python 3.
Distributed systems often utilize multithreading, event-based communication and nonblocking I/O to improve the overall system efficiency. These programming techniques impose challenges such as low-level threading, synchronization, thread safety, concurrent data structures, and non-blocking I/O. The open source ReactiveX library beautifully abstracts away these concerns, provides the required application plumbing, and extends the observable pattern on streams of asynchronous events. ReactiveX also has an active developer community and supports a growing list of languages, the most recent addition being RxSwift. It also implements binding to mobile and desktop platforms.
Avro is a framework for data serialization. By storing schema along with the message content, it encourages schema evolution. Producers can edit field names, add new fields or delete existing fields and Avro guarantees that the clients continue to consume the messages. Having a schema allows data to be written without overhead which results in compact data encoding and faster data processing. Although the exchange of structure-less messages between producer and consumer is flexible, we've seen teams facing issues with incompatible unprocessed messages in the queue during deployments. We've used Avro in a number of projects and would recommend using it over just sending unstructured messages.
Interest in the Elixir programming language continues to build. Increasingly, we see it used in serious projects and hear feedback from developers who find its Actor model to be robust and very fast. Elixir, which is built on top of the Erlang virtual machine, is showing promise for creating highly concurrent and fault-tolerant systems. Elixir has distinctive features such as the Pipe operator, which allows developers to build a pipeline of functions as you would in the UNIX command shell. The shared byte code allows Elixir to interoperate with Erlang and leverage existing libraries while supporting tools such as the Mix build tool, the IEx interactive shell and the ExUnit unit-testing framework.
We’ve been enjoying the rapid component-level UI testing that Enzyme provides for React.js applications. Unlike many other snapshot-based testing frameworks, Enzyme allows you to test without doing on-device rendering, which results in faster and more granular testing. This is a contributing factor in our ability to massively reduce the amount of functional testing we find we have to do in React applications.
One common problem in application development is how to schedule tasks that run outside the main process periodically or when certain conditions are met. The problem gets more complicated when unexpected events, such as application shutdowns, occur. The Hangfire framework, as our teams discovered, can do this and much more in the .NET environment. Hangfire is both easy to use and flexible, and it embraces a functional style. Particularly interesting is its ability to save a task's state so it can resume when an application restarts after a crash or shutdown.
Some of our ThoughtWorks teams have had very positive experiences with Phoenix, a server-side web MVC framework written in Elixir. In addition to being streamlined and easy to use, Phoenix takes advantage of Elixir to be extremely fast. For some developers, Phoenix evokes the joy they experienced when first discovering Ruby and Rails. Although the ecosystem of libraries for Phoenix is not as extensive as for some more mature frameworks, it should benefit from the continuing success and growth of support for Elixir.
In the previous Radar, we moved AngularJS into the Hold ring (where it remains in this edition). When it comes to Angular 2, we're seeing mixed messages. Over the past year some teams at ThoughtWorks have used Angular 2 successfully and consider it a solid choice. However, Angular 2 is a rewrite, not an evolution, of AngularJS, and switching from AngularJS to Angular 2 is not much different than switching from AngularJS to another framework. Given the, in our experience, superior contenders such as React.js, Ember.js and Vue.js, we're still hesitant to give Angular 2 a strong recommendation. We do want to highlight, though, that it is not a bad choice, especially if you bought into TypeScript.
Caffe is an open source library for deep learning created by the Berkeley Vision and Learning Center. It mostly focusses on convolutional networks for computer vision applications. Caffe is a solid and popular choice for computer vision-related tasks and you can download many successful models made by Caffe users from the Caffe Model Zoo for out-of-the-box use. Like Keras, Caffe is a Python-based API. In Keras, however, models and components are objects created directly in Python code, whereas Caffe models are described by Protobuf configuration files. Either approach has its pros and cons, and converting between the two is also possible.
DeepLearning.scala is an open source deep-learning toolkit in Scala created by our colleagues at ThoughtWorks. We're excited about this project because it uses differentiable functional programming to create and compose neural networks; a developer simply writes code in Scala with static typing. DeepLearning.scala currently supports basic types such as float, double, GPU-accelerated N-dimensional arrays as well as algebraic data types. We're looking forward to future releases of the toolkit which are said to support higher order functions and distributed training on Spark.
Instana is yet another entrant into the crowded application performance management space. The fact that it's built from the ground up for cloud native architectures differentiates Instana from many of its competitors. Features include dynamic discovery, distributed tracing and service health plus the ability to "time shift" your view of your infrastructure to the moment an incident occurred. It remains to be seen whether this product can gain traction over the combination of open source projects—such as Consul, Prometheus and the implementations of OpenTracing—that do the same thing; however it's worth taking a look if you need an out-of-the-box solution.
JuMP is a domain-specific language for mathematical optimizations in Julia. JuMP defines a common API called MathProgBase and enables users to write solver-agnostic code in Julia. Currently supported solvers include Artelys Knitro, Bonmin, Cbc, Clp, Couenne, CPLEX, ECOS, FICO Xpress, GLPK, Gurobi, Ipopt, MOSEK, NLopt and SCS. One other benefit is the implementation of automatic differentiation technique in reverse mode to compute derivatives so users are not limited to the standard operators like sin, cos, log and sqrt but can also implement their own custom objective functions in Julia.
Keras is a high-level interface in Python for building neural networks. Created by a Google engineer, Keras is open source and runs on top of either TensorFlow or Theano. It provides an amazingly simple interface for creating powerful deep-learning algorithms to train on CPUs or GPUs. Keras is well designed with modularity, simplicity, and extensibility in mind. Unlike a library such as Caffe, Keras supports more general network architectures such as recurrent nets, making it overall more useful for text analysis, NLP and general machine learning. If computer vision, or any other specialized branch of machine learning, is your primary concern, Caffe may be a more appropriate choice. However, if you’re looking to learn a simple yet powerful framework, Keras should be your first choice.
Knet.jl is the Koç University deep-learning framework implemented in Julia by Deniz Yuret and collaborators. Unlike gradient-generating compilers such as Theano and TensorFlow which force users into a restricted mini-language, Knet allows the definition and training of machine-learning models using the full power and expressiveness of Julia. Knet uses dynamic computational graphs generated at runtime for the automatic differentiation of almost any Julia code. We really like the support of GPU operations through the KnetArray type, and in case you don't have access to a GPU machine, the team behind Knet also maintains a preconfigured Amazon Machine Image (AMI) so you can evaluate it in the cloud.
The Kotlin programming language is on many of our developers' bucket lists to assess this year, and some have already used it successfully in production. It is an open source JVM language from JetBrains. Our Swift mobile developers like it as it is syntactically closer to Swift and equally concise. Our Java developers have enjoyed its seamless interoperability with the Java language and tools and found it easier to learn than Scala. Kotlin supports functional programming concepts but with less features than Scala. Developers on our teams who like static typing with the compiler catching null pointer defects found themselves writing fewer boilerplate tests.
We have been intrigued by the Physical Web standard created by Google. The idea of Physical Web is simple—beacons broadcast a URL—but the possibilities are broad. Basically, this is a way to annotate the physical world, tying objects and locations into the digital realm. The current transport mechanism is Eddystone URLs over Bluetooth LE, and sample clients are available. Although there are obvious security concerns with following randomly discovered links, we are most interested in use cases with customized clients where you can filter or proxy the URLs as required.
Teams building systems composed of microservices need to think about coordination techniques such as service discovery, load balancing, circuit breaking and health checking. Many of these techniques require teams to set up tooling, which is not always trivial. The Spring Cloud project provides tools for developers so they can use these coordination techniques in the familiar Spring environment. These tools support Consul, ZooKeeper and the Netflix OSS full stack, all tools that we like. Simply put, it makes it easy to do the right thing with these tool sets. Although our usual concerns with Spring still stand, namely that it hides too much of the complexity, you should consider Spring Cloud if you are in the ecosystem and need to solve these problems.
Widespread adoption of AR/VR as a collaboration and communication medium requires a modern and readily available video streaming platform. WebRTC is an emerging standard for real-time communication between browsers that enables video streaming within commonly available web technologies. The range of browsers that support this standard is increasing, but Microsoft and Apple have been slow to adopt WebRTC in their proprietary browsers. If momentum continues to build, WebRTC could form the future foundation for AR/VR collaboration on the web.