.
Evolve your Infrastructure
Kief Morris
Head of Cloud Engineering Practices
Thoughtworks
All too often automation tools are used to build infrastructure in the cloud that looks pretty much like the infrastructure we had in data centres. We envision a "final" architecture that is essentially static, and take it for granted that changing it later will involve a major programme of work. As a result, we end up with a fragile, inconsistent, unpatched mess in the cloud. Rather than spending most of our time fighting fires and rolling out patches, how about we build our infrastructure from the start assuming we'll need to change it before we've even finished building it? What would that look like?
Let's explore ways to implement cloud-based systems using automation tools like Terraform, CloudFormation, Ansible, Chef, Puppet, Docker, Kubernetes (anything, really) with change as a first-class concern. Apply agile engineering practices like Test Driven Development, Continuous Integration, and Continuous Delivery to your infrastructure. Adopt design patterns to make your infrastructure architecture loosely coupled and resilient. The concepts discussed here are relevant whatever your tools of choice, and should be of interest to any technical practitioner, architect, or decision maker involved in designing and implementing cloud-style infrastructure.
Presentations
Doing Infrastructure as Code? You Can Do Better
Effy Elden
Senior Consultant
Thoughtworks
Shohre Mansouri
Senior Consultant
Thoughtworks
Developing infrastructure as code often gets cumbersome. There are long feedback cycles for the developers from the moment they are done with a bit of coding until they find out that something is broken. With the fix there is going to be another wait time, often minutes, yet to find out another thing needs to be fixed. This wastes a lot of time and it is not ideal. There is also some bits of the code that may never be tested on the Dev or CI or even Test environments, for example some things that are specific to the Prod environment, may never be tested until we actually go to production.
In this talk we will show techniques that can be used to make developing Infrastructure as Code a better experience for developers, by establishing a faster feedback cycle and also producing higher quality well-tested code.
Core Decisions in Event-Driven Architecture
Duana Stanley
Lead Consultant
Thoughtworks
In an event-driven, (micro)services based architecture, we imagine a bunch of services with a single responsibility interacting with each other by emitting and reacting to domain events. Sounds nice.
Here are some of the decisions we need to make to get there:
- When do we use API calls vs events for interactions?
- Should we always prefer building up local replicas from events when we need data from other services?
- How do we design events?
- When do we use event sourcing?
We'll go over these questions and some principals that have helped us in designing distributed event-driven architectures for our clients.
Designing an Asynchronous Microservices Runtime with Kafka: State, Scale and Performance
David Peterson
Systems Engineering Manager, APAC
Confluent
Today's world operates under a new set of constraints and tries to solve problems that were never thought possible. Everything has changed.
This talk will cover the origins of how Apache Kafka came to be along with how it now underpins some of the worlds largest microservice ecosystems. Kafka and the microservice movement gathered momentum at the same time, Kafka for streams or data-in-motion and microservices as a new, fine-grained architectural principle. The thorn in the side for microservices has always been about the ability to handle, state, and scale. Kafka, meanwhile has always been good at distributing state, which means, that now as we build microservices on top of Kafka, there is a natural affinity between them. I will cover the evolution of microservices until the point of today and then work through how we can build stateful scalable microservices using Kafka. You will learn how scaling state works, and also how it can be queried with ‘interactive queries’.
Key takeaways:
- Scalable business processing
- Run it on rails: Instrumentation and monitoring
- Control flow patterns (start, stop, pause)
- Error handling strategies
Experiments in Teaching a Machine to Code
Karen Davis
Senior Data Engineer
Thoughtworks
I've set myself a goal. Can I get a machine learning algorithm to generate code that can pass a unit test? This talk will track the experiments I run as I try to achieve this.
In this talk, we'll explore:
- How to take an experiment based approach to machine learning development
- How to start small and iterate
- How transfer learning works for natural language processing and
- Get an overview of the LSTM architecture
Observability for Everyone
Inny So
Infrastructure Consultant
Thoughtworks
Andrew Jones
Lead Consultant
Thoughtworks
Have you ever tried to debug a production outage, when your system comprises of apps your team has written, third-party apps your team runs, with logs going into some system, application performance metrics going into another system, and cloud platform metrics going somewhere else? Did you find yourself switching tabs, trying to correlate metrics with logs and alerts and finding yourself in a huge tangle? It is a nightmare. In the data world, we talk about aggregating all our data so we can derive new insights quickly, but what about our operational data?
Observability is your ability to be able to ask questions of your system without having to write new code, or grab new data. When you've got an observable system, it feels like you have debugging superpowers, but it can be challenging to even know where to start. If you can even convince your colleagues to start, finding the right tools can be challenging.
In this talk Inny and Andrew will talk about why monitoring and logging are not sufficient anymore (if they ever were), observability basics, and demo an observability platform that you can use to start your observability journey today.
Automated Governance - Raising the Bar for Agility
Bal Sandhu
Consultant Developer
Thoughtworks
Satyam Argawala
Lead Consultant
Thoughtworks
The recent past has seen the Rise of the Platforms. The large scale adoption of platforms like Kubernetes and Istio reflect the desire (and need) for organisations to deliver value to their customers faster and more reliably. However the agility offered by these platforms is eroded by the archaic approach to governance, especially in larger regulated industries like financial services.
Automated delivery pipelines and infrastructure as code are the new bare essentials. In order to unlock the next level of agility it is critical to think about capabilities such as automated governance. This talk will demo an opinionated approach to automated governance that builds upon the foundations to help unlock agility in today's cloud native landscape.
Someone Far Away, Who You Don’t Even Know, Will Die
Josh Kinal
Lead Consultant: Experience Design
Thoughtworks
Tom Sulston
Principal Consultant
Thoughtworks
Josh and Tom discuss the unintended harm of software product design. From strengthening society’s privilege disparity to international war crimes, they'll show you how to avoid this trap.
They promise you will leave feeling empowered and inspired.