Technology Radar

AI workloads are driving organizations to orchestrate large fleets of GPUs for both training and inference. Teams increasingly work with model sizes that exceed a single accelerator’s capacity (even with 80 GB of HBM), pushing them toward distributed training and multi‑GPU inference. As a result, platform teams are building complex, multi‑stage pipelines and continuously tuning for throughput and latency. Discussions in this space included Nvidia DCGM Exporter for fleet telemetry and topology-aware scheduling to place jobs where interconnect bandwidth is highest.

Before this surge in GPU demand, Kubernetes had already solidified itself as the de facto container orchestrator — and it remains a strong substrate for managing AI workloads at scale, even as we also explored alternatives like micro and Uncloud. We’re tracking emerging GPU‑aware scheduling patterns — such as queueing and quota via Kueue, coupled with topology‑aware placement and gang scheduling — to co‑locate multi‑GPU jobs on fast GPU‑to‑GPU links (e.g., NVLink/NVSwitch) and within contiguous data center “islands” (e.g., racks or pods with RDMA). Recent multi-GPU and NUMA-aware API improvements in Kubernetes further strengthen these capabilities, improving cross‑device bandwidth, reducing tail latency and increasing effective utilization.

We expect rapid innovation in AI infrastructure as platform teams race to support the growing demand for AI coding workflows and the rise of agents elevated by MCP. In our view, GPU‑aware orchestration is becoming table stakes — topology is now a first‑class scheduling concern.

It’s evident AI is transforming how we build and maintain software, and it continues to dominate our recent conversations. As AI becomes strategically embedded across the software value chain — from using AI to understand legacy codebases to genAI for forward engineering — we’re learning how to better supply knowledge to coding agents. Teams are experimenting with new practices such as defining custom instructions via AGENTS.md files and integrating with MCP servers like Context7 to fetch up-to-date dependency documentation.

There’s also a growing realization that AI must amplify the entire team, not just individual contributors. Techniques like curated shared instructions and custom commands are emerging to ensure equitable knowledge diffusion. The tool landscape is vibrant: Designers explore UX Pilot and AI Design Reviewer, while developers prototype rapidly with v0 and Bolt for self-serve UI prototyping.

We also continue to debate spec-driven development — its scope, granularity and potential to serve as a single source of truth for incremental delivery. Yet amid the excitement, complacency with AI-generated code remains a shared concern, reminding us that while AI can accelerate engineering, human judgment is still indispensable.