Embracing hybrid AI: How Lenovo is leveraging local, on-device AI

Prem Chandrasekaran: Welcome, everyone, to yet another episode of the Thoughtworks Technology Podcast. My name is Prem Chandrasekaran. I'm one of the regular hosts on the podcast. Today, we are joined by a very special guest, Girish Hoogar, who is a global technology executive at Lenovo, leading the cloud and services group. They are also one of our very strategic clients. Welcome to the podcast, Girish. Do you want to quickly introduce yourself?

Girish Hoogar: Sure, Prem. Thanks. I think you gave me a really great introduction. As Prem mentioned, my name is Girish Hoogar. I head the global technology for cloud and software team, which is part of the IDG, which is the device group. I'm glad to be here, Prem.

Prem: Wonderful. What is it that you are up to these days, Girish? What's hot and happening at Lenovo Cloud and Software?

Girish: As usual, Prem, it's all about providing the best device experience to our customers. User-centricity is the highest priority at Lenovo. Any employee you talk to of Lenovo, they will say, "Okay, what are you doing?" When you ask them this question, they are saying, "How do I make my devices better for the best possible user experience?" Pretty much the same thing through hybrid AI. That's the topic that everybody's talking about. How do we improve device experience through hybrid AI is what we're trying to do.

Prem: Excellent. I guess it's 2026, and no conversation is ever complete without mentioning that AI word. Hybrid AI is not something that a lot of folks know about. Can you elaborate a little bit on what you mean by hybrid AI?

Girish: Yes, definitely. You see, it's nothing different. It's just the AI is added to hybrid. Hybrid is not something new to technologists across the world. Hybrid cloud has been there. When you talk about hybrid cloud, it's basically how do you utilize cloud, like whether you want to go to a local instance or to the cloud instance. Similarly, hybrid AI is nothing but, okay, do you want to use a local LLM on your device or you want to go to the cloud LLM? That's pretty much it in simple terms.

Prem: Let me probe a little further on that. Two years ago, the consensus was to just throw every AI workload into these massive cloud clusters. From your vantage point as an engineering leader, why has the central cloud model become a bottleneck for scaling AI, and why must the physical device landscape evolve to take on some of the heavy lifting?

Girish: Oh, great question, Prem. This is something that comes up often in the discussion with executives and engineers these days. One thing is that cloud is not going away, but the assumption that every AI interaction must travel to the centralized cloud is no longer viable. See, as AI becomes embedded in every device experience, everyday device experiences, enterprises are confronting with possible three realities. Latency affects the user experience, moving data continuously is expensive, and many categories of enterprise or personal data should not leave the device in the first place.

We at Lenovo think about PC and the broader device ecosystem as an active part of AI architecture. It's not simply a monitor or a screen connected to a cloud service. If you look at a modern device, which is like a Lenovo AI PC, I'm not sure whether you use a Lenovo AI PC, but anybody who's using a Lenovo AI PC has a CPU, GPU, more than one, and increasingly NPU capacity that can handle meaningful inference locally. Consider a product like Lenovo Vantage, which is one of the widely used application if you're using a ThinkPad.

When a user connects a new accessory, like a headset, or experiences camera or audio issues, or needs battery or performance optimization, those decisions often need to be happening immediately. It would make very little sense to send every signal to the cloud, meaning every device data or device signal to the cloud, and wait for a response and then take action. Local intelligence can identify the conditions of what the issues are of the user and respond immediately. See, the cloud still plays a critical role, Prem. Don't get me wrong. It provides the larger models, fleet-level intelligence, enterprise knowledge, policy management, and so on.

Device becomes the last mile to execute that intelligence that you require, or the remediation that you require. That's why we call it as device being the last mile in the execution layer. The way I see it is that the strategic shift is from centralized AI to a distributed intelligence. What do I mean is that the future is not cloud versus device. The future is one AI experience that intelligently distributed across device, edge, and cloud. For the end user, they don't care what you are doing in the background. What they care about is how are you going to solve what I want you to be solving in a very quick and intelligent way. Hope that makes sense.

Prem: Yes, absolutely. I think it's a multidimensional problem, like you very rightly said. We don't work with a dumb terminal, so might as well make use of the hardware capabilities that we have at our disposal. Of course, the latency and the data privacy concerns are obviously something that are prime in a lot of engineering leaders' minds. This hybrid model definitely makes a lot of sense.

Here's the thing. Architecting that middle tier, which sits between the user's device and the cloud, is an orchestration challenge. We are asking systems to make these split-second decisions: Should this query run locally on the device's silicon to save battery, or does it require the compute horsepower of the cloud? How do we design an intelligent traffic cop mechanism at the system level to handle this kind of routing without destroying the user experience that you just spoke about?

Girish: The key here is the user experience or the device experience that you mentioned. If you think about it, the real innovation in hybrid AI, where is that actually happening? If you think about it, the real innovation in hybrid AI is occurring at the orchestration layer. The user should never decide whether a request runs on an NPU, a private enterprise cloud, or a public hyperscaler. As a technologist yourself, you don't want user to making that decision. The platform must make that decision dynamically. Think of it as an intelligent routing and policy layer.

Nothing but just an intelligent routing and policy layer. What does it do? It considers the sensitivity of the data, the complexity of the task, connectivity, device performance, thermal conditions, battery state, so on. The list goes on and on. Let me give you an example with a-- A Lenovo device could perform intent detection, document classification, device health analysis locally. If a deeper reasoning is required, the orchestration layer can securely send only the relevant context to a larger cloud model. Cloud provides the answer, and the device takes the final action.

The system can also select different, multiple models, too. A small local model can handle a simple task, a domain-specific enterprise model can handle a company-related question or a domain-specific question. A larger foundation model may be used only when sophisticated reasoning is required. This is not complicated as it sounds, Prem. It is similar to how a traffic management system operates right now. It doesn't send every car or vehicle onto the largest highway. It continuously selects the best path based on the current conditions. For example, if you're using Google Maps, every time it reroutes you depending upon the traffic conditions and the weather conditions.

For Lenovo also, we also think it in an intelligent way. The opportunity for especially the Lenovo Cloud and Software Group is to build that orchestration into a common AI platform so that the individual product team do not have to solve routing on their own. We can provide that as a service of our platform. Net-net of it is the customer should experience one intelligent system, even though many models and computing environments may be working behind it. Again, the key is the user experience and the device experience. What customer is looking for is an experience of one intelligent system without trying to understand the complications behind it on which model is being utilized to solve their problem, or for them to get a best optimal experience. I believe the orchestration layer is where the real innovation is happening in hybrid AI.

Prem: You are in the midst of building out this platform, and you've also got a few customers live, isn't that right?

Girish: Yes. We have already built this platform, Prem. As you know, we have a platform which is called as the UDS platform, that's the legal word for it, the UDS platform, which has about 80 million devices hosted on it. It's a home for 80 million devices, 25 million active users. There's about close to 1 billion API calls monthly that are coming towards this platform. There is a huge responsibility here. On this platform, what we do is we are building reusable services.

When I started with Lenovo in 2019, there were only about five to seven reusable services on this platform. Today, I'm proud to say that there is close to 70 to 75 reusable services, with about 10 to 15 AI services that I spoke about. One of them is that whole intelligent routing that we have built. It's right now in a beta stage, which we want to expand and provide it to our desktop users to begin with.

Prem: You bring up a very important point here. There is a big chasm in the industry right now between what one would call AI experimentation and that very slick-looking demo versus something that actually runs and scales in production. Everybody acknowledges that it's relatively easy to build a very, very attractive-looking demo, while it is incredibly difficult to scale that across a global enterprise securely. What are some of the core software disciplines that organizations might be missing when trying to move from sandbox to production in your experience?

Girish: Brilliant question, Prem. See, as you know, prototypes or POCs demonstrate capabilities, while a production platform or a production application demonstrates reliability, security. The core distinction is where when an enterprise AI initiative actually struggles is when it is deployed to production. A prototype can be built and utilized by a few users. For example, I built an app or a website, actually, that tells me when to water my lawn and when not to, because I was tired of trying to water the lawn every day. Based on the weather patterns, the condition of the lawn, with all that into consideration, AI helped me to build an app, actually, that tells me when to water my lawn.

That's a personal app. If I have to scale it to thousands of users, that's a different story altogether. Deploying something, a capability or an application or a platform across 50,000 users, or millions of customer devices, requires a completely different level of engineering discipline. Again, at Lenovo, when we build a capability for a platform such as Vantage, again, I'm sorry I'm repeating this because Vantage is one of the highest daily active-users application that Lenovo has right now. It's got about 25 million daily active users, or even the device orchestration is one of the latest solutions that we offer, which is available for millions of devices.

With that scale, we have to think beyond the models, Prem. We have to consider hardware variations, operating system variation, regional requirements, security policies, which is very important, software dependencies, update mechanisms, telemetry, customer support, and even rollback. The list again goes on and on. With all this happening, now add AI to this mix, which introduces additional moving components. What are those additional moving components? The model can change. The prompt can change. The retrieval data can change. The policy can change.

Any one of these can alter the product's behavior even when the application code remains the same. See, that is why the organization needs to start thinking differently. We cannot be in this traditional software development lifecycle anymore. We need to think about something called as AI engineering lifecycles. See, that includes version-controlled models and prompts, automated evaluations, security data pipelines, and so on. It's again nothing different than what is normally being done. For example, open source compliance that we already do.

Similar processes can be applied to AI engineering to the models that we're going to be utilizing. Even testing must also expand beyond this traditional software testing. This is what I've been challenging the team these days. Don't talk to me about traditional testing anymore. Team needs to test accuracy. When you're using models, team needs to definitely test accuracy, hallucinations, bias, latency, cost, safety, and performance across different device configuration. The biggest mistake is treating AI as a data science project rather than a production software system. Net-net, a great model can create compelling demonstration just like the one that I wrote, like when to water my lawn. Only a great engineering can create a dependable product.

Prem: What you're really telling us is the importance of a very strong engineering discipline, shifting left a lot of the practices that one would want to scale. For example, you talked about testing, you talked about deployment, you talked about the ability to experiment to get insights from your customers incorporated back, and basically, optimize this loop so that you're ready for the AI age.

That's pretty fantastic, and us partnering has really, really been insightful even for Thoughtworks and the teams that are on the ground because of the scale that is involved. You talked about how there are these millions of devices sending data to the platform centrally and then this hybrid nature where some of it is federated out to these machines to work on versus some of it is done on the cloud. It's been a fantastic success story, an engineering success story for sure.

Girish: Executives, including me, we are very excited about the whole AI thing, how fast we are able to do the sprint. Earlier, even POCs took weeks to months. Right now, it takes hours to days to deliver. Having said that, it also introduces another complexity towards it like from demo to production. Of course, the path has been shrinking. Still, we need to be careful as developers as well as software providers. Let's not get too excited once we create a prototype and trying to take it to production quickly. It will definitely fail. It's important that we apply all those engineering practices.

Whether it's left-shifting, whether it is including AI, it's important to consider those practices because all those practices, at the end of the day, we are doing this to bring value to our customer. People tend to forget that the end result is what the customers should love and experience and enjoy. If that is loss, if customer is not your highest priority, then you have lost the whole process of building products. No one is going to utilize it if you're not building it for the customers.

Prem: Absolutely. That leads us to a related point on when the data is decentralized, like we just discussed, and some of it happens on the user's device itself, privacy obviously improves. How are tracking model drift bias ensuring security across thousands of these endpoints becomes a brand new challenge? How should engineering leaders approach governance and trust by design in this new hybrid AI world?

Girish: Yes, great question again, Prem. See, the key is to manage AI and devices in the same disciplined way that enterprise manage software today, just like I said earlier. An organization needs one central control plane that defines which models are approved, which device can use them, what data they can access, very important, and what actions they are allowed to perform. The intelligence, again, what I said before, the intelligence may run locally on the device, but the policy should be managed centrally. For example, imagine an AI model that detects, again, a camera, microphone, or a battery problem on a Lenovo PC.

The analysis can happen locally because you're collecting the user's data there. The enterprise platform should still know which version of the model is installed, whether it has passed the security testing, whether it is producing too many incorrect diagnoses, or whether it needs to be updated or disabled altogether. We do not need to upload user documents, conversation, or personal information to achieve this. The device can send the anonymous data such as success rate, response time, model confidence, and failure type. Another example is an enterprise assistant.

The company may allow it to summarize a local document but prevent sending confidential content to the public cloud. That policy should be automatically enforced for any enterprise assistant rather than letting the employees decide this. If this was in place, many embarrassments from a large enterprise companies could have been avoided, Prem. I don't want to mention names. You have seen. You have heard that code being leaked or important document being leaked publicly.

It's important that governance also needs to be built into the software delivery process. Before a model is released, automated checks should confirm that it is secure, compliant, and accurate, and approved by the intended devices and region. The simple principle is manage the policy centrally, execute intelligence locally, and collect all the information required to maintain trust, meaning collect only the information that is required to maintain trust.

Prem: What you seem to be saying is that it's this federated governance model where the policies are crafted centrally by these experts and then autonomously executed by the teams on the ground so that they can move at pace using automation, using all of the good things that we have been employing for the past several years.

Girish: Absolutely. Yes.

Prem: Let me switch topics here and then talk about these enterprise architects, engineering directors, and other leaders who are currently refactoring their roadmaps to avoid building tomorrow's legacy technical debt. What are the most critical architecture decisions they need to get correctly so that over the next six months or over the next year or so, they are able to prepare better for this hybrid future?

Girish: As an architect, as a developer, when you look at the legacy code, AI can definitely help there in a large way. For me, there are two things I feel generally has with the advancement of AI and the AI coding tools that are available; the two things that has made the life of the developers, architects, as well as the executor simpler one is the POC, where you demonstrate what works with a market or a customer and what doesn't. The second is refactoring the legacy code. There is a tech debt that always is a ticking time bomb.

Every sprint after sprint, we keep pushing it. With the advancement of AI, we are able to take care of these tech debts quickly enough. I believe the architects, the developers need to start thinking about using AI coding practices as well as AI tools to overcome this tech debt rather than thinking about building features first. Building features can always come, but for me, those two practices of building POCs to see if it's a viable feature or a solution with the market or the customer. The second is overcoming the tech debt is critical.

Prem: Any discussion on AI is incomplete without talking about some of the pitfalls. Basically, one of the things is tokenomics, meaning you're using these tokens, everybody's using these tokens, and there are these other stories about companies like Uber having exhausted their yearly budget and so on and so forth. Previously, a lot of the token costs were subsidized by the model providers, but now that free lunch is, I guess, coming to an end.

Now a lot of organizations are starting to see that amount of spend on AI is far exceeding what they had probably accounted for. There are cases where companies have exhausted their yearly budgets even before the half of the year is done. Firstly, is Lenovo encountering any such problems? If so, how are you dealing with some of these?

Girish: Great question, Prem. Yes, every company, including Lenovo, is facing. We call this as a token shock. Again, token shock is no different than the cloud shock we faced earlier. First phase of enterprise AI was focused on proving that the technology worked. The next phase is focused on proving the economics work. Again, it's not new. Same was the case with the enterprise cloud when it came into existence. Again, just to answer your question, the other day when I was in a brainstorm session with one of the engineers, the engineer was saying that, "Hey, the simple answer to token shock is to negotiate a better price per token. Earlier it was free; now the whole token price has gone up, but we can negotiate the better price per token." I told him, "Go back to the cloud age." Initially, our cloud bills were very, very high, too. What did we do in order to be much more responsible and optimizing the use of cloud instance? The answer was simple. We had to engineer systems that would consume fewer instances of cloud. Optimize it.

Similarly, we need to do the same thing that we need to engineer systems that can consume fewer unnecessary tokens. See, if you look at hybrid AI changes, the whole economic model in several ways. First, the device can process and compress context locally instead of sending the entire document or transcript to the cloud. Second, we can route requests to the smallest model capable of completing this task. We don't have to use a large model in order to detect a basic intent detection. Third, many recurring actions can be handled locally or cached.

In a device management environment, for example, common diagnostics can only require local models or on-device models to process and understand what's going on with the device. See, for large enterprise companies, which manage software experiences across large device footprint, even a small reduction in the cloud inference per interaction can translate into a meaningful savings. Cost, again, it's no different, Prem.

Cost must be a metric for engineering alongside performance, security, reliability, and customer experience. Again, it is no different than the cloud era. The cloud spend was also a metric for our developers and engineers. It's no different. Even the tokens should be a metric for our developers and our technologists. The goal is not to eliminate cloud AI, Prem. Don't get me wrong. It is to reserve expensive cloud intelligence for the moments when it can create a differentiated value for our customers.

Prem: Absolutely. You hit the nail on the head. I think our industry is such that history keeps on repeating itself, and this is the newest manifestation of the same problem that we had previously. It was the cloud, it was before that, it was client-server, and so on and so forth. What this conversation has really, really informed me is that at the end of the day, foundational engineering discipline, if you can have that, I think everything else will eventually fall in place sooner or later. That's really the insight. Any other messages that you want to convey to our listeners and to the other engineering leaders who are in this journey and are trying to figure out how to move responsibly in this space?

Girish: Yes, Prem, one of the things that I mentioned yesterday in a monthly roundtable conference that I have with engineers and architects and product managers was that the winning organization will not necessarily be the one which has the access to the largest and the best model. The ones that win will be the ones that can securely orchestrate models, data, devices, and cloud infrastructure in one reliable experience. That again comes to that customer experience. It's not access to the largest or the best model, but using that orchestration layer that we spoke about.

The one that builds the best orchestration layer that can use these models to their advantage to provide the best possible user and device experience will be the winner. I can bet you Lenovo has that unique opportunity. We are very close to our device. We understand device better than anybody else. We operate cloud and software platforms. We manage that last mile where AI must ultimately deliver value to our customer. Device becomes that last mile. Hybrid AI is the architecture, software is the orchestration layer, and a trusted, seamless customer experience is the outcome. That's how I put it, Prem, yesterday in that roundtable conference.

Prem: Absolutely. I think you could not have said it more aptly. With that, we have come to the end of yet another very exciting episode, if I may. Very insightful for me for sure, and I'm sure a lot of our readers will agree with that. Thank you very much, Girish, and we look forward to continuing partnering with Lenovo in the coming months and years. I'm Prem Chandrasekaran signing off from the Thoughtworks Technology Podcast.

Girish: Thanks a lot, Prem. Appreciate it. Take care.