Clearly, then, there’s no lack of will, resources or collective brain power behind the transformation movement. Which begs the question: Why does it so often go wrong?

Any analysis of the overall results of transformation projects can make for sobering reading. One recent study found over half of enterprises have mothballed or abandoned transformation initiatives. IBM has estimated 85% of transformation projects end in failure. And some of the world’s most iconic brands have grappled with unsuccessful transformation exercises, from the BBC, which in 2013 was forced to shut down its Digital Media Initiative and write off over US$100 million in associated investments; to General Electric, which recently announced the spinoff of its troubled GE Digital unit after it couldn’t guide the company out of a rough patch that saw GE lose its century-old place in the Dow Jones index.

Transformation is a complex process, and the reasons for failure are equally so. Some may be unique to the organization, or connected to forces outside the company’s control. But there are also some widely-held misconceptions about transformation that often stand in the way of enterprises aiming to become more agile and technologically empowered. Acknowledging the hard truths provides a head start in the effort to keep transformation on track.

Transforming doesn’t mean everyone’s doing the same thing

There is no doubt that it’s vital for transformation to be driven by a sense of purpose, and that there’s shared alignment throughout the organization on an ultimate desired outcome that’s directly relevant to the business.

“One of the more common mistakes I’ve seen is people diving into transformation without a business problem to solve, so it’s transformation for transformation’s sake,” says ThoughtWorks Principal Consultant Brigid O’Brien. “That means you miss the value metrics you really need to do something that’s meaningful for the customer and for the company.”

At the same time, in seeking harmony companies shouldn’t lose sight of the fact that successful transformation is on many levels complex and diverse. Shared alignment doesn’t mean every team is necessarily doing similar things, or even being measured the same way.

As David Robinson, Business Transformation Principal at ThoughtWorks, notes, transformation projects need to be evaluated across two fundamental dimensions – whether they create the desired value, and whether they continuously enhance the enterprise’s ability to create that value. “The organization essentially has to work on getting better at what it does at the same time it’s doing what it does.”

Bolstering organizational capabilities is something that teams may contribute to in different ways. Speed – the holy grail for many an enterprise – is a good example.

In that it accelerates the delivery of value to customers, “a transformation program that is really focused on the idea of speeding everything up and allowing the company to iterate much quicker is extremely desirable,” Robinson says. But in some divisions of the enterprise – risk, or finance say - simply doing everything faster could be unnecessary, costly, even dangerous, and efficiency may be a better goal.

“One of the key things about most transformations is that there’s no single measure of success,” Robinson explains. “There may be a few, depending on the part of the enterprise. Having clarity on what improvement in capability is needed for a certain part of the organization, and having very clear and transparent measures of success for that, enables progress.” 

There’s no question that transformation should be based on a single vision and desired outcome that all teams strive to achieve. But teams may be working on initiatives gauged by different metrics to improve their capabilities to contribute to that outcome.

Effective transformations are built incrementally, with teams tasked with delivering the “highest-value, lowest-effort change to get the benefit they’re after, and doing it again and again,” Robinson explains. “In other words, don’t try to launch a really long, involved, big bang project to make all the changes you dream of – do it incrementally so you can reap some of the benefits as you go, and keep in mind there may be multiple transformation goals.”

One advantage of this approach is that it can ensure transformation is easily extended to functions that are rarely on the front lines, and often overlooked, in transformation initiatives. Most enterprises understand that cross-functional teams blending IT and the business are an important building block for transformation, but may see less of a need to involve compliance, HR or finance, when in fact they have an important role to play by dismantling barriers to change in their respective realms.

“If you’re a function like risk or compliance, which is basically a gate for everybody, you impact the flow efficiency of an organization, so by improving the flow of work, you’re improving the effectiveness of that organization,” says Jonathan Pangrazio, Principal Consultant at ThoughtWorks. “You should be in the (transformation) discussions, thinking about your function and operation, and the ways to open that gate while still maintaining some level of control.”

“Integrating HR and finance into the value stream and ensuring they’re aligned with the delivery chain, helping them with standups and backlogs and ensuring they have their own sprints, helps them feel less like they’re off somewhere separate looking at this foreign way of working,” O’Brien says. “That evolves to where finance and HR teams have an easy transition to developing their own engineering capability and their own features, like value to the internal customer.”

This approach also means business leaders don’t need to agonize over mapping out the catch-all, universally applicable metrics by which a transformation project will be judged before it even gets off the ground. 

“If you really want to find a way to make a difference and to invest in something new, you’ve got to figure out how to stop spending so much on just taking care of what you’ve already got,” Robinson says. “That requires improvements in efficiency, quality and managing technical debt.”

Transformation is therefore a balancing act between facilitating change, and ensuring BAU is not neglected or deprived of resources to a degree that plants the seeds for bigger problems later on. It also requires understanding that, as Robinson puts it, “there is no free lunch” – that is, for every effort to learn something new, some amount of capacity and energy has to be reallocated and “other things aren’t happening, or aren’t happening as effectively as they were.”

“A much better approach is a realization that we have to walk and chew gum at the same time,” says Robinson. “It’s absolutely necessary that we continually invest in improving our capability. But we make that a manageable investment, we dedicate a small amount of our capacity to that, so that we can continue to generate a constant stream of value for customers while changing and improving our ability to do so.”

Transformation isn’t always horizontal

Balance is also required in terms of the leadership of a transformation program. Simplified, more horizontal structures in which management layers are reduced and everyone feels free to speak out about problems or opportunities can contribute to agility and therefore positive change. “Breaking down silos, moving decision rights from hierarchical, command-and-control type cultures into those that empower teams to make their own decisions, enables them to move faster,” Robinson says.

Yet transformation in the truest sense is about changing the nature of the business – and that’s massive, risky and unsettling enough an endeavor that “autonomy without directional leadership leads to chaos,” Pangrazio says. “You must set a direction, introduce clarity about where you’re going, and be working and managing to that.”

Transformation is thus best governed by a very particular kind of leadership. On the one hand, it should be bold and visionary enough to set the tone, rejig the lines of authority where necessary, and provide ‘cover’ for teams suddenly expected to make risky decisions. In public companies, the ‘cover’ role is even more critical, as any perceived failure could see the firm raked over the coals by investors or analysts.

This requires someone “high enough in the organization to provide air cover for the teams of people that they’re asking to step out and take decisions that they previously didn’t have the rights to,” says Robinson. “(Teams) have to believe that they’re not going to all get fired for a mistake. You have to have some kind of reasonable sponsorship who will be the blast shield from the blowback that happens when you inevitably fail and learn.”

At the same time the transformation leader must be willing to remove themselves from decision-making processes where it serves the interests of agility and transparency.

“It’s a very difficult place for executives to be, because it’s not how they’ve been trained,” O’Brien says. “One of the more compelling steps a (transformation) leader can take is letting their teams and the organization know that they don’t have all the answers – that they’re relying on the team to solve problems. That they will be there to help, but aren’t going to be making the decisions.”

“It requires a level of vulnerability that people who are in formal leadership positions are often really uncomfortable with,” agrees Robinson.

Transformation therefore does not necessarily involve rendering the HIPPO – the highest paid person’s opinion - extinct. “Sometimes the voice of the HIPPO is saying exactly what needs to be said,” O’Brien points out. “It’s not always a bad thing.” However it does call for a new breed of HIPPO, which is “more balanced in the way they manage – not as dictatorial, and not as HIPPO-like,” says Pangrazio. And when tasked with steering a transformation, these leaders should be afforded a high level of support, with each incremental success used to build the appetite and momentum for future change.

“They’re staking their careers on these decisions,” O’Brien points out. “We’re asking them to take very challenging steps against everything they’ve been taught to be true. There’s a lot of fear and anxiety in that. The first step is to meet them halfway, take things slow, take lower-risk steps to immediate transformation goals so their level of confidence grows.”

‘Failing fast’ isn’t enough

If leaders are to convince people to take risks as part of a transformation project, they must also communicate a degree of tolerance for failure. Even if large-scale disaster is avoided, missteps are virtually inevitable as teams experiment with new approaches – and these missteps should be forgiven, even encouraged provided they’re evaluated and used as a basis for improvement. 

The necessity of trying and learning from failure has given rise to the ‘fail fast, fail often’ mantra that now rings throughout the halls of tech and traditional companies alike. But there’s a fine line between accepting a certain amount of productive failure and making failure a goal in its own right. The former approach understands failure as part of a broader learning process; the latter can lead straight to the proliferation of the kind of half-baked ideas or experiments that thrive in an environment free of repercussions.

“You want to build a high tolerance for failure, but a low tolerance for incompetence,” Pangrazio explains. “That should be supported by some ‘back to basics’ leadership that makes it clear what’s acceptable and what’s not, ensuring there’s accountability.”

“Failing isn’t the intent – it’s the mechanism by which we learn,” adds Robinson. “The learning is the intent. It’s important to be careful and to make that distinction.”

Similarly, it’s learning, rather than ‘failing’ fast that creates a more responsive and change-ready organization. “At the end of the day, the only competitive advantage you’ll ever have is the ability to learn a little bit faster than your competitors,” Robinson says.

Therefore while ‘fast’ failure can be a good starting point, or useful to prune a huge list of ideas into something more manageable, it shouldn’t be a guiding principle. A better mantra might be ‘fail fast – and then learn even faster.’ 

It’s not only the people and culture

So much in transformation – setting a direction; and persuading teams to embrace change, take risks and learn from the results – depends on people, and creating an organizational culture that fosters dynamic interactions among them.

Consultants and studies often single out culture as a deciding factor in transformation. But another common mistake is concentrating on the organizational aspects of transformation without laying the technological foundations to match.

Some common transformation roadblocks, such as the inability of separate departments to effectively share data, can’t be addressed by altering the way people work; they require deep (and occasionally painful) changes to systems and enterprise architecture.

“More often than not what’s described as transformation is an organizational restructure of people into different divisions,” Pangrazio notes. “That can save a lot of costs and create a good uplift, but often doesn’t hit the mark in terms of fundamentally changing the way services are delivered and making the organization more competitive. To really transform your business model, you need to change the underlying systems and technology. If you don’t, you’re never going to achieve the levels of process and capability autonomy required to keep pace with less encumbered challengers, and technology will always remain an anchor.”

Overhauling core systems is not always an easy choice to make. “In many organizations, it’s a career-defining call,” says Pangrazio. “Will your colleagues and investors support it? A lot of senior executives take the path of least resistance. In highly complex, costly environments, people seek simplicity, and the hardest thing to do is fundamentally reorganize and restructure that technology core.”

By the same token, while an injection of new blood or skills may be welcome, it’s generally not possible to ‘hire your way’ to transformation while neglecting the technology side of the equation.

“Organizations often figure the answer is new people, and sometimes it is, but more often than not what you need is already there,” Robinson says. “Your ability to cultivate and motivate people, and create an ecosystem around them that really unlocks the capabilities they have, is the harder road.”

Responsive doesn’t mean responding to everything

The ability to better deliver benefits to the business - whether increased revenue or lower costs - as a result of delivering value to clients, based on insights derived from relevant data, is an important measure of transformation success. But becoming a more responsive organization shouldn’t be confused with doing everything customers (or your data) seem to be telling you.

Enterprises are taking in more data on their customers, and how their products and services are received and used, than ever. The eagerness to put this data to use in development and decision-making can fuel what Pangrazio calls the “baby bunny” problem – a burgeoning stable of products or projects launched to address fast-developing trends or customer demands.

If this isn’t accompanied by hard investment and changes to systems that boost the company’s development capabilities, the enterprise may be in danger of spreading its resources too thin, or only capable of superficial, front-end ‘digital’ change. “There’s a finite amount of additions to the bunny family that you can make before you get so complex that to make any real change can actually be detrimental to the organization,” says Pangrazio.

In pivoting toward customer-centricity and the greater use of data, every now and then the organization also needs to slow down, take a collective breath and admit it can’t do it all. Part of using data effectively is the ability to ruthlessly prioritize and reduce work in progress. Insights that connect directly to opportunities may warrant urgent action – while others may deserve nothing more than a cursory nod.

“It’s about pruning,” says O’Brien. “In innovation, you’re planting a lot of seeds, and as they sprout up sometimes you need to be willing to cut them, because with each one you’re introducing more costs and complexity.”

This makes it vital to always have a clear idea of what is ultimately of value to the enterprise. “You need to understand and think about how the company makes money, as well as its other goals,” Pangrazio explains. “Innovation needs to have a relationship to commercial or other targets, whether those are to be more sustainable or improve the world.” 

In summary: Don’t transform, evolve

Having examined the stereotypes that can stop transformation in its tracks, what are the secrets to transformation success? The answer, of course, is: It depends.

“We have patterns, we have tools and wisdom gained through experience trying to help organizations with the process,” says Robinson. “But even if we perfected the 217-step plan to get you to operate in the most amazing way, it would be for whoever we did it for, not for you. Because you’re in a different market, with a different group of people and a different set of constraints and intentions. The real differentiator is going to come from something unique in your organization’s DNA. If you ever find a transformation silver bullet, don’t fire it.”

At its most basic, transformation implies a company turning into something else – which may not be the best way to envision the process needed to thrive in a more technology-driven, disruption-intensive market environment.

Rather than setting out to ‘transform,’ the change-ready organization engages in constant analysis, examining the data around existing processes and offerings to establish where these can be tightened or enhanced by the effective application of technology - even jettisoned if necessary. Its core mission and values may stay the same, but organizational and technological infrastructure develops to advance the enterprise’s ability to recognize and respond to change.

While transformations have an end point, “this process is something that’s never done,” O’Brien notes. “It’s continuous improvement, acknowledging your organization should never be satisfied and is always striving.”

“It’s a perpetual exercise,” agrees Pangrazio. And as such it’s more accurately described by a different term.

“Transformation is something you’ve done, when what we’re really talking about is evolution,” he explains. And that means the main question for any enterprise faced with constant shifts is no longer how it should transform in response – but, ‘Can you continue to evolve at the pace that’s required to keep up?’”