Note: I use the term Enterprise IT to include traditional Enterprise IT and the Digital. Product and Engineering departments where they exist outside IT.
One of the key functions of Project Portfolio Management (PPM) in IT is that of allocating finite funds to a subset of projects that vie for funding. When it works well, PPM becomes an effective agent of capital allocation within enterprise IT by funding promising projects and terminating underperforming ones. In principle at least, this is not very different to how a venture capitalist might manage their portfolio by investing in promising ventures and freezing funding or writing off investments in ventures that don’t show promise. In practice, it generally works somewhat differently.
In this scheme of things, IT can’t help but resign itself to attempting to deliver scope on plan. We can talk about benefits validation only when we move to a mode where IT solves problems jointly with business/product rather than just deliver scope. Benefits are realized when problems are verifiably solved.
However, the transition from delivering scope to solving problems requires structural change. Typically, IT program and project management organizations worry about delivering scope on plan while product management (or business) worries about solving problems. These two competencies need to work closer together for the transition to happen. Similar to the merger of development and operations advocated by DevOps we need a merger of product and delivery/engineering (ProDel/ProEngg?). Once we have cross-functional teams consisting of empowered product and delivery people, we can turn our attention to the question of how to validate benefits now that we are set up to deliver benefits.
Nothing validates benefits like sales, one may assert. However, not everything we build is market-facing. Even market-facing products may have back-end projects. Besides, sales is also a function of marketing effectiveness, market conditions, competitors etc. and so it can be misleading to rely exclusively on it. But perhaps the biggest drawback of having to rely on sales feedback is that it is often too late in the day. Iterative product development calls for benefits validation prior to the product/enhancement being generally available for sale.
On the other hand, usage analytics provide a better and generic way of measuring the increase or decrease in uptake after each release. Each release has to be as small as possible in order to get close to the conditions required for a controlled experiment. Frequent, small releases are only possible with Continuous Delivery and so it becomes a prerequisite for institutionalizing the habit of benefits validation.
Analytics are needed to validate benefits whether the product is internal or market-facing. They need to be factored into development efforts just like how performance and other operational requirements are taken into consideration. Most enterprise IT is lacking in this regard and it may take a while to institutionalize analytics. In the meantime, user surveys may serve as a stop-gap technique of data-driven, benefits validation.
Benefits validation is meant to inform subsequent rounds of funding. If benefits aren’t accruing as expected, it may be better to cut losses and divert funds to other projects. However, business cases are sometimes funded at one go. Even if execution is planned as a number of releases, the funds for all releases (meant to realize a business case) is sanctioned at one go. Thus, single-stage funding fails to take advantage of benefits validation.
Disclaimer: The statements and opinions expressed in this article are those of the author(s) and do not necessarily reflect the positions of Thoughtworks.