ThoughtWorker Blogs

Hang around my colleagues at ThoughtWorks and you soon get the impression that the only good Enterprise Service Bus (ESB) is a dead ESB. Jim Webber refers to them as Egregious Spaghetti Boxes. So it's not uncommon to hear tales of attempts to get them out of systems that don't need them.Battle was joined at one client and it brought to mind my younger days playing D&D. Webber swings but misses as the ESB is AC 2, Evan gets a hit and rolls 2d8 for 6 damage. Erik finally kills it by casting "Summon Request Stream Map".So what was Erik's decisive spell? Essentially the idea was to take a simple request and show how the data for the request and response made their way through the layers of the application. Erik printed out all the code that you needed to read to understand how this would work - which ran to several pages. He also produced this diagram.It's currently fashionable in agile circles to do Value Stream Mapping as a way to uncover waste in a software development process. I think of this as a request stream map because it similarly takes a request and shows how it moves through the layers allowing us to visualize what's going on and think about the cost and value of the layers.Layering is an essential tool for building software applications. But like most essential things in life, excess can be almost as much of a problem as too little. A visualization like this (or the multiple pages of code) can help you find where "just enough" is.One hazard, however. If you do need to transform data from one form to another, it's usually better to a few little transformations than one big transformation. You want to avoid unnecessary transformations not compress the ones you need.

While I’m coding, I usually have a bunch of very helpful pixies hanging around my desk. (They’re Dan’s pixies really, since he thought of them first; I’m just borrowing them.) The pixies are bored, and just waiting for a job to do. So, when I’m coding a class, they look out for opportunities to help out that class. When I’m coding the Game of Life, for instance, I write a Gui class that lets me toggle the cells on the grid. Then I have to work out what happens when I toggle the cells. I could do it in the same class - in the gui - but fortunately the pixies step in to prevent me making these poor design decisions. “Oh, I’ll do that for you!” one of the pixies calls out. (They usually start with this phrase, and they’re all called Thistle.) “Thanks, Thistle! Do you know what you’re doing?” “Um, not really. What’s toggling a cell? Why’s that valuable? What is it you want me to do for you again?” “I need you to handle the cell living and dying when I toggle it.” “Oh, okay!” Thistle says. “I don’t like the ‘and’ word so much, though. It makes me feel like I’m doing two things at once. What do you call that? The living and dying thing?” “Hm.” I think about it while the pixie taps his foot impatiently. “I’d call it a lifespan, maybe. Can you handle the cell lifespan for me? Just let me toggle the cells. I also need you to tell any observers that there have been some changes to the cells, and give them a way of finding out where those changes are. I think they’ve already got an idea of what they want there.” “Really? Both things?” “Well, there’s no point doing one if you don’t do the other. It’s all part of the same role.” “If that’s what’s valuable to you then I’ll do it,” he says. “Just pretend I’m there for the moment; I’ll be back when you need me.” “Fine,” I say. So I use a mock pixie in place of the real one. I create an interface which does what the pixie’s going to do: IHandleCellLifespans. (See, it’s an interface that starts with “I”, and it represents a role that the pixie is playing for me. This is a role-based, anthropomorphized interface.) So, now we have code which compiles. Of course, the real code in the Gui is null, or maybe a null object pattern - I might create something like IHandleCellLifespans.KILLING_THEM_ALL if I’m feeling particularly mean. But that’s all right, because Thistle the pixie will step in when it’s time. So, I run the code. I’ve usually written an automated scenario. It doesn’t matter whether I run the scenario or step through the game manually; both result in the same thing happening, or not happening - no matter where I click the mouse, no cells appear. Pixies are notoriously unreliable. Since I can’t rely on the pixies, I inject a new class to handle the dependency instead. I decide to call it the Engine, for the moment, and I write an example of how to use it and what it should do for me. The next step is the Next button. I think about how this would work in the Engine, and start writing some code to show how the Engine needs to behave. I’ll need to calculate the number of neighbours, and apply the rules accordingly. One of the pixies pipes up, “Oh, I’ll do the neighbours!” and another one says, “Oh, I’ll handle the rules!” “Fantastic!” I’m so trusting; I always forget what these pixies are really like when it comes to getting the work done. “If I’m going to count the neighbours,” Thistle says, “I’ll need some information about where I’m counting from, and what’s around me.” “Ah, I can get the information from the cell itself,” I say. “No, don’t do that. It’s fine where it is; I’ll just sit inside the cell and do it from there. Can you give me something that lets me know where the other cells around me are? Then I can do the work for you.” “Sure,” I say, “the Engine knows where the life is. I’ll just give you access to the Engine and let it play that role for you.” “Can’t I do it instead? I’m bored,” one of the other pixies asks. “Just give me the information from the engine, and let him talk to me instead.” “Um, Okay.” Of course, when I try to run it then I find out that all the pixies have mysteriously vanished, and I end up assigning the role to the Engine anyway, or one of its anonymous inner classes. Having it defined as a different role means that it’s easy to move this responsibility around. Maybe I’ll create a World to look after the cells, and let that do the job instead; the pixies certainly aren’t very helpful. “What do you mean, we’re not very helpful?” Thistle says. “Look at your code. You haven’t written anything that isn’t needed by something else, so there’s less code to maintain. Because we jump in all the time to try and do jobs for you, every time you can assign a new responsibility to something else, you do - that’s the single responsibility principle in action; none of your classes are doing too much. And you can replace us with something else that does the job at any time - that’s the Liskov Substitution principle. The roles we perform are clearly named. It’s been easy to describe the behaviour of each class using mocks to stand in when we’re not there, and the examples are very readable. You can also use them to work out whether your code still works or not, by running them as tests.” “Okay. I can’t see myself relying on you guys for bigger, enterprise stuff, though.” “What do you mean?” Thistle looks offended. Oops. “Well, let’s say that I’ve got a shop, and I need the tills to talk to stock control.” “Controlling stock? Oh, I’ll do that!” one of the pixies announces excitedly. “What, all on your own?” “Well, I’ll probably delegate it to a team, but that’s my responsibility - you don’t need to worry about it. I’ll be there when you need me. Just pretend I’m there for now. How would you like to find me? What kind of stuff are you going to send to me, and what do I need to do with it? What would you like back?” So I start with something simple - a URL that I’ll use to find Thistle the pixie, some domain objects that I want to send him, and some objects that I’d like back in return. We talk about how to get the information across, whether some of the tills might provide different stock information, how to talk to the claims department about the quality of the goods we’re selling, and whether I’ll be okay if the claims he gives me have more information than I need. “Hold on,” I say. “You’ve got me playing this game now. I’m not a Claims Department. I’m not going to do the job myself. I’ve got a home to go to!” “Meh, never mind,” Thistle replies. “I’ll be sitting with this team over here, coding the stock control. We’ll just pretend you’re doing the job; we’ll mock you out for now.” “How will I know that I’m doing the job correctly?” “We’ll have to talk to each other occasionally. Is that going to be hard? We’ll write some scenarios over in our team that describe how we’re going to use you.” “What if I make a mistake?” “Do you know what mistakes you’re going to make already?” “No,” I confess. “I’m sure there will be some, though.” “When you make a mistake, we’ll deal with it at that point. Sound good?” I think about it. I reckon I could write some code that pretends to be doing the job of the Claims Department and responds correctly to the way they’ve described how they’re going to use me - just for those examples - then I could go home and Thistle would never know. I knock up a quick stub and slip it into the stock control team’s scenarios, then I disappear too, just like the pixies. I figure I’ll start coding a real Claims Department that does a more robust job tomorrow. When I get back the next morning, all the pixies performing the role of Stock Control have been replaced with code too. The Stock Control team claim that they’ve never even seen them. I corner Thistle again. “You’re really not very helpful, are you?” Thistle looks sulky. “Of course we are! Look at your architecture. You’ve got simple messages going back and forth. Your code is very tolerant of extra information, as is the code on this side. You’ve got lovely RESTful URLs, because you were thinking about how you’d like to find us, instead of us providing you with some weird mechanism that doesn’t match exactly what you wanted. We’ve got clean interfaces and APIs. There are no extra columns in your database, because you only replaced exactly what we said we’d do in the first place. You’ve got scenarios to describe how we work together, and at a unit level examples to describe how you’re delegating responsibility to the other pixies. It’s a lovely, maintainable system with a fairly flat cost of change. Isn’t that what you wanted?” I nod thoughtfully. “I think it would have been easier for me to just write the code instead of going through you all the time.” “Ah,” says Thistle, “but then you’d have code that was easy to write, instead of code that’s easy to use.” I think about how they made me fill in the role of the Claims Department. “You never did any of the work, though! I could have done that job myself; put myself in each of those roles and then replaced myself with real code. That would have let me create consumer-driven interfaces just as easily as using you.” Thistle shrugs. “If that’s what works for you, sure.” “Won’t people think I’m a bit mad? If I start talking about how I’m personally going to use a particular class, or how I’m offering to do a job for another?” Thistle looks at me with raised eyebrows, then gestures at all the other pixies clustered around my desk. “I think it’s a little late to be worrying about that now,” he says. Originally published at lizkeogh.com. Please leave any comments there.

There may be a niche in the market, but is there a market for the niche? How do you create a successful proposition?  If the answer was obvious there wouldn’t be so many failures out there in the market place. It is easy to commence on a journey of product development with a hunch and clearly there is no substitute for validating ideas in the flesh.  That something at ThoughtWorks we do; helping clients test and learn, rapidly building ideas into tangibles that can be piloted at low cost and low risk before investing in significant build and spend.  However, sometimes a little more rigour is required before you commit to commencing a project in earnest. That rigour needs to be focused.  What often happens is this rigour turns into a research phase that turns into a project itself.  It need not be this way.  There are certain things you can do, certain questions to ask as you set out on the journey of creating a new, compelling customer proposition.  What follows then is a strawman model to help test potential propositions before moving forward with them.  There are three components to the model, the customer, the environmental context and the organisation or company. All too often propositions are rooted in the organisation.  They make assumptions about the demand or usage. This model attempts to broaden the analysis and focus upon the customer and the why the proposition will be attractive to them.  The model supports questions that may be asked to help shape thinking, test hypotheses and validate thinking. I do not propose that this should become a major research exercise  (for example market sizing is a huge effort in itself), rather a tool for asking the right questions, and if the answers are hard to come by, maybe that suggests more thought is required in refining the proposition. So here goes, a model that provides a framework for considering new customer value propositions.  It’s just an initial idea and I’d welcome feedback and suggestions. Customer Before you get too carried away with the proposition, a good starting point would be the customer.  Who are they and what do they do.  Let’s remember that your customer is not everybody.  Your proposition in unlikely to be appealing 24/7.  The challenge is to segment your target market and identify the triggers for action. The persona: Who do? Personas are a useful tool for bringing the customer to life.  Much has been written about them, but they are a useful tool for extracting broad data into specific stories that describe individuals. Realise that it is unlikely you will design for everybody. Start with the market that you are targeting, how large is it and what is its propensity to spend? Then within that target market segment the target customer base into different profile customers (personas). You need to understand which persona, which customer profile is most important – prioritise them and focus on the highest value.  This may mean deciding between high volume, low margin mass market and low volume, high margin niche appeal.  This decision needs to be made as early as possible to ensure the proposition remains focused and doesn’t try to be all things to all people, satisfying none. Values, needs, wants and desires People are not empty vessels waiting to consume and be filled with your proposition.  Their behaviour is driven by their values, needs, wants and desire.  These may be fundamentally rational (to satisfy a basic human goal) or emotional (to demonstrate status). They are cultural and time based.  Thinking in these terms helps you understand how the proposition will appeal to the customer at different levels.  Let’s take an example of this; a new mobile phone. Before we think about what the product must do, what are the values that the persona associates with the phone. Is our target market a technophile or a technophobe? Jan Chipchase who works for Nokia includes ethnography in his research to understand how people use their phones; women carry them in their handbags, men in their pockets or their belts. The basic need that the phone must meet to satisfy the customer, she must be able to make and receive calls.  If the product is unable to meet these needs it is not fit for purpose and the phone proposition will inevitably fail. Just making phone calls meets the need but there are additional wants that should be satisfied for the product to be more compelling.  It’s a hassle to remember the number of every person she rings, the customer wants to be able to store numbers and see the number of the person who is calling. Having the ability to see a photograph of her daughter as a screen saver on her phone is neither a need not a want.  The phone is useful and usable without that.  But the customer desires to personalise her phone by having a picture of her daughter on it.  Desirability is the key differentiator of the iPhone.  It doesn’t need to compete on features, it is a cool device that people talk about.  And here is a key decision you need to make on your proposition journey.  Are you looking to compete on parity or whether you want to make a difference. Questions What is the basic need that the proposition is trying to fulfil? What counts as hygiene? What does the customer need to be satisfied? What does the customer want in addition to being just satisfied What do other competive products do to maintain feature parity (if you feel you really need to compete on features alone – bad move!) Few people would argue they don’t want simplicity and clarity in their interactions with products.  How could your product to make life easier for the customer? What will make the customer feel good in themselves about owning the product? What other products are “cool” or desirable to your target market.  How can you leverage the essence of those products? Context So now we are beginning to understand who the customer is, it is time to nest the proposition in terms of their context.  The old maxim that a half drunk bottle of water in a desert is worth its weight in gold, but on the streets of a city is worthless trash, should be remembered.  Even the best of propositions will deliver little value if they not only consider the customer, but also the context in which they apply: time, demand and usage. Trigger So the next step in the model is to ask why, when and how will the customer be attracted to the proposition. What is the trigger that drives the customer to move from awareness (assuming you have that) to action?  There is no point in a financial services company trying to sell me a car loan if I am wealthy enough to own my own car, or I do not drive.  Understand what triggers the customer to be interested in the proposition, when and why this happens.  How can your proposition be at front of mind when the trigger is set. Questions What lifestyle / lifestage events will trigger? Internal events personal to the customer; leaving school, getting a first job, getting married, moving house, retiring etc External events that they have no control over (think about sports sponsorship and tying a proposition to that sport, or tying a proposition to a celebrity e.g. Michael Jackson..) Environment It is very unlikely that the proposition will be wholly unique.  What is the competitive landscape, what noise will it need to be heard above to capture the consumers attention.  Whilst you may review the immediate competitors to see where threats and opportunities lie, what can you learn from other, unrelated products or domains?  How can you fuse together concepts from outside your immediate focus to bring new innovation to your product?  Scenario planning may come in useful, playing out different outcomes for different timelines other than that which you plan for. Questions What is the competitive landscape? What can you learn about similar but unrelated propositions? Have you considered the political, environmental social and technical influences using the old PEST analysis? Have you considered different scenarios and how your proposition would play out under them; what unplanned disruptors could get in the way, or how could your proposition done differently disrupt the market? The experience engine Enough of the customer and externalities, what will the proposition look like and why will the target customer go with it? There are three engines within the organisation that drive the proposition, the experience, delivery and value engines.  So… Utility To be any good, the product has got to offer basic utility.  It has to do what it says it is going to do.  Sadly, too many products and customer propositions end there.  A utility product will match the consumers needs.  This is where most enterprise software sits… What are the key customer needs that the proposition must fulfil? What is the basic core functionality that must be met, what are the features that must be offered to gain traction in the market place? What features that are typical on competitor products that we could do without? Quality I could call this next box usability (as this follows the UXD model) but I think it goes beyond just usability.  What is the quality of not only the immediate interface, but also with the supporting functions?  For example, if you have a call centre to back up the proposition, how many layers of IVR are you forced through? Have you considered usability? Is the packaging aesthetically pleasing? The “happy path” customer journey may be well framed, but what about the “sad path”?  What about when things go wrong, what about when customers don’t act in the way you expect of predict them to act? Brand It is easy to get carried away with a new idea before thinking about what it means to the brand.  Typically there will be a strategic roadmap and whilst the proposition may be attractive it may not fit into where the brand is going. Is the proposition complementary to the overall brand direction or does it require a new brand and identity? Does the proposition support / leverage the brand? Does the brand already ‘do it’ under another guise (are you reinventing a wheel that has already been tried somewhere, sometime in the organisation’s history?) How will it be marketed? Community Finally, what is the ‘buzz’ that the proposition will create, what will get people talking and sharing it and how will you create this buzz. Is there a social network component built in that gets people talking and connected?  How will it get people talking in external networks? What will cause people to recommend it to others? How can customers become part of its evolution? What of the proposition will get people passionate, what will drive them away? Delivery engine People A successful proposition needs not only a talented, passionate and committed team to deliver it to market, it also needs a similar team to run it and support it when it is live.  It is a common failing for a rogue “skunkworks” team to emerge in an organisation and develop what appears a compelling proposition, only to have it knocked back and closed down by the “Business as Usual” processes inherent in the organisation Who do you need to make the proposition successful?  What is the team? Who will create the proposition and who will lead it?  Is it IT led or business led? What are the cross-organisational boundaries that the proposition crosses and how will these be eliminated? Who will take ownership of the proposition once it crosses over into the market? Process What are the processes that will be required to sustain the proposition? If the proposition will require changes to the organisation, how will they be managed, communicated and rolled out? How will the proposition be supported once it is let loose in the market? How will it be communicated to customers? How will you create new sales – sales force. Technology What is the technology that will underpin the proposition? Is it possible to test the ideas using rapid languages such as Ruby on Rails before committing it to the enterprise Java stack? What integration is really necessary and what can be worked around? How can you deliver a beta version in the shortest period of time? How will you avoid heavyweight frameworks and develop incrementally to deliver value early and often? How performant and scalable must the innovation be? Value Engine At its most simplistic, how much will the proposition cost and how much revenue will it generate?  Does it offer cost saving opportunities?  Are there intangible benefits that will be accrued?  Ultimately is it a viable proposition that is worth pursuing, or will the cost to develop and run outweigh the value it will add?  Building out a financial model can take time, in the first instance this should be a napkin analysis, a wake-up call to make sure there is value in the proposition before too much time is invested in it. Cost Every day someone is working on the proposition it is costing you money.  The quicker you can get something to market the faster you will start seeing a return on your investment, similarly the sooner you can “get something out there”, “test and learn” the sooner you can kill a proposition that does not fulfill its promise. How quickly can you get a beta to market? How many people, how many days? What will the cost be to develop the infrastructure? Do you have the skills in house or will you need to go external? Benefit / Revenue At its most crude, how will the proposition make money, but there may be more to what we wish to achieve.  Is the proposition actually going to cut costs, a result of regulatory pressures or a CSR initiative? What are the benefits that will be accrued – both tangible (e.g. financial) and intangible (e.g. social, environmental etc) If you are selling units are you going for high volume low margin or low volume high margin? If it an on-line proposition “advertising” is often seen as the source of revenue. There are two additional components to the model… Implementation Having a compelling proposition is one thing, it is another to successfully communicate it and roll it out to target customers. In a crowded market place, how will the proposition stand out? What are the brand values it will communicate? What is the story that customers will hear and how will they hear that story? How will customers interact with the proposition, what channels will you use to take it to market? What is the roll out strategy? Retain and grow Winning customers is only the first step.  A successful proposition will maintain a long-term relationship with its profitable customers, maintaining the warmth they have to the original proposition and cross-selling and up-selling new ones. How will you retain them and turn them into repeat customers and passionate advocates of the proposition? How will the proposition grow lifetime customer value? What can be cross-sold or up-sold? What can you bundle? How will the proposition deal with churn? OK, so it’s not a perfect model and by no means complete.  There’s some duplication in the thinking and many questions missing, but as any model it can be used to guide and prompt thinking and ensure there are no elephants left in the room when the first line of code gets cut.  I’d welcome any comments on its usefulness, utility and direction.

At some point last year I was asked to review the architecture of the software behind a large and popular website. The resident architect explained how he had followed a modern approach, decoupling the web front-end from back-end services that provide content. To gain further flexibility he had put the front-end and the services on an ESB, mostly to cater for the possibility to make the content available to other consumers. In short, the architecture diagram looked a lot like many others: nothing to see here, move on. The diagram above only shows one of the content services, which for the sake of this article is a service that provides contact details for a person. Based on conversations with the project sponsors I began to suspect that at least the introduction of the ESB was a case of RDD, ie. Resume-Driven Development, development in which key choices are made with only one question in mind: how good does it look on my CV? Talking to the developers I learned that the ESB had introduced “nothing but pain.” But how could something as simple as the architecture in the above diagram cause such pain to the developers? Was this really another case of architect’s dream, developer’s nightmare? Often a picture says more than a thousand words, but at the same time traditional architecture diagrams simply do not provide enough information. (I’ve argued this before, here and here for example.) The answer to my questions, and the anxious project sponsors’ question whether the project would deliver on time, lay in a different picture. The picture should not only show the layering of the system in more detail than the architecture diagram but it should also visualise the complexity introduced by the ESB. It’s often possible to produce more detailed diagrams automatically from source code, configuration, or runtime inspection but in this case I couldn’t find a way to do that, which is which I drew the following: In the centre column this diagram shows the layers involved, from the Struts action that needs to look up details by an ID down to the back-end that provides a “RESTful” service to retrieve the details. The top box contains the front-end layers while the box in the middle represents the ESB. The layers labelled in italics are frameworks or infrastructure software, the rest is project-specific code. Key to this diagram are obviously the coloured representations of data travelling up and down the stack. Primitive types are grey, Java objects are purple circles, XML documents are shown as turquoise hexagons, and HTTP messages as green blobs. Mule messages, which are Java objects, are shown in dark purple. If one representation is wrapped in another it’s displayed inside the wrapper. The diagram highlights that it’s a simple piece of data that travels all the way down the stack (actually further until it hits a relational database at some point) and, equally, that the data needed by the action is available in the right form from the service. All that the intermediate layers are doing is wrapping and unwrapping the data in various ways. In fact, out of the four layers of project-specific code three layers do nothing but wrapping. No wonder the developers felt some pain. As a side note: the diagram also makes it obvious that the wrapping is asymmetric between the WsToRestHandler and the RestServiceWrapperImpl, which makes the point of this layer boundary even more questionable. This diagram made people more interested in the cost of what had just been a simple, innocuous line in the architecture diagram; which was the point of the exercise. So, what would this diagram look like without the ESB? Easy: Apart for the obvious simplicity the diagram highlights that every layer has a purpose and there is only one wrapping, which is not undone either. Would it surprise you if I told you that the project went live without the ESB? Footnotes 1) This article is by no means implying that any of the mentioned technologies are useless or harmful. Their use was harmful in this case. 2) The pattern of wrapping and unwrapping the same data through multiple layers is not new, it was also quite common in early 3-tier architectures. (In fact, some people called their architectures n-tier; I guess to show that the number of tiers was somewhat arbitrary but large.) We had a term to describe this pattern: the wormhole pattern. 3) This project is not an isolated incident of a Wormhole ESB. I have personally seen several in the past years.

We are happy to announce the release of CruiseControl.rb 1.4.0. This release adds support for three distributed version control systems - Git, Mercurial and Bazaar - in addition to Subversion.CC.rb remains easy to install, pleasant to use and simple to hack. Since the source has now moved to a git repository, it is easier than ever to fork and contribute. We're looking forward to your pull requests!Downloads are available from both Rubyforge and Github.

What's the most common programming language in the world?

I'm not sure how you could go about measuring this, but one thing you'd need to do is consider what we mean by programming. My candidate answer considers that the most popular programming language is one used widely by people who do not consider themselves as programmers. This language is Excel, or more generally spreadsheets.

Spreadsheets are easily used for small tasks, but are also used for surprisingly complex and important things. Often I've seen professional programmers gulp when they realize that some vital business function is being run off some spreadsheet that they'd find too complicated to muck with.

In general, we've not had much success with programming languages for these kind of LayProgrammers. Whenever someone talks about some new environment that's going to allow people to specify complex behavior "without programming" I mention COBOL, which was originally designed to get rid of programmers. So it's important to consider what Excel can teach us about programming environments.

One property of spreadsheets, that I think is important, is its ability to fuse the execution of the program together with its definition. When you look at a spreadsheet, the formulae of the spreadsheet are not immediately apparent, instead what you see is the calculated numbers - an illustration of what the program does.

Using examples as a first class element of a programming environment crops up in other places - UI designers also have this. Providing a concrete illustration of the program output helps people understand what the program definition does, so they can more easily reason about behavior.

So why do I feel we need this particular Neologism? Essentially because I think it deserves more thought. We pass by illustrative programming examples without really thinking about them or what makes them special - or even that they are special in some way. We've used illustrative programming for years, but we've not paid enough attention to it. We've not thought enough about what are its essential qualities and what its strengths and weaknesses are.

I've chosen the term "Illustrative Programming" to describe this, partly because "example" is so heavily used (and illustration isn't) but also because the term "illustration" reinforces the explanatory nature of the example execution. Illustrations are meant to help explain a concept by giving you a different way of looking at it - similarly an illustrative execution is there to help you see what your program does as you change it.

When trying to make a concept explicit like this, it's useful to think about the boundary cases. One boundary is the notion of using projections of program information during editing, such as an IDE that shows you the class hierarchy while you are working on the code. In some ways this is similar, as the hierarchy display is continuously updated as you modify the program, but the crucial difference is that the hierarchy can be derived from static information about the program. Illustrative programming requires information from the actual running of the program.

I also see illustrative programming as a concept beyond the classic REPL loop of dynamic languages. REPL loops allow you to explore execution, but they don't make the examples front and center in the way that a spreadsheet does its values. Illustrative programming techniques put the illustration in the foreground of your editing experience. The program retreats to the background, peeping out only when we want to explore a part of the illustration.

I don't think that illustrative programming is all goodness. One problem I've seen with spreadsheets and with GUI designers is that they do a good job of revealing what a program does, but de-emphasizes program structure. As a result complicated spreadsheets and UI panels are often difficult to understand and modify. They are often riven with uncontrolled copy-and-paste programming.

This strikes me as a consequence of the fact that the program is de-emphasized in favor of the illustrations. As a result the programmers don't think to take care of it. We suffer enough from a lack of care of programs even in regular programming, so it's hardly shocking that this occurs with illustrative programs written by lay programmers. But this problem leads us to create programs that quickly become unmaintainable as they grow. The challenge for future illustrative programming environments is to help develop a well structured program behind the illustrations - although the illustrations may also make us rethink what a well structured program is.

The hard part of this may well be the ability to easily create new abstractions. One of my observations of rich client UI software is that they get tangled because the UI builders think only in terms of screens and controls. My experiments here suggest to me that you need to find the right abstractions for you program, which will take a different form. But these abstractions won't be supported by the screen builder as it can only illustrate the abstractions it knows about.

My colleagues Rebecca Parsons and Neal Ford have been spending a lot of time involved in thinking along these lines too. So here's some thoughts that Neal had in an email exchange

One of the few people to take illustrative programming seriously is Jonathan Edwards. He's come up with many very imaginative ideas as to what such an environment should look like. His vision of illustrative programming is also closely bound to the notions of projectional editing and controlled copy-and-paste.

The trigger for me in wanting to coin a term here, is the use of illustrative programming by Language Workbenches by people like IntentionalSoftware. These Language Workbenches encourage you to build illustrative DSLs. Using illustration is important in this case since this should help engage lay-programmers, which is one of the aims of using DSLs. The challenge is to do this without falling into the trap of poor program structure.

You’ve logged into your on-line banking checked your balance, paid your bills.  What do you do now?  Click on the logout button?

What do you expect will happen now?  Well given that you have actively chosen to log-out (it’s not something you are likely to click on my mistake), you’d expect to do exactly that.  Logout.  The next screen you get will probably be something that thanks you for on-line banking, with a cross sell for a product or two.

That’s what I assume most customers would expect.  So what are Alliance and Leicester thinking about with this screen?

The customer has clicked log-out but they are still logged in?

Why?

“You are still logged in to Internet Banking - before you go have a look at Your offers.”

Excuse me, I logged out, I don’t need to be logged in for you to show me offers.

Worse: “Are you sure you want to log out?”

OF COURSE I WANT TO LOG OUT!!! Why else would I have clicked the link.

Alliance and Leicester fail here in a fundamental usability rule, that of managing the customer’s expectation. In an application where security isn’t paramount this would be an error, in an application where customers expect their action of leaving their secure accounts will do exactly that… but doesn’t, is inexcusable.

web site news: AMP, an Australian financial services company, ran an internal conference called Amplify. They asked me to talk about agile software development. I thought about how to make this best fit into the overall flow of the conference, particularly since I expected a significant part of the audience to not be part of IT. I settled on talking about how IT projects can be infrastructural or strategic. This classification alters how you approach the projects, in particular on the way IT and business people should collaborate.

Service Orientated Architecture (SOA) is something that is easy for the lay person to understand.  (Try getting a techie to explain REST and you will see the attraction of SOA to the business person - it’s understandable!)  Understandable, in my non-techie hands, is dangerous.  I am entirely unqualified to pass judgment on it, but there are a couple of  things I’ve observed and have been on my mind when I’ve seen SOA nastiness going on.  So excuse me whilst I wax lyrical.

Problem 1. IT haven’t got a clue.

My (lay) understanding of SOA: IT build ’services’ that can be consumed by different applications.  SOA enables us to remove duplication and build the foundations for a scalable architecture that will accommodate changing requirements as the business evolves and grows.  Herein lies the problem; IT build ’services’, second guessing what the business actually needs from said service.

Exhibit One:  Customer Details Service. It exposes details of customers to any application that will use information about customers.   It was designed by the architect in isolation based upon what IT believe a Customer Details Service will be required to do (e.g. the nature of the fields, the domain etc).  This is even though no application has been built, yet alone specified for (”we just know we are going to need customer details”).  It’s putting the cart before the horse.  But IT go ahead and build the service anyway, because they own SOA.   At a later date the business articulate requirements for a new downstream application that requires Customer Details.  It’s the Corporate Business whose domain is Corporate Customers.  But what happens?  The service doesn’t quite meet their requirements.  The fields are wrong.  The Customer Details Service fits the domestic consumer model but not the corporate customers model.  What gives? More likely than not the downstream application.  The corporate customer has to be shoe-horned into the domestic customer service. I’ve seen this done.

Lesson 1. Don’t build SOA in a void.  Get out of that architectural ivory tower and engage with the business (if you can get them to listen - see next point). Better still engage in Guerrilla SOA.

Problem 2. The business haven’t got a clue.

One of the sad realities of the corporate world is that walls that have sprung up and created internal silos that are difficult to bridge.  As the business, the consumer of technology, I want IT to deliver to my requirements, no more, no less.  If I am in the domestic consumer part of the business, frankly I don’t care about Corporate customers.  I’m fighting for my budget, and hell, if this SOA thing is going to cost more than doing a closed application that fits only domestic customers, that only I can use I don’t care.  I’m not going to pay for a “Customer Details” service that does anything except give me what I need to know about my customer.

Lesson 2. The architects should facilitate the discussion.  SOA is as much about your business vision as it is technical architecture.  Unless the business grasps what you are trying to do, drives the solution and requirements are both local and global, before long you’ll see some grand services that few use in the core and chaos is the periphery where the real business is done.

Bottom line?  All too often architects fail because they tend to focus upon the architecture part of SOA rather than the services.    Unfortunately, because of the siloed nature of so may organisaitons, unless it is driven by the architects it is unlikely to gain traction.  If there is a maxim that should be followed when considering SOA in an organisation, it is probably instilling the notion of ‘think local, act global’.

web site news: At QCon San Francisco 2008 Rebecca Parsons and I gave a talk about how agile approaches work with enterprise architecture groups. At the moment there's a lot of distrust and conflict between agile project teams and architecture groups. We dig into why this is so, and explore ways that these groups can work together.