Software Product is a bunch of functionality assembled together and tied within the boundary of business rules. Each line of code which is written to implement a functionality has a potential bug in them. And each valid bug raised requires a change in the code written.

Are these forming a vicious circle ? Let me explain.

In a typical agile methodology, we write the Unit Tests first and then the actual functionality. So that means we actually know how the system would be tested (to some extent). But after writing the functionality, the developer calls a fellow tester to do a Dev Box Testing. Dev Box Testing is generally done on a developer’s machine with a very minimal set of test data (Thanks to developers laziness ! :)). It’s done to ensure that the intended functionality is properly developed and there are no major bugs. It also gives developer the chance to debug the application in case of any exception (if encountered). So till now we have done Unit Testing and Dev Box Testing. What next? Post the developer check in, the application is generally deployed in a Test Environment for a further round of testing. This testing is intensive and is iterated over a large set of sampled data. This kind of full fledged testing often results in uncovering of many bugs.

So we are still left with bugs even after Unit Testing –> Dev Box Testing. So if we see the whole process process,

We write code –> We induce bug –> We find bug -> We write code to fix bug –> We induce bug –> …

I have something to confess:  I’m an information addicted. I’m always reading books, checking out news, following a few hundred blogs, following another few hundred people on twitter and the list just goes on and on. I bet you do the same.

First of all, this is not about how to just pass over your next university exams but how to keep yourself updated or learn something with all the information we are throw everyday. It’s not possible (for me at least) to keep track from everything I read. People like me reads hundreds of information sources, mostly online. Besides that we also read books, attend conferences, meet new people, etc. We are literally bloated by information from everywhere.

how can we manage to digest this huge amount of information we face everyday ?

We all know there’s no unique answer for that question. Everybody has a different approach when it comes to learn something. Some people prefer to take notes while reading, some people prefer to record audio snippets, some people like to create songs that resemble the information, (your method here). As you might think, I happen to have my own technique (probably not only mine) and it’s being very useful to me.

I read somewhere else a good analogy: we have two “databases” in our brain. One is the short-term memory database and the other is the long-term one.  When we read something, every piece of information goes to our short-term memory which is something like a “heap” of unclassified information . As this “database” isn’t classified, our brain doesn’t know if the information is useful or not.  Later on, it can be even 10 minutes, when we try to recall some information from it, we may not be able to easily do it because isn’t classified yet. But no worries : how many times happened to you that you remembered something just a few hours later while doing something else? That happens because our searching process is asynchronous.

Ok, our information searching process is asynchronous, so what ? It literally means, your brain is still learning some subject even after you stopped thinking about it. It’s like dispatching a background process. The problem is you need to constantly remember that subject in order to transfer it to your long-term memory.

You can achieve that by keeping a sort of learning log. I’ve tried many tools to help me keeping track of what I want to learn. For instance, a simple text editor, an iphone app. You just need to ensure you have quick and easy access to it….As I’m a very moody person I keep changing, right now, for instance, I’m using a folder of my browser’s favorites bar which I sync across my computers. Sometimes I also use the favorites features from twitter. As I’m always checking them, I keep remembering the subjects I have demonstrated interest. Most of this links are shortcuts to blog posts, interviews, podcasts, which I may have read/listened only once. But just the fact I’m recalling the title, it helps me to internally recall the content I read before from that source. In case I don’t remember what’s the content is about I just go there and skim over the text so I get the general idea. If even doing so, I don’t understand the content, then I mark it to read it later using a javascript snippet.

What happens when my learning log stack is getting excessively big ? Just use your common sense! If you get to the point where you always need to recall every item in your list, something is wrong! So, how many items should you have in your list? It may vary from person to person. In my case I found 10 to be a good size. It doesn’t mean I need to first learn all 10 items and them move to the next 10 items. Some items can be in your list for a relatively long time, it’s strongly dependent on how complex is the subject and how fast you can learn.

Defining when you “learned” something is very subjective. In my opinion, you can’t know everything. My “learned” definition is when I get to the point where I can explain the subject to someone else and they properly understand it. Having a time constraint can help too, like imposing yourself a 15 days limit for the items in your list.

Resources used:

• Learning log (official way) http://www.csudh.edu/titlev/learninglog.htm
• Read it later http://readitlaterlist.com/

Probably Related posts:

1. RELEASED – “Rails 2.1 – What`s new ?” Book
2. Why I have been away
3. Google Summer of Code

I've been on tour in South East Asia for the last few days and it's been great fun, I must confess. For those who may have waited for an article and didn't see it on time, I apologise - and I'll try...

A one stop shop for Sumeet Moghe's thoughts about learning in the modern enterprise.

Luiza Pagliari and I, Paulo Caroli, gave a talk on the history of Continuous Integration (CI) and CruiseControl at the 2010 FISL conference (International Free Software Forum).

We talked to Martin Fowler and other colleagues at Thoughtworks and to gather (and validate) data on CI and CruiSeControl history. Below is short presentation with this data.

Continuous Integration and CruiseControl history


Please contact me if you are interested in the CI history topic and blog about it. I will add your links to this blog post.

I’m currently in the process of implementing Seph, and I’ve reached an inflection point. This point is the last responsible moment to choose what I will target with my language. Seph will definitely be a JVM language, but after that there is a range of options - some quite unlikely, some more likely. The valid choices are:

• Target Java 1.4
• Target Java 5/6
• Target Java 7
• Target Java 7 with extensions

Of these, the first options isn’t really interesting for Seph, so I’ll strike it out right now. The other three choices are however still definitely possible - and good choices. I thought I might talk a little bit about why I would choose each one of them. I haven’t made a final decision yet, so that will have to be the caveat for this post.

Before talking about the different choices, I wanted to mention a few things about Seph that matters to this decision. The first one is that I want Seph to be useful in the real world. That means it should be reasonably fast, and runnable for people without too much friction. I want the implementation to be small and clean, and hopefully as DRY as possible - if I end up with both and interpreter and just-in-time compiler, I want to be able to share as much of these implementations as possible.

Java 5/6

The easiest way to go forward would be to only use Java 5 or 6. This would mean no extranice features, but it would also mean the barrier to entry would be very low. It would mean development on Seph would be much easier and wouldd in general make everything simpler for everyone. The problem with it would mainly be implementation complexity and speed, which would both suffer compared to any of the Java 7 variants.

Java 7

There are many good reasons to go with Java 7, but there are also some horrible consequences of doing this. For Seph, the things that would make things from Java 7 is method handles, invoke dynamic and defender methods. Other things would be nice, but the three previous ones are the killer features for Seph. Method handles make it possible to write much more succinct code, not generate lots of extra classes for each built in method, and many other things. It also becomes possible to refer to compiled code using method handles, so the connection between the JIT and the interpreter would be much nicer to represent.

Invoke dynamic is quite obvious - it would allow me to do much nicer compilation to bytecode, and much faster. However, I could still build the same thing myself, to much greater cost and it would also mean inlining wouldn’t be as easy to get.

Finally, defender methods is a feature of the new lambda proposal that allow you to add new methods to interfaces without breaking backwards compatibility. The way this works is that when you add a new method to an interface, you can specify a static method that should be called when that interface method is invoked and there are no other implementations on the concrete classes for a specific object. But the interesting side effect of this feature is that you can also use it to specify default implementations for the core language methods without depending on a shared base class. This will make the implementation much smaller and more flexible, and might also be useful to specify required and optional methods in an API.

The main problem with Java 7 is that it doesn’t exist yet, and the time schedule is uncertain. It is not entirely certain exactly what the design of the things will look like either - so it’s definitely a moving target. Finally, it will make it very hard for people to help out on the project, and also it won’t make Seph a possible language for people to use until they upgrade to Java 7.

Java 7 with extensions

It turns out that the interesting features coming in Java 7 is just the tip of the iceberg. There are many other proposed features, with partial implementations in the DaVinci project (MLVM). These features aren’t actually complete, but one way of forcing them to become more complete is to actually use them for something real and give lots of feedback on the feature. Some of the more interesting features:

Interface injection

This feature will allow you to say after the fact that a specific class implements an interface, and also specify implementations for the methods on that interface. This is very powerful and would be extremely helpful in certain parts of the language implementation - especially when doing integration with Java. The patch is currently not very complete, though.

Tail calls

Allowing the JVM to perform proper tail calls would make it much easier to implement many recursive functional algorithms easily. Since Seph will have proper tail calls in the language, this will mean that I will have to implement this myself if the JVM doesn’t do it, which means Seph will be slower based on this. The patch seems to be quite good and possible to merge and harden to the JDK at some point. Of all the things on this list, this seems to be one of things that we can actually envision see being added in the Java 7 or Java 8 time frame.

Coroutines/continuations

Both coroutines and continuations seem to be possible to do in a good way, at least partially. Coroutines might be interesting for Seph as an alternative to Kilim, but right now it seems to be a bit unstable. Continuations would allow me to expose continuations as a first class citizen which is never bad - but it wouldn’t give me much more than that.

Hotswapping

Hotswapping of code would make it possible to do agressive JITting and then backing out from that when guards fail and so on. This is less interesting when we have invoke dynamic, but will give some more flexibility in terms of code generation.

Fixnums, tuples, value types

We all want ways of making numbers faster - but these features might also make it possible to efficiently represent simple composite data structures, and also things like multiple return values. These are fairly simple features, but have no real patch right now (I think).

Light weight code loading (anonymous classes)

It is horrible to load byte code at runtime in Java at this point. The reason is that to be able to make sure your loaded code gets garbage collected, you will have to load each chunk of code in a new class in a new classloader. This becomes very expensive very fast, and also endangers permgen. Anonymous classes make this go away, since they don’t have names. This means you don’t actually have to keep a reference to older classes, since there is no way to get to them again if you lost the reference to them. This is a good thing, and makes it possible to not generate class loaders every time you load new code. THe state of this seems to be quite stable, but at this point JVM dependent.

The price

Of course, all of these lovely features comes with a price. Two prices in fact. The first price is that all the above features are incomplete, ranging from working patches to proof of concepts or sketches of ideas. That means that the ground will change under any language using it - which introduces hard version dependencies and complicates building. The other price is that none of these features are part of anything that has been released, and there are no guarantees that it will ever be merged in Java at any point. So the only viable way of distributing Seph would be to distribute standard build files with a patched OpenJDK so that anyone can download and use that specific JDK. But that limits interoperability and causes lots of other problems.

Somewhere in between

My current thinking is that all of the above choices are bad. For Seph I want something inbetween, and my current best approach looks like this. You will need a new build of MLVM with invoke dynamic and method handles to develop and compile Seph. I will utilize invoke dynamic and method handles in the implementation, and allow people to use Rémi Forax’ JSR 292 backport to run it on Java 5 and 6. When Java 7 finally arrives, Seph will be more or less ready for it - and Seph can get some of the performance and maintainability benefits of using JSR 292 immediately. At this point I can’t actually use defender methods, but if anyone is clever enough to figure out a backport that will allow defender methods to work on Java 5 or 6, I would definitely use them all over the place.

This doesn’t actually preclude the possibility of creating alternative research versions of Seph that uses some of the other MLVM patches. Charles Nutter have shown how much you can do by using flags to add features that are turned off by default. So Seph could definitely grow the above features, but currently I won’t make the core of the language depend on them.

• nottingham-http-cache-channels-01.txt (tags: cache squid http ietf)
• mnot’s Weblog: Cache Channels (tags: cache squid http)

.. to http://www.akshaydhavle.com/blog alongwith all posts and comments..

See you there! And sorry for the inconvenience.

Following the recommendations of Corey Haines, Michael Guterl, James Martin and Michael Hunger I decided to get Kent Beck's screencasts on Test Driven Development which have been published by the Pragmatic Programmers.

I read Kent's 'Test Driven Development By Example' book a couple of years ago and remember enjoying that so I was intrigued as to what it would be like to see some of those ideas put into practice in real time.

As I expected a lot of Kent's approach wasn't that surprising to me but there were a few things which stood out:

• Kent wrote the code inside the first test and didn't pull that out into its own class until the first test case was working. I've only used this approach in coding dojos when we followed Keith Braithwaite's 'TDD as if you meant it' idea. Kent wasn't as stringent about writing all the code inside the test though – he only did this when he was getting started with the problem.

  The goal seemed to be to keep the feedback loop as tight as possible and this was approach was the easiest way to achieve that when starting out.

• He reminded me of the 'calling the shots' technique when test driving a piece of code. We should predict what's going to happen when we run the test rather than just blindly running it. Kent pointed out that this is a good way for us to learn something – if the test doesn't fail/pass the way that we expect it to then we have a gap in our understanding of how the code works. We can then do something about closing that gap.
• I was quite surprised that Kent copied and pasted part of an existing test almost every time he created a new one – I thought that was just something that we did because we're immensely lazy!

  I'm still unsure about this practice because although Ian Cartwright points out the dangers of doing this it does seem to make for better pairing sessions. The navigator doesn't have to wait twiddling their thumbs while their pair types out what is probably a fairly similar test to one of the others in the same file. Having said that it could be argued that if your tests are that similar then perhaps there's a better way to write them.

  For me the main benefit of not copy/pasting is that it puts us in a mindset where we have to think about the next test that we're going to write. I got the impression that Kent was doing that anyway so it's probably not such a big deal.

• Kent used the 'present tense' in his test names rather than prefixing each test with 'should'. This is an approach I came across when working with Raph at the end of last year.

  To use Esko Luontola's lingo I think the tests follow the specification style as each of them seems to describe a particular behaviour for part of the API.

  I found it interesting that he includes the method name as part of the test name. For some reason I've tried to avoid doing this and often end up with really verbose test names when a more concise name with the method name included would have been way more readable.

  A couple of examples are 'getRetrievesWhatWasPut' and 'getReturnsNullIfKeyNotFound' which both describe the intent of their test clearly and concisely. The code and tests are available to download from the Prag Prog website.

• One thing which I don't think I quite yet grasp is something Kent pointed out in his summary at the end of the 4th screencast. To paraphrase, he suggested that the order in which we write our tests/code can have quite a big impact on the way that the code evolves.

  He described the following algorithm to help find the best order:

  • Write some code
    • erase it
    • write it in a different order

  And repeat.

  I'm not sure if Kent intended for that cycle to be followed just when practicing or if it's something he'd do with real code too. An interesting idea either way and since I haven't ever used that technique I'm intrigued as to how it would impact the way code evolved.

• There were also a few good reminders across all the episodes:
  • Don't parameterise code until you actually need to.
  • Follow the Test – Code – Cleanup cycle.
  • Keep a list of tests to write and cross them off as you go.

Overall it was an interesting series of videos to watch and there were certainly some good reminders and ideas for doing TDD more effectively.

One of the other neat ideas I was reminded of when watching Kent Beck's TDD screencasts is the value of 'calling your shots' i.e. writing a test and then saying what's going to happen when you run that test.

It reminds me of an exercise we used to do in tennis training when I was younger.

The coach would feed the ball to you and just before you hit it you had to say exactly where on the court you were going to place it – cross court/down the line and short/deep.

The point of this exercise is that it's relatively easy to just hit the ball over the net but to have control over exactly where the ball's going to go is way more powerful albeit more difficult.

This applies to TDD as well – it's easy to write a failing test but much more useful to write a failing test which fails in exactly the way that we expect it to.

I've written previously about the value of commentating when pair programming and calling your shots is a really easy way of keeping the conversation flowing in a pair.

I like to ask not only how a test is going to fail but why it's going to fail in that way. I think it's a pretty good way of making sure that you as a pair understand exactly what you're doing. It also slows the pace just enough that you're not coding without thinking about what you're doing.

I quite like Bryan Liles' take on this which he described in a talk at acts_as_conference 2009. Bryan suggests that we first write a test and then either make it green or change the error message that you're getting.

We should know whether or not the test is going to go green or the error message is going to change before we run the test and the idea is that we want to either get the test to pass in one go or at least get a step closer to a passing test.

I’ve been dropping a few hints and mentions the last few weeks, and I thought it was about time that I took some time to preannounce a new project I’m working on. It’s going to be much easier writing my next few blog posts if people already know about the project, and my reasons for keeping quiet about it have mostly disappeared. It’s also a moot point since I talked about it at the Emerging Languages camp last week, and the video will be up fairly soon. And I already put the slides online for it, so some things you might have already seen.

So without further ado, the big announcement is that I’m working on a new language called Seph. Big whoop.

Why?

I already have Ioke and JRuby to care for, so it’s a very valid question to ask why I would want to take on another language project - outside my day job of course. The answer is a bit complicated. I always knew and communicated clearly that Ioke was an experiment in all senses of the word. This means my hope was that some of the quirky features of Ioke would influence other languages. But the other side of it is that if Ioke seems good enough as an idea, there might be value in expanding and refining the concept to make something that can be used in the real world. And that is what Seph is really about. That blog post I wrote a few weeks ago with the Ioke retrospective - that was really a partial design document for Seph.

So the purpose of Seph is to take Ioke into the real world while retaining enough of what made Ioke a very nice language to work with. Of course, being the person I am, I can’t actually avoid doing some new experimentation in Seph, but they will be mostly a bit safer than the ones in Ioke, and some of the craziest Ioke features have been scaled back a bit.

Some features

So what’s the difference? Seph will still be prototype based object oriented, in the same way as Ioke. It will definitely consider the JVM its home. It will be homoiconic, and allow AST manipulation as a first class concept - including working with message chains as a way of replacing blocks. It will still have a numerical tower. It will use almost exactly the same syntax as Ioke. It will still allow you to customize operators and precedence rules.

The big difference. The one that basically makes most all other design changes design themselves is a small but very important difference: objects are immutable. The only way you can create new objects is by creating a new object that specifies the difference. This can be done either by creating a new child of the existing object, or creating a new sibling with the specified attributes changed. In most cases, the difference between the strategies isn’t actually visible, so it might be an implementation strategy.

Now once you have immutable objects but still focus on polymorphic dispatch, that changes quite a lot of things. It changes the core data structures, it changes the way macros work, it changes the flow of data in general. It also changes what kinds of optimizations will be possible.

Another side effect of immutability is that is becomes much more important to have a good module story. Seph will have first class modules that ends up still being simple Seph objects at the same time. It’s really a quite beautiful and simple scheme, and it makes total sense.

If you’re creating a new Object Oriented language, it turns out that proper tail calls is a good idea if you can do it (refer to Steele for more arguments). Seph will include proper TCO for all Seph code and all participating Java code - which means you’ll only really grow the stack when passing Java boundaries. This will currently be done with trampolining, but I deem the cost worth the benefit of a tail recursive programming style.

I mentioned above that objects are immutable. However, local variables will be mutable. It will also be possible to create lexical closures. I’m still undecided whether it’s a good idea to leave a big mutable hole in the tyoe system, or whether I should make it impossible for lexical closures to mutate their captured environment. Time will tell what I decide.

Stealing is good

Seph believes in reusing concepts other people have already made a great job with. As such, many pieces of the language implementation will be stolen from other places.

Just like in Ioke, the core numbers will come from gnu.math. This library has served me well, and I’ll definitely continue to use it. The big difference compared to Ioke is that the gnu.math values will be first class Seph object, and won’t have to be wrapped. Seph will also have real floats instead of bigdecimals. This is a concession to reality (which I don’t much like, btw).

Seph will incorporate Erlang style light weight threads with an implementation based on Kilim (just like Erjang).

As mentioned above, the core data structures will have to change. And the direction of change will be towards Clojure. Specifically, Seph will steal/has stolen Clojures persistent data structures, all the concurrency primitives and the STM. I don’t see any reason to not incorporate fantastic prior art into Seph.

As mentioned above, the module system is also not new - it’s in fact heavily inspired of Newspeak. Having no globals force this kind of thinking, but I can’t say I would have been clever enough to think of it without Gilad’s writings, though.

Basically everything else is copied from or inspired by Ioke.

Isn’t mutability the essence of Ioke?

If you have worked with Ioke, or even heard me talk about it, you might have gotten the impression that mutability is one of the core tenets of Ioke. And your impression would be correct. It wasn’t until I started thinking about what a functional object hybrid version of Ioke would look like, that I realized most of things I like in Ioke could be preserved without mutability. Most of the macros, the core evaluation model and many other pieces will be extremely similar to Ioke. And this is a good thing. I think Ioke has real benefits in terms of both power and readability - something that is not easy to combine. I want Seph to bring the same advantages.

Will you abandon Ioke now?

In one word: no. Ioke is still an experiment and there are still many things that I want to explore with Ioke. Seph will not fill the same niche, it won’t be possible for me to do the same experimentation, and fundamentally they are still quite different languages. In fact, you should expect an Ioke release hopefully within a few weeks.

So will it be useful?

Yes. That’s the whole goal. Seph will have an explicit focus on two areas that Ioke totally ignored. These areas are concurrency and performance. As seen from the features above, Seph will include several powerful concurrency features. And from a performance standpoint, Ioke was a tarpit - even if you wanted to make it run faster, there wasn’t really anything to get a handle on. Seph will be much easier to optimize, it’s got a structure that lends itself better to compilation and I expect it to be possible to get it to real world language performance. My current idea is that I should be able to get it to JRuby performance for roughly the same operations - but that might be a bit optimistic. I think it’s in the right ballpark though. Which means you should be able to use it to implement programs that actually do useful things in the Real World ™.

Is it available?

No. At the current point, Seph is still so young I’m going through lots of rewrites. I would like the core to settle down a little bit before exposing everything. (Don’t worry, everything is done in git, and the history will be available, so anyone who wants to see all my gory mistakes will have no trouble doing that). But in a nutshell, this is why this is a preannouncement. I want to get the implementation to the stage where it has some of the interesting features I’ve been talking about before making it public and releasing a first version.

Don’t worry though, it should be public quite soon. And if I’m right and this is a great language to work in - then how big of a deal is another month of waiting?

I’m very excited about this, and I hope you will be too! This is an adventure.

I've been watching Kent Beck's TDD screencasts and in the 3rd episode he reminded me of a mistake I used to make when I was first learning how to test drive code.

The mistake happens when testing collections and I would write a test which would pass even if the collection had nothing in it.

The code would look something like this:

[Test]
public void SomeTestOfACollection()
{
	var someObject = new Object();
	var aCollection = someObject.Collection;
 
	for(var anItem : aCollection)
	{
		Assert.That(anItem.Value, Is.EqualTo(...));
	}
}

If the collection returned by someObject is empty then the test will still pass because there is no assertion to deal with that situation coming up.

In Kent's example he is using an iterator rather than a collection so he creates a local 'count' variable which he increments inside the for loop and then writes an assertion outside the loop that the 'count' variable has a certain value.

In my example we can just check that the length of the collection is non zero before we iterate through it.

[Test]
public void SomeTestOfACollection()
{
	var someObject = new Object();
	var aCollection = someObject.Collection;
 
	Assert.That(aCollection.Count(), Is.GreaterThan(0));
	for(var anItem : aCollection)
	{
		Assert.That(anItem.Value, Is.EqualTo(...));
	}
}

It's a relatively simple problem to fix but it's certainly one that's caught me out before!

• The worlds longest photo exposure (tags: photography longexposure pinhole)

I’ve just got back from Tokyo, where I was honoured to be invited to give the keynote at Agile Tokyo 2010. Agile conferences have a short history in Japan – both Agile Tokyo and Agile Japan are only two years old – and there was a real buzz of excitement, with lots of smart people (280 delegates attended) and interesting conversations. So first of all I’d like to thank Yoshi Nagase and his wonderful team at Technologic Arts for hosting me, Yoko Yoshikawa for taking such good care of me during my time there, Gihyo for organizing the conference and for inviting me, and my fellow presenters for some great discussions.

Agile Tokyo 2010 by Yoko Yoshikawa

The most surprising discovery for me was that agile is considered new and somewhat subversive in Japan. Japanese IT companies have been doing waterfall on large projects for a long time now, and it is deeply entrenched. Indeed, everybody I spoke to confirmed that waterfall worked perfectly well for them as a software development methodology. This was staggering to me, since outside Japan large software projects run with waterfall routinely go over budget and end up with either cuts in scope or poor quality – several recent reports, such as this one, put IT project failure rates in the range of 60-70%. This appears not to be the case in Japan – it is perhaps the only country that could make waterfall work reliably and repeatably. As my host, Yoshi Nagase, commented, this is deeply ironic in a country that created Toyota and lean manufacturing.

However there is certainly an interest in agile now. Several large systems integrators have started running small agile pilot projects, and apparently the results have been successful. Why the sudden interest in agile if waterfall can deliver high quality software on time and on budget? The answer is easily extrapolated from this example of a project plan I saw at one of the companies I visited – the plan has been changed somewhat to protect the guilty, but not in any essential details – it is a pretty standard plan following the V-model.

For me, the most egregious problem with this plan is that the planning stage for phase 2 of the project comes before the testing stage for phase 1. So anything discovered during the testing and user acceptance stages cannot be fed in to phase 2 – it has to wait for phase 3. Effectively, that means that the feedback loop from discovering a problem in your requirements for phase 1 will have to wait until the end of phase 3 to be delivered to users – a cycle time of 1 year. This makes it extremely hard to respond to customer feedback. Even in the best case scenario, whereby you discover a requirement during the planning stage of a phase, you must wait 6 months for it to be delivered to users.

Japanese companies seem to be realizing that they simply cannot maintain a competitive advantage in the current economic climate with this kind of cycle time. They must get valuable software into the hands of users, and incorporate feedback from them, much faster. This of course is the value proposition of agile methods – the first principle of the agile manifesto, which also inspired the title of our book, is this: “our highest priority is to satisfy the customer through early and continuous delivery of valuable software”.

The risk is that Japanese companies simply have very little experience doing agile delivery at all, let alone on large projects. Adopting agile in an effective way involves serious organizational change. Cross-functional teams including developers, testers, analysts, testers, operations staff, and project managers must be created. Developers must be trained in practices like test-driven development, continuous integration, refactoring, and incremental development. Testers will have to start collaborating more closely with analysts and developers. Managers will need to be trained in agile delivery methodologies – and they will need to make mistakes and gain experience. Teams will need new tools.

However if there is any country that can make such a large change in an organized and efficient manner, it is Japan. I’m really excited to see what agile methodologies do for Japanese IT, and also to see what the Japanese software delivery community does to agile – no doubt we will see many new ideas and evolutions. It’s an exciting time to be in IT.

On the majority of the projects that I've worked on at ThoughtWorks we've held a showcase at the end of each iteration to show our client what we've been working on and finished over the previous one or two weeks.

The format of these showcases has been fairly similar each time but the people who attended has tended to vary depending on the situation.

As part of the project being worked on at ThoughtWorks University we've run a showcase at the end of each week which the whole team have been attending.

Toby pointed out that having everyone there didn't seem to be the best use of people's time since the majority of the developers in the room didn't actually have any input during the showcase.

I've often felt the same in that situation although we used a similar approach on a project I worked on last year and it seemed to be quite useful for addressing the human aspect of the project.

The client was able to meet all the people on the team and equally the developers were able to gain some insight into the types of conversations that the analysts were having with the client.

In that particular example we had around 10-15 developers so there was quite a big gathering but on the other projects where we've had the whole team at the showcase there's only been 3 or 4 of us.

On the last project I worked on we took it in turns to go to the showcase. I think it's quite useful to have at least one technical person in a showcase as they're able to give instant feedback on implementation details such as how difficult it will be to change the way a certain piece of functionality works.

A lot of the time we didn't have any input but it was interesting that quite often we'd come out of those showcases with more understanding of what the client was trying to achieve with a specific feature.

If we don't have all the developers in every showcase we do tend to have them all there for the first showcase at least as that tends to be the one where the most stakeholders will be present.

Overall though it's a call to make depending on the situation and the difficulty in judging the value of attending the showcase seems to come about because a lot of the benefits of attending are indirect whereas staying at our desk and coding has a direct benefit to the project.

One of the interesting side effects of using SmartGWT is losing the ability for people to select text on the page. SmartGWT has some pretty nifty widgets and the one to use when you want text selectable is HTMLPane. Be careful not to confuse this with GWT’s own HTMLPanel

I’ve been meaning to write about this for a while, since I keep saying this and people keep getting surprised. Now maybe I’m totally wrong here, and if that’s the case it would be nice to hear some good arguments for that. Here’s my current point of view on the subject anyway.

A specter is haunting the Java community - the specter of generics.

Java introcued a feature called generics in Java 5 (this feature is generally known under the name of parametric polymorphism in the literate). Before Java 5 it wasn’t possible to create a reusable collection that would ensure the type safety at compile time of what you put in to that collection. You could create a collection of for example Strings and have that working correctly, but if you wanted to have a collection of anything, as long as that anything was the same type, you were restricted to doing runtime checks, or just having good tests.

Java 5 made it possible to add type parameters to any other type, which means you could create more specific collections. There are still problems with these - they interact badly with native arrays for example, and wildcards (Java’s way of implementing co= and contravariance) have ended up being very hard for Java developers to use correctly.

Java and C# both added generic types at roughly the same time. The C# version of generics differed in a few crucial ways, though. The most important difference in implementation is that C# generics are reified, while Java generics use type erasure. And this is really the gist of this blog post. Because over and over I hear people lament the lack of reified generics in Java, citing how good C# and the CLR is to have this feature. But is that really the case? Is reified generics a good thing? Of course, that always depends on who is asking the question. Reified might well be good for one person but not another. Here you will hear my view.

Reified? Huh?

So what does reified generics mean, anyway? It is probably easiest to explain compared to the Java implementation that uses type erasure. Slightly simplified: in Java generics doesn’t exist at runtime. It is purely a fiction that the compiler uses to handle type checking and make sure you don’t do anything bad with your collection. After the generics have been type checked, they are used to generate casts and type checks in the code using generics, some metadata is inserted into the class file format, and then the generic information is thrown away.

In contrast, on the CLR, generic classes exist as specific versions of their class. The same class with different generic type arguments are really different classes. There are no casts happening at the implementation level, and the CLR will as a result generate more specific code for the generic code. Reflection and dynamic type checks is also possible on the CLR. Having reified generics means basically that they exist at runtime, that the virtual machine knows about them and handles them correctly.

Multi-language virtual machines

The last twenty years something interesting has happened. Both out hardware and software has gotten mature enough that a new generation of virtual machines have entered the market. Traditionally, virtual machines for languages were made for specific languages, such as Pascal, Lisp and Smalltalk, and possibly except for SECD and the Warren machine, there haven’t really been any virtual machines optimized to running more than one language well. The JVM didn’t start that way either, but it turned out to be more well suited for it than expected, and there are lots of efforts to make it an even better platform. The CLR, Parrot, LLVM and Rubinius are other examples of things that seem to become environments rather than just implementation strategies for languages.

This is very exciting, and I think it’s a really good thing. We are solving very complex problems where the component problems are best solved in different ways. It seems like a weird assumption that one programming language is the best way of solving all problems. But there is also a cost associated with using more than one language. So having virtual machines act as platforms, where a sharked chunk of libraries are available, and the cost of implementation is low, makes a lot of sense.

In summary, I feel that the JVM was the first step towards a real viable multi-language virtual machine, and we are currently in the middle of the evolution towards that point.

Solving the problems

So why not add reified generics to the JVM at this point? It could definitely be done, and using an approach similar to the CLR, where libraries are divided into pre and post reified makes the path quite simple from an implementation standpoint. On the user side, there would be a new proliferation of libraries to learn - but maybe that’s a good thing. There is a lot of cruft in the Java standard libraries that could be cleaned up. There are some sticky details, like how to handle the API’s that were designed for erased generics, but those problems could definitely be solved. It would also solve some other problems, such as making it possible for Scala to pattern match on type parameters and solving part of the problem with abstracting over primitive types. And it’s absolutely possible to do. It would probably make the Java language into a better language.

But is it the only solution? At this point, making this kind of change would complicate the API’s to a large degree. The reflection libraries would have to be completely redesigned (but still kept around for backwards compatibility). The most probable result would be a parallel hierarchy of classes and interfaces, just like in the CLR.

Refified generics are generally being proposed in discussions about three different things. First, performance, second, making it easier for some features in Scala and other statically typed languages on the JVM, and thirdly to handle primitives and primitive arrays a bit better. Of these, the first one is the least common, and the least interesting by far. JVM performance is already nothing short of amazing. The second point I’ll come back to in the last section. The third point is the most interesting, since there are other solutions here, including unify primitives with objects inside the JVM, by creating value types. This would solve many other problems for language implementors on the JVM, and enable lots of interesting features.

The short stick

I believe in a multi language future, and I believe that the JVM will be a core part of that future. Interoperability is just too expensive over OS boundaries - you want to be on the same platform if possible. But for the JVM to be a good environment for more than one language, it’s really important that decisions are made with that in mind. The last few years of fantastic progress from languages like Rhino, Jython, JRuby, Groovy, Scala, Fantom and Clojure have shown that it’s not only possible, but benificial for everyone involved to focus on JVM languages. JSR 223, 292 and several others also means the JVM is more and more being viewed as a platform. This is good.

Generics is a complicated language feature. It becomes even more complicated when added to an existing language that already has subtyping. These two features don’t play very well together in the general case, and great care has to be taken when adding them to a language. Adding them to a virtual machine is simple if that machine only has to serve one language - and that language uses the same generics. But generics isn’t done. It isn’t completely understood how to handle correctly and new breakthroughs are happening (Scala is a good example of this). At this point, generics can’t be considered “done right”. There isn’t only one type of generics - they vary in implementation strategies, feature and corner cases.

What this all means is that if you want to add reified generics to the JVM, you should be very certain that that implementation can encompass both all static languages that want to do innovation in their own version of generics, and all dynamic languages that want to create a good implementation and a nice interfacing facility with Java libraries. Because if you add reified generics that doesn’t fulfill these criteria, you will stifle innovation and make it that much harder to use the JVM as a multi language VM.

I’m increasingly coming to the conclusion that multi language VM’s benefit from being as dynamic as possible. Runtime properties can be extracted to get performance, while static properties can be used to prove interesting things about the static pieces of the language.

Just let generics be a compile time feature. If you don’t there are two alternatives - you are an egoist that only care about the needs of your own language, or you think you have a generic type system that can express all other generic type systems. I know which one I think is more likely.

These days, reading the Goos book, by Steve Freeman and Nat Pryce, it reminded me of a project I worked on a while ago. It was a one year old system, poorly tested, integrating to a handful of other systems, and the code-base… well I prefer not to remember. Despite this scenario, I joined the team to help them implement some new functionalities.

I remember sometimes it was difficult to write tests, the classes were tightly coupled, with no clear responsibilities, several attributes, bloated constructors, etc. And despite our best effort, working around the bits that were preventing us from writing the tests, we felt we were getting down the wrong road, trying to do it in such a crappy code-base. As a result some of our tests were massive! A bunch of lines of mocks, stubs, and expectations, making it impossible to understand their purpose.

What have I learned?

Reading one of the book chapters I learned that the same qualities that makes an object easy to test also makes the code responsive to change. In my situation, the tests were telling me how clumsy the code was and how difficult it would be to extend it.

I also learned that when we come across a functionality that is difficult to test, asking ourselves how to test it is not enough, we also have to ask why is it difficult to test, and check whether it’s an opportunity to improve our code. The trick is to do it driven by tests, so we can get rapid feedback on code’s internal qualities and on whether it’s doing what it’s supposed to do.

So they introduced a variation for the well-known TDD cycle— “Write a failing test” => “Make the test pass” => “Refactor“. As described on the figure below (extracted from the book), if we’re finding it hard to write the next failing test for our application, we should look again at the design of the production code and often refactor it before moving on, until we get to the point that we can write tests that reads well.

Extracted from Growing Object-Oriented Software, Guided By Tests— Steve Freeman and Nat Pryce

An Example of a Smell Tests Might Be Telling You

Reference data rather than behavior

When applying “Tell Don’t Ask” or “Law of Demeter” consistently, we end up with a coding style where we tend to pass behavior into the system instead of pulling values up through the stack. So picking up the famous Paperboy example, before refactoring the code applying the “Law of Demeter” the code and test would look something along the lines of the snippet showed below.

class Paperboy
  def collect_money(customer, due_amount)
    if due_amount > customer.wallet.cash
      raise InsuficientFundsError
    else
      customer.wallet.cash -= due_amount
      @total_collected += due_amount
    end
  end
end

it "should collect money from customer" do
  customer = Customer.new :wallet => Wallet.new(:amount => 200)
  paperboy = Paperboy.new
  paperboy.total_collected.should == 0
  paperboy.collect_money(customer, 50)
  customer.wallet.cash.should == 150
  paperboy.total_collected.should == 50
end

We can easily see that the test is telling us it knows too much detail about Customer class implementation. We can see its internals, which objects it’s related to, and even worse, we’re also exposing implementation details of its peers. So it’s clear for me that it needs some design improvement. My main goal here is to hide Customer implementation details from the users of the Paperboy class. Which means that I don’t wanna see anything but Customer and Money classes referenced on the test!

class Paperboy
  def collect_money(customer, due_amount)
    @collected_amount += customer.pay(due_amount)
  end
end

it "should collect money from customer" do
  customer = Customer.new :total_cash => 200
  paperboy = Paperboy.new
  paperboy.total_collected.should == 0
  paperboy.collect_money(customer, 50)
  customer.total_cash.should == 150
  paperboy.total_collected.should == 50
end

The method customer.pay(due_amount) wraps all the implementation detail up behind a single call. The client of paperboy no longer needs to know anything about the types in the chain. We’ve reduced the risk that a design change might cause ripples in remote parts of the codebase.

As well as hiding information, there’s a more subtle benefit from “Tell, Don’t Ask.” It forces us to make explicit and so name the interactions between objects, rather than leaving them implicit in the chain of getters. The shorter version above is much clearer about what it’s for, not just how it happens to be implemented.

All the logic necessary to collect the money is inside the Customer object, so it doesn’t have to expose its state to its peers.

Now it’s safer to continue writing new failing tests to our objects.
Remember, listen to the tests!

Last year, I missed the announcement, and therefore the Agile Coaches Gathering UK organised by Rachel Davies (author of Agile Coaching). Fortunately I happened to join twitter late last year and saw the announcement for this year’s gathering shortly after its announcement. I bought a ticket as soon as I could. It’s a good thing too since I think I bought one of the last few tickets for a restricted sixty person gathering.

This year’s gathering, held at historic Bletchley Park spanned a day and a half with Friday evening opening the Open Space format to be used on Saturday. Bletchley Park ended up an awesome venue, and the open space format worked well with a number of spaces located outside. Given the UK actually had a summer this year, it turned out very nicely.

One of my favourite things about this sort of gathering is getting a whole bunch of people working in many different environments, contexts and getting them to share ideas and approaches about what works for them. We had a mixture of experienced coaches and coaches new to the role yet there was something about being there on a Saturday and a passion for it that brought people together. What follows are my notes from each of the sessions.

Experiences from the coaching discipline (not just agile practices)
My first session of the day explored other coach’s experiences working or exploring what coaches from other disciplines do. One person shared their experience working with a swimming coach and how they helped them get better at their practice.

This theme, looking at other coaching disciplines to see what they do, definitely ran throughout the day although I’m cautious of taking the role too literally. We discussed this during this session as most people being coached (life coaching, sports coaching, etc) tend to do so on a voluntary basis. Working as agile coaches for organisations, not everyone is necessarily there on a voluntary basis.

Another aspect that, I think differs, is that most coaches aren’t necessarily experts in the field they’re coaching in. For instance, sports coaches are often not people who’ve been the most successful in the world. In my experiences, agile coaches are really there to act as shepherds to help people get the most out of thinking and working with agile practices. Unlike other coaches, this often requires a level of mastery than what most other coaches have in their field. I’ve seen coaches act dangerously with not enough experience with agile practices.

I still think there is value in looking outside of our discipline for ideas and methods of working, yet conscious of the appropriate context to use them in (and there is always a context).

Presentation is not facilitation
Tobias Mayer is really well known in the Scrum community, and proposed a session to rant about the way that most training is done. I think there are plenty of examples where Death by Powerpoint is interpreted as effective training and whilst I agree with the premise, I’m not sure I agree with the end conclusion. I think presentations have a place, just maybe not in training. I also don’t necessarily think that training is solely about trying to trigger a complete mindshift in people.

When a coach gives up
I met Xavier Quesada Allue at XP2009 and again at Agile2009 so was interested to hear the premise around is it okay for coaches to give up and when to give up. This was one of those sessions located in the sun and some interesting stories about dealing with difficult teams or organisations where it doesn’t make sense to try.

We didn’t attempt to clarify, at the start of this session, what giving up meant to everyone. This probably made it difficult to get an agreement about when or why to give up, although we heard some very interesting stories.

My take on this, is that coaches end up needing to prioritise their work, just like everyone else and thus, not everything is going to get taken care of at the same time. Working with people also means you cannot predict how quickly people will move, or how they will react, therefore there is no way that you can always set completely achievable goals (it relies on others, not just yourself to make them happen). As a result, as a coach, you need to be comfortable with things not always going as planned.

I think that coaches also have the benefit of seeing the system that is driving a lot of the behaviour for the people on teams and unless something is done with that, then they will continue to behave as they always have. Both of these take time.

Game sense approach (what I learned coaching rugby)
Another inter-disciplinary coaching session that I attended briefly although a very loud bus made it difficult to concentrate and I ended up leaving because I had difficulty participating during this session.

Double loop learning + Defensive routines
If you ever get a chance to listen to Benjamin Mitchell in a safe environment, he’s quite the riot. If anything, perhaps it’s his self-deprecating and admitting his own faults and looking back at mistakes amidst his jokes that makes it so warming.

During this session, he talked about a book he’d been reading discussing how people react, and talking about different “models” in which the author classified people and using that to be able to project behaviour in different circumstances.

Once again, Benjamin reminded me of some key points I’ve learned over the years like:

• What you say and what people hear are completely different
• Avoid positional or solutions-based negotiation. Lay your interests on the table and you’ll often end up in a better position.

I’m not really clear about what double loop learning is, so I’m going to have to add that to my to do list as well.

New books

• Training from the back of the room
• The Skilled facilitator
• Situational leader
• Discussing the Undiscussable

曾经我认为，敏捷的各种实践，只要有了标准化的动作，加上一点点定制，加上PDCA/SDCA，就能做好。迈向敏捷之路，是可以唯一定义并且重复实施的。比如说持续集成，大师说 “在自己的计算机上启动一个自动化build”是重要的──我们把它叫做“本地构建”。做不好本地构建，提交构建失败率就会高，对持续集成的信心就会失去。这个问题，和它的解决方案，都是确定且可重复的。

可是这个团队让我吃惊了。他们一直没有真正意义上的本地构建。但他们真的相信持续集成──不仅仅是几个领导，是整个团队，真的相信：如果每天发现的小问题不能及时解决，那么到版本上线时一定会被客户骂得狗血淋头。他们不想被骂，他们也不想同一个战壕里的战友被骂，所以他们就把持续集成做好了。尽管没有真正意义上的本地构建，就靠着原始简单的代码评审，更认真的态度，他们就真的把持续集成做好了。

这个团队颠覆了一些正在我脑中渐渐成型的东西，让我想起了一些原本印象深刻但被渐渐遮掩起来的东西，比如敏捷宣言的第一条：个体与交互 重于 过程和工具。

面对巨大组织时强烈的无力感、无助感会让“敏捷推行人”们（包括我自己在内）不自觉地选择把自己的定位拔高──也许真是想帮助更多的人，也许只是想从残酷的现实中逃开，抑或两者兼而有之。但这些人不会做高层面的事情。于是离开低层面、离开具体事务的结果就是不知道该做什么，就是把敏捷推行简化为过程和工具的推行。

有一个敏捷推行人对我说，我负责几十个项目的改进，如果每次只到一个项目去做具体的事，对整体的帮助很小。我对她说，也许你救不了所有的鱼，但被你救的这一条鱼，它会在乎 ，它会得救。在反复思考如何拯救所有鱼的过程中，其实你什么也没有做，没有一条鱼真的得救。

敏捷的改变，最终是一个一个人的改变。过程和工具会帮助你。但如果把敏捷简化成过程和工具，你就在一步步远离敏捷，因为你已经违背了敏捷宣言的第一条原则。

曾经我认为，敏捷的各种实践，只要有了标准化的动作，加上一点点定制，加上PDCA/SDCA，就能做好。迈向敏捷之路，是可以唯一定义并且重复实施的。比如说持续集成，大师说 “在自己的计算机上启动一个自动化build”是重要的──我们把它叫做“本地构建”。做不好本地构建，提交构建失败率就会高，对持续集成的信心就会失去。这个问题，和它的解决方案，都是确定且可重复的。

可是这个团队让我吃惊了。他们一直没有真正意义上的本地构建。但他们真的相信持续集成──不仅仅是几个领导，是整个团队，真的相信：如果每天发现的小问题不能及时解决，那么到版本上线时一定会被客户骂得狗血淋头。他们不想被骂，他们也不想同一个战壕里的战友被骂，所以他们就把持续集成做好了。尽管没有真正意义上的本地构建，就靠着原始简单的代码评审，更认真的态度，他们就真的把持续集成做好了。

这个团队颠覆了一些正在我脑中渐渐成型的东西，让我想起了一些原本印象深刻但被渐渐遮掩起来的东西，比如敏捷宣言的第一条：个体与交互 重于 过程和工具。

面对巨大组织时强烈的无力感、无助感会让“敏捷推行人”们（包括我自己在内）不自觉地选择把自己的定位拔高──也许真是想帮助更多的人，也许只是想从残酷的现实中逃开，抑或两者兼而有之。但这些人不会做高层面的事情。于是离开低层面、离开具体事务的结果就是不知道该做什么，就是把敏捷推行简化为过程和工具的推行。

有一个敏捷推行人对我说，我负责几十个项目的改进，如果每次只到一个项目去做具体的事，对整体的帮助很小。我对她说，也许你救不了所有的鱼，但被你救的这一条鱼，它会在乎 ，它会得救。在反复思考如何拯救所有鱼的过程中，其实你什么也没有做，没有一条鱼真的得救。

敏捷的改变，最终是一个一个人的改变。过程和工具会帮助你。但如果把敏捷简化成过程和工具，你就在一步步远离敏捷，因为你已经违背了敏捷宣言的第一条原则。