Responsive Support

We have a couple of old apps with a development cycle like this: Business person calls developer describes problem. Developer codes and tests solution against the production database. Developer applies changes to production system. Developer checks in code changes. How many problems did you spot? No unit tests, no CI, no safegaurds against corrupted data, no roll back plan, yes, yes, yes. But the biggest problem of all? The business thinks this is great!

The business percieves that they:

1. Can request a change without requiring any analysis of the impact of the change or describing it in too great detail
2. Don't have to worry about justifying the change from a business value point of view
3. Don't have to test in those annoying staging environments which are too hard to maintain and setup
4. Don't have to worry about asking for the wrong thing - because they can just ask for something else straightaway
5. Don't have to wait too long for code changes to be deployed
6. Still have the right to complain if they find a critical bug

Essentially we've absolved them of the responsibility to carefully consider the change being requested, justify the change, and thoroughly test the change before it's deployed, whilst giving them a fast turnaround.

What's more, the developer also thinks this is great! They get to be a customer focused team player, with the respect of the business - these developers are often regarded as 'the best' by the business. Not to mention a certain amount of percieved job security because 'this system would fall over without me to maintain it'.

Are we servicing the business well with this arrangement? To some extend this depends on the level of developer doing the work, but in most cases a developer who allows this to happen is a C-level developer who thinks they're an A or a B and therefore 'the normal rules of development do not apply to me'. This, in turn, means the application code quality diminishes over time and as more changes are made, more changes become necessary to fix regression bugs. Eventually then, this 'fast turnaround', 'responsive' mode of operation builds up the technical debt of the application, it becomes legacy, harder and slower to maintain, and eventually needs to be replaced.

How do we turn this situation around? With difficulty! By the time you've got to this situation (or inherited it like me) it may be too late to save the code. If it's not too late it requires education of the business to understand the long term damage to the health of the application, and it requires education or (more usual) replacement of the developer supporting the application. It also requires someone with analysis skills to understand the business domain in which the application sits and the role of the application in the business process and to translate the business change requests into development requirements / test cases. All of this is perceived as a costly overhead - especially when 'we just call Bob and he fixes it now, but you want us to fill in forms and do testing...'. If it IS too late to save the code it may even be better to let the current sub-standard process continue and expend energy on the replacement system.

User stories are not a breakdown of a requirements document

I’ve been interviewing a few business analysts recently (yes we still need them despite having nominal business product owners). Many of them claim some agile experience in their past and manage to mention the words ‘incremental development’ but very few appear to think in terms of ‘iterative development’.

A brief word on incremental vs iterative here: Iterative development involves going back to already written functionality, reviewing it, and deciding what, if anything, needs to be done to improve it. Incremental development is developing a large system in pieces – this is not the same. Compare...

Incrementally building piece by piece with a clear upfront design

Iteratively improving on an initial vague design

 

(And just calling sprints 'iterations' does not make your process iterative!!!!)

I know back when I was a BA in a waterfall environment I tried very hard to think of everything during the requirements gathering phase so I could include it in the requirements specification, and a good BA should naturally do so. But in transitioning to an agile environment this often results in BAs simply ‘compartmentalising’ a traditional requirements document into smaller chunks and calling them User Stories.

So what’s the problem with this?

1. It removes the ability for the business to prioritise the functionality. Imagine a text search function – a BA may specify the multitude of different ways the search should return results given various common search operators (and / or / + / etc) in one user story. This negates the option to have a story for just a basic text search and a lower priority one for more advanced search options (the business may decide the benefits of the advanced search are not as great as another feature). 
2. It increases the chances of gold plating. Good BAs can always think of improvements that _could_ be made to software and they tend to put all that knowledge in one story. But the _best_ BAs will recognise which improvement has the biggest business benefit, and will separate the stories based on potential business priority, rather than a blanket statement like 'The text search should be like Google'.
3. It slows perceived progress. Instead of spending 1 day getting a basic search working and demonstrable to the end users, the developers have to spend a week getting the indexing and search engine customised, which in turn prevents them from delivering other features. The value of agile is the fast feedback mechanism it provides, the more we do to stay in this zone the better.
4. It makes you blind to opportunities. If you've already thought of the 'solution' at the start of the project, you're not going to look for alternative better solutions later on. That's why vagueness is sometimes a good thing - it encourages delaying decisions until the last responsible moment - the moment at which you have the most information available to make the best decision. E.g. 'with the feedback from users of the basic search we think we should add a date filter rather than improving the text search options'.

I encourage BAs and product owners in early sprints of a product to write stories that they will probably never release into production. By this I don’t mean that they should lack precision or functional coherence, but simply that whilst they may be basic enough to show how something could be delivered and can form a basis for further discussion on improvements required, they will probably need to add more ‘feature’ before release - or they may decide that the basic version is adequate of course!

[off topic] Carcassonne on the iPhone/Pad

Downloaded Carcassonne a few weeks ago on the iPad - the game of the year in 2001 consisting of tiles and wooden 'followers'. Like all the best games, deceptively simple yet very rich in game play strategies. In fact, I hadn't realised just how rich it was until playing the in built AI players. Initially, losing frequently to the 'Easy' players I eventually learned the strategies to overcome them and managed to get my level to at least on par with the strong players, without really understanding the subtle changes I'd made to my game play.

Haven't tried my new found powers on any humans yet, though I'm pretty sure I'm a better player now than a month ago.

The app also lets you play against human opponents or a mix of AI and human, and has a very nice 'solo' game too with completely different rules which is addictive. Highly recommended all round.

Scrum For Team System Product Cumulative Flow

If you've used the Scrum For Team System (SfTS) templates for TFS then you may have come across the Product Cumulative Flow Diagram which in my view is the most useful chart for a scrum project. The standard scrum process has only 'Not Done', 'In Progress', and 'Done' states. We extended our states to include 'Ready' - the state of a PBI which is ready for development but not yet in progress. We also split out 'In Progress' to indicate the 'stage' of the work - dev, review, test but this probably isn't always necessary.

With these additional states the product cumulative flow diagram radiates a lot of information. It shows the entire product backlog in terms of story point estimates and the state of the backlog over time. A typical (idealised) cumulative flow diagram is shown below for a completed project. This diagram gives an indication of velocity, product scope changes, effectiveness of the grooming process, and the size of the work in progress, all in one picture.

The project should start with a large backlog of ‘Not Done’ PBIs with a small set of these ‘Ready’ for development. During each sprint some of these Ready PBIs will be assigned to the sprint and will be worked on, setting them to ‘In Dev’. Ideally within the sprint these will move to ‘In Test’, be tested, and marked as ‘Done’. Sometimes work in progress will be carried over to the following sprint. At the end of each sprint some new PBIs will be added to the backlog during the sprint review. Within each sprint some time should be dedicated to ensuring there are enough ‘Ready’ PBIs for the following sprints to work on, this may require breaking down large PBIs and re-estimating the pieces. SfTS in TFS can generate the cumulative flow diagram automatically, and this can be used to indicate potential problems with the project. For example:

1. PBIs not being created or estimated early enough in the release – the curve should rise steeply at the beginning and then level off with one or two stories being added at sprint review, and fluctuations in the size of the project when large PBIs are broken down into more detailed ones. Towards the end of the release the height may even reduce if PBIs are de-scoped from the release 
2. If testing is delayed (not performed within the sprint) then the ‘In Test’ work will accumulate – if this happens expect a late surge of PBIs / Bugs when the testing does start 
3. The diagram should display obvious cycles – In Progress / In Test work should approach zero at sprint boundaries 
4. At minimum the rate of items moving from ‘Not Done’ to ‘In progress’ should be matched by the rate of items being moved from ‘In Progress’ to ‘Done’ – testing should keep up with development. 
5. If too little grooming is being done the Ready PBIs will drop close to the In Progress line 
6. If too much grooming is being done the Ready PBIs will rise up towards the Not Done line – ideally there should be between 2 and 3 sprints worth of Ready PBIs at any one time.

Unfortunately, this chart is not available with the Visual Studio Scrum and MS Agile templates for TFS, though of course you could write your own.