Software life spans

All software has a life span – same as humans. In general a software’s life is much shorter – how much software is actually in use 10 years after it was written? – but it follows a similar pattern. In the childhood years it is nurtured and cared for, and grows relatively quickly, and is the proud achievement of its creators. Then for a while it stands on its own, though still receiving regular attention, getting minor improvements made, and generally being kept healthy. Slowly though it falls into a state of disrepair, it may have lost some of its former gloss, and parts may have fallen into disuse. Finally it is decommissioned, replaced with a newer solution.

My belief is that once software gets past middle aged there is no way back. It has become legacy code and the chances of restoring it to a healthy state are minimal. Unfortunately, some code transitions to legacy very quickly (often before it has even made it to production). Code transitions to this legacy state for a number of reasons:

1.  The software is written in an obsolete language / framework which very few good developers would want to work in (good developers choose the new technologies), resulting in C level developers maintaining the system. 
2. The language / framework is fine, but the architecture is brittle. Few good developers want to polish turds, resulting in C level developers maintaining the system. 
3.  The architecture and language are fine but there are no automated builds, deployments, tests, comments, or documents explaining what it does. Good developers will work on this software, but prefer to work on new software, resulting in level C developers maintaining the system. 
4.  The architecture and language and automation are fine, but the application is large and complex making it difficult to extend without a lot of investigation. Good developers will work on this software, but prefer to work on new software, resulting in level C developers maintaining the system. 
5.  There are no good developers in your team because you have no new project work, resulting in C level developers maintaining the system. These developers have worked on the software for long enough to leave it in worse shape than they started.  The slippery slope… 
6.  The software was written entirely by C level developers in the first place. It is a tangled ball of mud and is legacy before it has even gone live.
7. Software is considered ‘complete’ by the business rather than being reviewed regularly for improvements. Eventually, the business finds that the system cannot be modified efficiently and accurately enough to support their changing processes 

We (development teams) need to address these issues. Let’s look at them one by one:

1. Ensuring modern languages and frameworks are used is relatively easy, though we need to invest more time in planning upgrade paths for software – the progression of .Net Frameworks has been fairly straightforward migration (except for 1.0 to 1.1 in the very early days). Almost all modern languages share a common DNA and many frameworks are available in different flavours.
2. A clean architecture and understandable design is basic coding practice, but all too often solutions are plain hacky. Using NDepend and similar code design inspection tools can help assess existing architectures, but really all that is required is some smart thinking and regular code reviews for compliance. 
3. All modern apps should be built using TDD / BDD, CI, with high code coverage and appropriate levels of commenting and documentation. Integrated wikis for both users and tech staff should be mandatory. 
4. Modern apps are complex, but we often make them more so by coupling the internals of the app needlessly. Designing apps as composites of services can decouple the component parts leaving each one more understandable to the developers charged with maintaining them, and more easily modified individually. 
5.  Hire better developers – or at least only use C-level developers on project work with good developers. 
6. See above
   
7. Create a culture of continuous improvement – state in every meeting you have with the business ‘Good Software is never complete’ – there will always be a way to effectively and efficiently extend it to bring more benefits to the business. This is not a negative statement ‘you’ll never see the end of this project’, it’s a positive statement ‘this software will be a long term partner for the business and will grow and evolve in ways we can’t even imagine right now’ 

Legacy code is a curse for an organisation. It demotivates A level developers making them more likely to move on, and provides a safe haven for C level developers making them more likely to stay. The slow downward progression of skills in the dev team makes it more likely that more code will become legacy which exacerbates the issue.

C-Level developers

I’ve heard, and I think it’s true, that the best ‘A’ developers hire other A level developers. Whereas the next best B developers hire C level developers. Whether this is because of pride, fear of being challenged, belief that the C levels best fit the needs of the company I’m not sure, but it is a real problem that can result in entire teams containing no A level devs and very few Bs (I’ve inherited a team like this before and it’s a real challenge). 

Part of the problem is that most C and B levels believe they are As, in fact self-awareness of what a developer does not know is a good sign that they’re advancing. So what’s the definition of the levels? In basic terms it boils down to how they affect the software on the spectrum of code quality. C developers will tend to leave code in a worst state than they found it, whereas A developers will always improve it.

1. C developers will forget to comment, will add unnecessary coupling, not develop all the tests they need, avoid refactoring, will adopt cut and paste coding, add more bugs, not test their own work. Essentially they will produce huge technical debt whilst patting themselves on the back when they deliver a piece of functionality on time.
2. B developers will generally follow standards but are not able to pin point software issues which give them an uneasy feeling
3. A developers will identify problems left by C developers and will take the time to fix them.

Over time, with enough C level developers working on the software, the technical debt burden becomes so high that adding features is prohibitively expensive and the software has moved into a legacy state. At which point a new project is started (hopefully using some A developers). What is the solution to this issue? Only hire As and Bs – better to not hire at all than to hire Cs. Or if you are hiring Cs make sure they have some recognition of their limitations and ensure they understand that they won't be praised for speed at the expense of quality. 

What if you don’t have the technical skills to hire the As? They are not actually that hard to spot – they will talk more about their craft than about what the software that they wrote in past jobs actually did. They will want to tell you about the architecture of the distributed message queuing system they implemented rather than the fact that $4gazzillion of transactions were processed daily. They will want to tell you about the automated build processes they introduced to the team rather than the order processing screens they put together. They will be able discuss technical debt and continuous integration rather than giving you blank stares.

Btw, looking (a long way) back to my development days I can recognise the traits of the C level developer in myself – ‘the quick solution is best!’, ‘the client was really happy that I got that functionality out on time’, ‘never mind that was a copy paste of another screen with a few changes’, ‘this change is simple, I’ll just make it in production and fix up source control later’. Which is obviously why I became a manager... ;-)

Grooming the Backlog

Grooming the backlog was a concept I first heard attending a Scrum Master training course and at first felt like an appeasement to those non-agilists who ridiculed the no-forward-planning approach espoused by the scrum practitioners. However, we’ve found it very useful to focus a small amount (5%) of time in each sprint to looking into the future to

1. Make sure we have enough well defined (Ready) work in the backlog for the next 2-3 sprints
2. Reprioritise the items in the backlog
3. Break down large PBIs into smaller ones
4. Remove any duplicates
5. Re-estimate any new or large PBIs

Each one of these activities are quite distinct and require different resources and skills and so we do them collectively with the entire scrum team, scrum master, and product owner present.

We normally have a grooming 2-3 hour meeting per sprint – usually in the middle of the sprint since all the other meetings occur at the sprint start or end and it’s good to spread these things out. In the meeting we review those PBIs marked as Ready and re-affirm that we’re happy with them, then assess whether we have enough Ready PBIs (2-3 sprints worth), if not we’ll ask the product owner to identify some other stories with high priorities and start fleshing out the conditions of acceptance until the team is happy that the detail is sufficient that we could start development. We’ll then review the point estimate – which may have gone up or down depending on the clarifications made.

Some of the PBIs selected by the product owner may have been epics in which case we’ll break these down into smaller components, usually leaving some in a more vague ‘not ready’ state and only focusing on the higher priority items. Over time this process can result in some PBIs expanding as they are refined which can often result in discovering a lower priority PBI sitting in the backlog which has actually been done as part of the expanded PBI, for this reason every couple of sprints the scrum master and product owner review the backlog to identify any duplicate PBIs which could be removed. The product owner may also remove any PBIs which he or she feels is simply no longer required.

Often I hear of agile teams who define the PBIs for a sprint at the start of each sprint. This probably works well for longish sprints for mature products with maintenance releases at the end of each sprint but if you’re working on a long term project and need some picture of the current scope maintaining and grooming a backlog is essential.

Scrum definition of ready

In order to maximise the efficiency of the sprint planning meetings it is important to frequently review the list of Not Done Product backlog Items (PBIs) and, where required and / or possible, add clarity to the information. This process is called Grooming the Backlog and should be done by the Product Owner and team together. During grooming the PBIs will be moved from ‘Not Done’ to ‘Ready’. During the sprint planning meeting we restrict ourselves to only choosing PBIs that are 'Ready' for development.

Roughly 5-10% of the time on the project should be spent grooming the backlog. This can be done by anyone at any time, but often it is worthwhile booking meeting time to do it explicitly if the project requires it (an experienced Product Owner who actively grooms the backlog themselves may negate this need). Note that not all of the backlog should be groomed, in fact it is just as important to not over groom the backlog – spending too much time refining requirements for distant sprints increases the risk that these requirements will change in the meantime.

Each project may create its own definition of ‘Ready’, however a suggested minimum is as follows:

1. Each PBI should be worded as a function with business value and with a clear ending – e.g. ‘As an applicant I can create a resume online using a template so that my resume looks professional’ rather than ‘As an applicant I can manage my resume’ (too vague)
2. Each PBI should have conditions of acceptance defined indicating how it will be tested. This should include negative tests – e.g. phone number must contain digits only
3. Each PBI should include ‘non functional’ details such as user permissions, logging, performance requirements
4. Each PBI should have an estimate in story points, assigned by the development team (devs + BAs + Testers) using planning poker techniques
5. Each PBI should be small enough to comfortably fit in a sprint. Each project may decide on a cut-off for story point estimates – e.g. anything greater than say 8 needs to be broken down further before being considered ‘Ready’
6. If applicable, each PBI may provide lo-fi mock ups / diagrams / brief documentation of the anticipated functionality. However avoid too much design work up front.
7. Both the Product Owner and team must agree that the PBI is ready

Handling bugs in Scrum

I'm sure the title of this post would make a few Scrum zealots heads spin, steam coming out of their ears, etc. However all the agile TFS templates have Bug work items, implicitly acknowledging that most teams would need to use them. So how do bugs fit into a Scrum process? I'll start off by appeasing the zealots saying that in an ideal world we wouldn't raise bugs in a scrum project. PBIs should be sufficiently small and well groomed for the team to build, test, and mark them as done within the sprint boundaries. If any discrepancies are found during the sprint they should just be reworked and retested. It's not worth raising bugs for work in progress.

However, if bugs are found relating to code that was marked as Done in an earlier sprint then a Bug can be created in TFS to schedule and track the fix of the bug in a future sprint. Note that it is worth making sure that the Bug is not just an enhancement in disguise (if so it should be raised as a PBI). One principle of agile is to have a potentially shippable code base at the end of each sprint. This doesn’t necessarily mean the code has to be bug free – many bugs have acceptable workarounds, which can be managed through user training / documentation. Therefore bugs raised do not automatically get added to the next sprint, though reviewing the current bug list is always a good place to start during sprint planning. So a bug in one sense is just another PBI in the backlog which can be prioritised up or down along with the rest.

Should Bugs be given story point estimates? There are two arguments here. Firstly a genuine bug indicates that we need to perform so rework on code we previously had considered to be complete. Therefore every bug should be assigned a 0 point value since we’re not really delivering any more functionality and having a point value would taint our project data (make us look more productive than we really are).

On the other hand, regardless of whether developers are working on bugs or PBIs we still want to be able to measure their velocity from sprint to sprint – how much work, on averge, can be completed in a sprint. We can also measure the total points for PBIs vs Bugs to give an indication of rework and a prediction of future rework for the incomplete PBIs in the backlog.

My recommended approach is to give all Bugs point estimates in the same way as PBIs (and using the same relative scale) but to be aware when calculating metrics such as hours per story point that the bug story points should be removed from this calculation (but the hours to fix the bugs included).

Also, since Bugs are generally smaller pieces of work compared to PBIs it is sometimes worth setting a minimum value on PBIs when initially estimating the backlog to leave ‘room’ for some smaller Bug estimates later – e.g. all initial PBIs are estimated at 3 points or above. The alternative is to group a number of bugs together (usually ones relating to the same feature/page/component) to make a larger PBI, but I dislike this approach since it prevents bugs from being prioritised separately.

Scrum is a Mirror

I recently organised Scrum training for the internal development and PMO team here at Nine Entertainment, consisting of two days of Srum Master training for 17 project managers / lead developers and 1 day of effective user story writing for around 20 business analyst types. One of the things that the trainers said early on is that 'Scrum is like a mirror' highlighting the best and worst things about your product and development process.

Now, at first I didn't really grok this at all, but after mulling it over I am coming to the realisation that this is true. Retrospectives if done regularly and correctly highlight issues with your process very quickly - whether this be technical design problems, lack of responsiveness by the product owner, too much micro management by the scrum master, they can all come quickly to the surface and solutions be discussed. But it's not just the process that has a mirror held up to it's face, the product gets the same treatment every sprint review. The stakeholders get to answer the question 'is this the product we wanted?' time and again.

Compare this with a waterfall approach and the difference is clear. Lack of engagement by a product owner - we'll just make sure we document everything. Stakeholders not sure what they want - we'll force a decision and stick to it. Not sure if the design is up to scratch - don't worry it will come out in testing. Waterfall avoids looking at the process or product too closely and instead reassures itself by looking at deadlines and budgets - if we're meeting them we must be fine. No matter if we build the wrong product for the wrong people within the allocated budget and timeframe

Scrum Training

This week we held certified scrum master, 'Writing Effective User Stories', and 'Scrum Primer' courses for about 40 staff which I'd organised through Rowan Bunning at Scrum With Style. We had the benefit of having two trainers for most of the sessions with Simon Bennett in town for Agile Australia. Very interesting and enlightening sessions even for those, like me, who thought they pretty much knew it all. I'm sure it will generate a few posts i nthe coming months as the lessons sink in...