Monthly Archives: February 2014

Succeeding with Agile Development – Part V

In this six-part blog series, Shirley provides practical advice on adopting Agile Development methodologies.

optimizing GGB

Agile Development is a software development methodology based on incremental, iterative development, where requirements and implementations evolve through collaborative, self-organizing, cross-functional teams. Agile Development promotes product delivery as a time-boxed series of functional and potentially shippable product increments that enables teams to rapidly respond and adapt to change.

Part V – Optimizing

Backlog Grooming (or Backlog Meet and Greets)

When teams begin to adopt Agile practices, they are often faced with sizing and committing to stories during sprint planning – when they hear about the stories for the first time. This can lead to onerous Sprint Planning meetings (where the team often loses focus and reverts to any means to get the meeting over with), unclear requirements, increased implementation risk and over- or under-commitment (depending on your team’s propensity for risk.)

While this is inevitable in the beginning, the key to Agile success is to begin to develop a ‘story horizon’ that’s longer than a given sprint. Ideally, your sprint horizon should be either a release (a subset of stories determined to meet a particular customer or market need that justifies a release announcement) or 2-3 sprints. Grooming the backlog beyond this will often lead to discarded work as requirements change or evolve.

If your planning horizon is a release, I recommend taking 1/2 day for a Release Kick Off meeting at least 2 sprints prior to beginning the implementation. During this meeting (ideally held offsite) the Product Owners will walk the whole team through the goals, features and expectations of the release. This may include performance or scale metrics that must be met (e.g. must be able to handle n concurrent users or x,000 transactions/second). Once the team understands the overall objectives, the Product Owners will walk the team through the backlog as a series of high-level stories. The team will seek clarification and discuss possible implementation approaches. This meeting should not seek to fully understand or to size the stories, but rather to establish common ground and to uncover issues or questions (technical, requirements, etc.) that need to be addressed before a story can be sized.

Once the Release Kickoff meeting is complete, or if you are just working through a backlog of stories and releasing incrementally, you should plan to spend at least 1 hour per week grooming the backlog. During this time, the team will begin with a brief high-level review of the backlog before discussing and sizing (previously unsized or deferred) stories from the prioritized backlog. If issues arise that prevent a story from being sized, it will be deferred and flagged for the next grooming meeting.

Backlog grooming is an incredibly worthwhile exercise that sets aside time to ‘meet and greet’ upcoming stories in advance of their implementation. It provides developers and QA valuable time to consider upcoming stories, to discuss ideas offline and to improve the quality of their estimates. Schedule a fixed day/time every week for backlog grooming and make it a priority for the team (I prefer Wednesday mornings – since they avoid Sprint Planning and wrap up activities.) Despite their protestations of ‘too many meetings’ the team will quickly appreciate and value the impact of the investment.

Story Sizing

In traditional, waterfall, processes schedules often comprise developer and QA estimates (measured in hours or days). Data is then plotted on a Gantt chart that includes a contingency factor (%age overhead, based on prior experience of the team’s sustained productivity), vacation days, holidays, etc. Progress against goals is measured in %age complete vs. estimate.  Also, development time is often estimated individually per feature and separately from QA – with little or no accommodation for issues that will arise during integration and testing. This approach is fraught with difficulties that often lead to schedule overruns. One example is that when asked for an estimate of  %completed, developers often give an answer that is better described as %time spent: “I’ve spent 80% of the time, so I’m likely 80% complete”. This often has little or no bearing on the actual level of completeness (remember the old 80-20 rule?) and introduces the risk of discovering schedule overrun late on the project.

Agile follows a timeboxed approach, where the team makes a commitment to completing product increments (stories) within a given timeframe (see Establish a rhythm for the team).This is a significant shift in thinking for many teams and is often hampered by them regressing into traditional estimating and trying to plan the sprint duration using traditional methods.

To shift the team’s thinking, I recommend introducing the concept of relative sizing. That is, estimating the size of a story  (including everything required to deliver a functional product increment; Development, QA, Documentation, etc.) relative to other work. To get teams started, I  introduce relative sizing using a number series that closely resembles the Fibonacci scale (1/2, 1, 2, 3, 5, 8, 13, 20…) ). As a point of reference, I associate the smallest increment (1/2 point) with some piece of work that everyone in the room can grasp. For example, ‘1/2 point represents the work required to make the smallest change to an application’: think changing a label on a UI. The 1/2 point sizing includes code changes and check-ins, developer testing, QA setup and testing, and refreshed screen shots. The size of every other story is then calculated relative to this.

Progress is then measured during the sprint in completed stories (those that are ‘done’.)

Planning Poker

Planning Poker is an estimating exercise that helps to maintain the objectivity of story sizing by reducing the influence of any single member of the Scrum team on the sizing or others. You can purchase Planning Poker Cards inexpensively, or download a Planning Poker App. Just make sure that everyone committing to the sprint has the ability to contribute their size estimate (even if it’s just using a piece of paper with the number sequence, or fingers and toes!)

The team collectively discusses each story, seeks clarification from the Product Owner and formulates a high level approach to deliver the product increment. Technical or other risks are discussed and QA voices how they will acceptance test the feature. The Scrummaster then asks everyone to estimate the level of effort (using relative sizing) to complete the story and every member of the delivery team privately selects a ‘card’. I like  teams to countdown “3, 2, 1” to ensure that everyone is ready then ask everyone to hold up their ‘cards’ at the same time. Typically, there will be a lot of consensus in the room on the size of a given story. However, it’s important to look at the outsiders. For example, if the bulk of the team estimates 8 points but one member holds up a 20 and another a 2, there’s likely something interesting to pursue. Ask open-ended, non judgmental questions of the ‘outsiders’ “That’s interesting, why a 20?” to understand their reasoning. In my experience, their reasons are often very valid and affect the resulting points sizing. In the example, it could be that the extent of testing has not been considered by the bulk of the team and that the size needs to be increased to accommodate the additional work. Or, the effort to implement could be much smaller because it can leverage work thats already been completed that perhaps only one or two people are aware of. In either case, take the time to hear each of the perspectives, have the team discuss and re-size the story.

The insights gained during planning poker are often really valuable and will improve the quality of your estimating. The key is to remember that Planning Poker is not designed to force everyone to a common score, but to gain a common understanding of the work involved and to base the size on their best collective knowledge.

Identifying Latent Impediments

While the Daily Stand up is intended to uncover issues that are impeding progress (“do you have any impediments?”, it’s often not as simple as asking the question.  Team members always want to show that they are doing their best to demonstrate progress. Admitting to being ‘stuck’ or ‘unclear’ can amount to feeling vulnerable, like they are letting the team down (this is especially true in nascent teams.) It’s the Scrummaster’s responsibility to uncover these ‘latent impediments’ and to help determine a path forward.

Key indicators of latent impediments are:

  • Tasks not moving across the board (remember, tasks are estimated to be 1 day or less – if they are not moving daily, you likely have an issue)
  • Developer or QA giving the same update in two or more scrums, often with the words “I’m about to reach a breakthrough”

When a latent impediment is identified (or even suspected), the Scrummaster can ask for a team member to help with  one or more next steps to accelerate a breakthrough:

  • Pair programming
  • Whiteboarding
  • Revisit Task Breakdown (it may be that issues or ideas have emerged since the original task breakdown that affect the implementation approach)
  • Code review

While it may appear that the Scrummaster does not ‘trust’ the process, the opposite is actually true. The process is highlighting the fact that a problem is building. Addressing it quickly will help keep the sprint on schedule and will continue to engender teamwork, trust and openness.

QA Automation

As a product matures, the effort to regression test existing functionality increases with every release. Manual (black box) regression testing becomes increasingly labor intensive and prone to error. Like Unit Testing, QA (functional) test automation will pay significant dividends and will allow your limited QA resources to focus on usability and ‘feel’ of the product, rather than performing repetitive testing.

I recommend using a scripted, data-driven test framework like Selenium or Watir to give you maximum flexibility.

Begin to make progress by adding automation stories to the backlog, prioritized based on risk and ‘bang for the buck’. That is, start with functional tests for the most fragile areas of code (those known to break often) or the most cumbersome parts of the regression testing (multi-browser, multi-setup). The key is to make at least some progress on QA automation every sprint to build out a portfolio of automated regression tests that can be run as part of every build to identify functional bugs at the earliest (and cheapest) opportunity.

Investing in optimizing your Agile process will result in greater team confidence and reduced delivery risk. It will necessarily require a commitment of time that will impact your team’s velocity initially. However, the increase in clarity and common understanding will more than compensate in the results achieved.

And, while you are implementing new ways to increase the productivity of the team, take a moment to reflect on how far you’ve all come. You’ll be pleasantly surprised…and it’s another reason to celebrate!

Succeeding with Agile Development – Part IV

In this six-part blog series, Shirley provides practical advice on adopting Agile Development methodologies.

ggb big picture

Agile Development is a software development methodology based on incremental, data-driven development, where requirements and implementations evolve through collaborative, self-organizing, cross-functional teams. Agile Development promotes product delivery as a time-boxed series of functional and potentially shippable product increments that enables teams to rapidly respond and adapt to change.

Part IV –  The Big Picture

As you continue to adapt to the Agile process and focus the team on getting stories and tasks to ‘Done’, it’s equally important look at the ‘big picture’. This will help identify issues early (possibly even before the team knows themselves) and help you to steer the sprint and the team to success.

Swarming

Once Sprint Planning and Task Breakdown are completed and the post-its are on the Scrum board, it’s time to start the Sprint.

In my experience, Engineers want to work on the ‘next new thing’ and a fresh Scrum Board full of untouched stories can look like fertile ground. This is where you need to follow some simple rules:

  1. Swarm. Encourage the team to assign as many developers, designers and QA to each story as is reasonable (based on how well the work can be parallelized).

  2. Put stories on the board in priority order and make sure your team knows to open stories top down – not cherry pick their ‘favorite’ stories.

  3. Never open more stories than you have developers on the team. In fact, encourage the team to allocate at least 2 developers per story so that they can collaborate, pair program and solve problems together to expedite story completion.

Sprint Health

Since the goal of a Sprint is to deliver a potentially-shippable product increment, it’s important that stories are completed prior to the Sprint Review.

An early indicator of the wellbeing of a sprint is how the Scrum Board looks overall. While it’s important to track the progress of each story, it’s equally important to take a moment every day to step back and look at the board in its entirety.

A healthy sprint will show some subset of stories Done (or on their way to Done), some that are In Progress and others that are not yet open. Since the  progress should reflect the priority of the stories, the general trend should be in the form of a diagonal line across the board (see Fig 1). An unhealthy sprint will have all (or the bulk) of the stories open and in a partially complete state (see Fig 2)

Entering the final week of a sprint with too many stories open will necessarily prevent the team from getting to ‘Done’. It will lead to QA or code debt being carried forward to a future sprint and will impact the team’s ability to take on new stories.

Closely manage the team’s opening of stories. If your scrum board begins to look like Fig 2, pull the team together immediately and strategize on ways to get back on track. These could include backing out committed code (that cannot be QAed prior to the end of the Sprint), halting progress on lower priority stories and/or seeking assistance from another Agile team.

Fig 1: Healthy Sprint Scrum Board (diagonal line indicates overall progress)

Fig 2: Unhealthy Sprint Scrum Board (vertical line indicates too many open stories)

Continuous Integration and Unit Testing

It’s never too early to start investing in unit testing. Identifying and fixing bugs at the most fundamental (unit) level will pay huge dividends in overall product quality. ‘Building it right the first time’ (with unit tests) will enable your teams to focus their creative energy on delivering new features, not fixing old code. In a nutshell, they will be able to constantly take on new challenges and have more fun.

Of course, it’s unlikely that you (or any company) can afford to stop all development to go back and write unit tests for their existing code base. Instead, make a commitment that 100% of new and modified code will have unit tests and make it part of your process.

First, choose one or more  unit testing frameworks (e.g. JUnit, PHPUnit, xUnit) depending on your code base.  Create and prioritize one or more ‘developer’ stories (As a developer, I would like to…) to deliver a working framework and a set of illustrative unit tests that can be used as a reference by other developers. Work with your team to identify two developers to lead the unit testing effort (ideally, they will have a passion for or experience with test driven development) and time box the first deliverables to a single sprint. Ask the developers to pair program the framework and the initial tests. Like feature development, this approach will yield a higher quality result and will uncover issues that may be overlooked by a single developer. It will also create the inertia needed to get things off the ground.

Have the Unit Test leads host a brownbag to walk peer developers through unit testing  (why, how and when) and review the framework and existing tests.

Add a column to your scrum board  between ‘In Progress’ and ‘Ready for QA’ that is ‘Unit Test Dev”. Tasks should not move to QA until the associated unit tests have been written or modified to reflect the new functionality.

Note: Keep a close eye on unit tests that are commented out or disabled. If a unit test breaks when code is committed, it’s a pretty good sign that the check-in has broken something fundamental. Developers must modify the code and/or the unit test until the unit tests run successfully – and not be allowed to disable failing tests.

Continuous Integration (CI)

Building unit tests without a continuous integration (CI) tool will likely cause your unit testing investments to fail. Establishing a CI tool that runs every unit test against (at least daily) builds will reinforce the value of the investment and avoid unit test atrophy.

Select and implement a CI tool ( popular tools include Bamboo, Hudson, and CruiseControl). Identify someone in the Engineering team (ideally, they will self-select) to take responsibility for ‘Build and Release’ tools.

Setup your CI tool to execute the full suite of unit tests after every commit/build, or at least once daily. Report build and test failures to the team and list recent check-ins to help identify which one is responsible for the failure. Make it the team’s priority to ‘fix the build’ before continuing to work on open stories or tasks.

Measure your progress

While code coverage is a useful metric, it can be disconcerting for a team starting out when few, if any unit tests exist (i.e. starting at 2.7% code coverage can make it feel like a steep climb ahead). Instead, begin by measuring and reporting the total number of unit tests – and assess progress based on %increase each sprint.

However, don’t rely solely on the numbers. Using a strictly quantitative assessment, can incent the wrong behavior, with developers creating  unit tests to achieve the metric rather than creating effective tests.

It’s important to combine the quantitative unit test metric with a qualitative assessment by a peer developer. I recommend having developers pair program their first few unit tests with the Unit Test lead(s) and performing regular code reviews on unit tests.

Every team has to start somewhere and, by making unit testing a priority, you will reinforce the value and begin to see the investment pay off in higher quality code.