Monthly Archives: March 2014

Evolving Beyond Scrum

optimizing GGBWhile I recommend following the core tenets of Scrum Agile, I have designed and implemented extensions to the Scrum process that increase the effectiveness of Agile development  in product companies and reduce overall risk.

If you have already adopted Scrum Agile, you will be familiar with the following:

  • Self-organizing, cross-functional teams
  • Scrummaster to oversee and facilitate the process for all the teams
  • 2 (or at least, fixed) week Sprints
  • Relative sizing
  • Sprint Planning and Commitments
  • Breaking down committed stories into tasks of 1 day or less
  • Iterating development deliverables to QA in ‘QAable chunks’
  • Tracking status with physical and/or technology scrum boards and daily standups
  • Steering Sprints to a successful conclusion
  • Hosting Sprint Reviews and Retrospectives

Adapting for improved effectiveness

While the core principles of Scrum are highly beneficial in their own right, companies developing products or solutions that have to meet rigorous goals of scale, performance, backward compatibility, quality, etc. need to evolve their Agile practices to incorporate techniques that provide a basis for architectural evolution, design consistency, and team optimization.

Needs more information or NMI TM stories

Needs More Information or NMI stories are an extension to the Scrum process that account for research, architecture, design and prototyping activities that are often necessary before a large story or group of stories can be effectively sized. By spawning an NMI during Story Sizing (Backlog Meet and Greet) and prioritizing it high in the backlog (typically for the next sprint), Agile teams can increase their confidence that the implementation story (or stories) can be committed to and delivered upon.

NMI’s are expressed as a series of questions that need to be answered before the related story (or stories) can be sized.  NMIs are typically small in size relative to the original story or group of stories and can employ technical prototypes, white board sessions, architecture reviews, technical research, etc. to answer the open questions.

NMI deliverables include

  • Additional story breakdown (if the work is able to be broken down further)
  • Sizing of the story or group of stories
  • Task breakdown

Once the NMI is complete and the related story or group of stories are sized, the team can confidently commit to delivery based on their velocity in the Sprint Planning meeting.


In an effort to ensure a potentially shippable product increment at the end of each Sprint (and to optimize the team’s efforts) it’s important that all members of a ‘story team’  stay on the story until it’s ‘done’. This can at times result in team members being idled for periods during the sprint.

In the event that this is a developer, the last thing you want as an Engineering leader is for them to open another story and start checking in code that may or may not be QA-ed by the end of the sprint (since it will lead to more technical debt.) Instead, have your Developers, QA and Designers each create and manage a Slacklog. A Slacklog is a list of items that an idled team member can work on that does not create debt for anyone else. For example, writing or improving unit tests, prototyping with new technologies, writing QA automation tests.

While at first blush, having your team ‘idled’ during a Sprint seems counter-intuitive to productivity, the opposite is actually true. Creating some ‘pain’ in the process will help it improve. No-one likes to be idled, especially Engineers, and the team will find ways to optimize their efforts. For example, if Developers are idled because they held onto all the tasks until the story was almost complete – and now are waiting for QA to finish – they will seek ways to better phase delivery of the work. If QA Engineers are idled because they are waiting for something to test or for bugs to be fixed on a story, they and the Developers will be motivated to invest in unit testing and QA automation scripts. Trust me, it works…

Handling Unplanned Escalations 

In an ideal situation, Sprint Planning and execution would proceed without any surprises – and all committed stories would be completed in the Sprint. However, we live in a (necessarily) imperfect world. One where customers, partners and the vagaries of software and infrastructure often lead to unplanned escalations that the team has to deal with.

“Above the Line”

Rather than build in a bug fix contingency, I prefer to create a section at the top of the scrum board to track and manage items that are introduced after Sprint Planning. I call this section ‘Above the Line’. Stories, tasks and bugs can only be introduced to an active Sprint by the Product Owner or Scrummaster and are added ‘Above the Line’.

By following this model all unplanned items introduced after Sprint Planning will be very visible, allowing you to measure the team’s actual velocity, quantify the impact of unplanned work on committed stories, and most importantly, look at where the exceptions are originating from in order to reduce them.

Managing ‘Above the Line’ Work

There are two approaches I have found to be effective in managing unplanned work in a Sprint. The approach you choose will depend on the complexity of your product, the degree of shared code knowledge and your team’s ability to switch gears.

  1. “Swarming”

Leveraging the entire Scrum team to work on escalations carries the implicit understanding that work placed ‘Above the Line’ takes precedence over committed stories and could cause one or more committed stories to be dropped from the Sprint.

With this approach, ‘Above the Line’ items fall into two categories and should be treated differently to reduce context switching:

  • Critical – Fix Immediately: Developers and/or QA should drop whatever they are working on and immediately swarm to find, implement and test a resolution.
  • Fix In This Sprint: Developers and/or QA continue to work on their current tasks. As they complete a task and move it across the board, before they pick up the next task for their current story, they look to see if there is a task ‘Above the Line’ that needs to be picked up.

2.  “Assigned Resources”

Another approach to handling escalations is to assign ‘Above the Line’ to a small ‘swat team’ for the duration of a Sprint. In essence, you reduce the velocity of the team to allow for escalations to be handled.  While this approach dramatically reduces ongoing interruptions on the Story team, depending on skillsets/experience, it may require pairing with Story developers for knowledge transfer. Also, it’s likely that the ‘swat team’ will be idled at some point during the Sprint, creating an opportunity to work on Slacklog tasks.

Agile for Desktop IT, Build and Release, and Technical Operations

Teams that have a high degree of interrupt-driven work every sprint (a big part of their responsilbities is responding to exceptions), such as Desktop IT, Build and Release, or Technical Operations benefit from planning with a reduced velocity.

Begin by tracking and sizing all the ‘Above the Line’ items and maintaining an open mind to what was committed. When your team has completed 3-4 sprints, average the counts for ‘Abolve the Line’ and Story points. This will provide you with a reasonable estimate for the #interrupt points per Sprint and #availablevelocity. When Planning, use the #availablevelocity.

Taking this approach will ensure your IT, Build and Release and/or Technical Operations teams are able to respond in a timely way, but are not purely interrupt-driven. They are also able to move key initiatives forward in a planned and predictable way.

Terms ‘NMI’ and ‘Slacklog’ are trademarks of Shirley A. Foster


Succeeding with Agile Development – Part VI

In this final of this six-part blog series, Shirley provides practical advice on adopting Agile Development methodologies.

scalingAgile Development is a software development methodology based on incremental, iterative development, where requirements and implementations evolve through collaborative, self-organizing, cross-functional teams. Agile Development promotes product delivery as a time-boxed series of functional and potentially shippable product increments that enables teams to rapidly respond and adapt to change.

Part VI – Scaling


Velocity is the sum of the story points a team completes in a single Sprint. Velocity for any given team will depend on the team composition and the calibration used for relative sizing.

Measuring and understanding a team’s velocity is important for several reasons:

  • During sprint planning, use the team’s velocity to corroborate the cumulative story points being committed. Note: Target velocity for any given sprint should be adjusted to reflect  time that delivery team members will be absent (e.g. vacation, holidays).
  • As the team grows and/or invests in knowledge transfer, you can measure and take advantage of changes in velocity.

Story Size and Sprint Commitments 

Once you are measuring velocity, a good rule of thumb for successful sprint planning is not to commit to any story greater than 50% of the total velocity of the team (i.e. if the average velocity of the team is 75, no individual story should be 40 points or greater.) Taking on larger stories will impose unnecessary risk on the team’s ability to complete the sprint, since they could ‘blow up’ with unexpected additional work and extend beyond the boundary of the sprint. If your backlog contains stories that are  larger than the acceptable size for your sprint velocity, engage the Product Owner and Delivery team. Either break the story down into smaller stories with greater sizing accuracy, or refine the sizing with additional clarification from the Product Owner or by technical discovery (see ‘NMI Stories’ in an upcoming post on  ‘Extending the Agile process’)


While measuring your team’s velocity using the original story sizes is a major step forward, you can increase the quality of your velocity measurement – and improve the team’s ability to  size – by resizing stories when they are complete. Resizing should occur as close to the completion of a story (definition of done) as possible. Since resizing requires the involvement of everyone who worked on the story (developers, QA, etc.) I recommend resizing any stories that are newly  ‘done’ immediately after the daily standup. This is best done at the scrum board where the story team can see all the resulting tasks and should take no more than 1-2 minutes. Ask the story team to collectively review how the story implementation had proceeded and listen for things like “the API turned out to be harder to stabilize than I’d expected. I had to build a lot more sample code” or “the implementation was so much easier than I expected (developer). It was much more challenging to test (QA)”. Essentially, let the team reflect on their implementation experience then pose the question. Do you still feel this is an 8? (that is, less than 1/2 the size of this 20?) Let the team come up with their resized number and write it on the story card next to the initial sizing.

Having this data will not only allow you to better measure the team’s velocity from sprint to sprint, but will also provide you a way to gauge how teams are sizing overall (are some teams more effective/accurate at than others?) and work more closely with the teams who need additional help.

Burn Down Charts

Burn down charts are used to track the progress or health of a sprint. Days in the sprint are plotted on the X-axis and total points are plotted on the Y axis.

Screenshot 2014-03-06 14.23.52

Assuming a sprint or iteration progresses exactly to plan, the burndown would occur in a linear fashion, ending in 0 on the last day of the sprint.

Rarely do actuals track to the ideal scenario and, instead, points are added to the sprint, work takes longer than estimated, etc. While some Agile teams map their progress across all stories daily – by estimating how many points they have completed for open stories – I prefer to only burn down time when a story is complete – to avoid the risk of teams overstating the work completed. Plot your burndown using a single line for the ideal burndown and (most likely) a staggered line showing #points remaining that changes as stories are burned down completely.

If your burndown chart shows the team getting well ahead of ‘ideal’; burning down story points faster than planned,  consider adding stories to the sprint. More likely, if your team is running behind the ‘ideal’  (on an overall basis – not any specific story) use this as an early indicator that you may have to drop or reduce scope of incomplete stories in order to deliver a potentially shippable (coherent) product increment.

Don’t be surprised if your ‘actual’ burndown goes up, not down from day to day. This could be a case of a story being resized (more accurately) after sprint planning, or more likely that work has been added to the sprint. When this happens, it’s another indicator of risk. Use the data to help you and the team decide if they are on a successful trajectory. Renegotiate commitments, trade for smaller stories with the Product Owner or trim individual story scope to bring the team back on track.

Like many things in Agile, burndown charts are not an absolute indicator of success or failure. It’s your job to manage the overall delivery risk and the burndown chart is a useful tool in that effort. Burndown charts showing ever increasing points remaining, or a lack of any stories fully burned down is a key indicator that something is off-track. Get together with your team and review what’s contributing to the dynamics you are observing (it could be that 3 stories are about to be completed), if the risk is real and what can be done to steer the team to success.


I’m sure as you’ve read these posts that it’s become clear that I believe that Engineering leadership has two responsibilities:

  • Risk management/mitigation
  • Team success and recognition

Agile provides the framework and the insights necessary to identify issues early and to work with the team to achieve a successful outcome. Agile empowers the team, through relative sizing, self-organizing, sprint commitments and collaboration to control their destiny in complex and rapidly changing situations. And, it provides the ability to demonstrate progress to the rest of the organization and establish a regular delivery cadence.

Together, Agile provides the tools and techniques necessary for even the most complex software project to be managed successfully. Successful teams and also highly motivated teams. Celebrate every little win as you adopt Agile – from a task being moved across the board, to a story being completed, to the Sprint Review demo. Take the time to reflect on how rapidly the product is evolving and how far you’ve come as a team (remind them how things used to feel when deliverables were late, QA was overwhelmed, etc.) and recognize everyone’s efforts.

Lastly, enjoy seeing your Agile proteges evolve. Before you know it, they will be guiding others in how to adopt the agile principles and it will be hard to remember life before Agile.