I recently had a conversation with a friend about Scrum and he mentioned a challenge his team was experiencing. They were a fairly young team with less than a year under their belts, but I have frequently encountered the same challenge and wanted to share my thoughts. Essentially, the problem involved an observed discrepancy between the team’s velocity and capacity. To define these terms, velocity is a relative measurement of the amount of work done in a sprint while capacity is the total number of hours a team has to do the work. The discrepancy they observed was that they were only able to allocate about 30% of their capacity before hitting a ceiling with their velocity. In other words, the average amount of work they were able to complete in a sprint, when estimated in hours, was only about 30% of their time. To make matters worse, several members of the team, including the scrum master (who was also a developer), felt that the solution was to allocate more hours and just work harder. The scrum master even went so far as to unilaterally reduce estimates in order to bring in more work during a sprint. Apparently, the team’s management supported this approach and my friend (who constantly works overtime anyway) was very discouraged about the direction things were going with Scrum and I certainly cannot blame him.

In my own experience, excess capacity is not really a problem. It simply means that the team is not able to accurately estimate the hours required to complete a work item. This is usually caused by poor estimation skills (which can only be improved with practice) or the mid-sprint discovery of unplanned work. Teams that are transitioning to Scrum almost always estimate poorly and teams that work with legacy code are often unable to predict the actual work required for any given work item. In situations like these, the best approach, in my opinion, is to establish an average velocity as quickly as possible and use it as a measure of how much work to attempt in a given sprint. Once a team is able to consistently complete their commitments, they can focus on improving their velocity. The metric of capacity is not needed for any of this and to be honest I don’t find it useful for anything.

Unfortunately for my friend, the real challenge his team is facing is organisational. Their scrum master is functioning as a local authority and is empowered to do so by the existing management structure. This is a hierarchical and authoritative mentality common in traditional development organizations and it will not go away willingly. It can take many forms. Sometimes the product owner is the local authority, sometimes it is a senior engineer or even an external manager. Sometimes there are competing authorities. Regardless of the form, the result is a top-down hierarchical arrangement where decisions are made outside of the team and are passed down from authority to authority and the team is expected to comply. On a mature agile team, this organizational structure is inverted. The team is empowered by the management structure to make their own decisions and determine their own direction. The team works together without positions of authority and makes decisions based on consensus.

The entire agile movement is largely premised on the idea that this organisational arrangement is more effective at producing value than the traditional top-down arrangement. Unfortunately, it is rare to see a management structure willing give up top-down control once they have become accustomed to it. Instead, it is common to see situations similar to my friend’s above where an organization will superficially embrace the motions of an agile methodology but not actually embrace the underlying values. For organizations that are not already well established in agile, it is very difficult change. In order for a true transformation to occur, management must genuinely support and value the principles of agile and that requires intelligence, courage and vision–qualities that are rare to find in combination anywhere.

Below is a synopsis of the conversation referenced above.

[Here is the quote I promised to send you regarding your question on working overtime.]

“The Agile mindset views recourse to overtime, other than on an exceptional basis, as detrimental to productivity rather than enhancing it. Overtime tends to mask schedule, management or quality deficiencies; the Agile approach favors exposing these deficiencies as early as possible and remedying their underlying causes, rather than merely treating the symptoms.”

Sustainable Pace

Thank you. I will use this as ammo in our next retrospective when I bring up overtime hours.

Good luck!

Ultimately it doesn’t really matter if the numbers for your estimates are big or small. Their only purpose is to provide a loose projection of when future work might be complete.

So for example let’s say you have a backlog of 100 work items that you want to complete in a Scrum project. The first step would be to look at each item and assign a numeric value to the amount of work involved. It’s just a rough guess, but you want to be as honest as possible. You may have a team full of speedsters that believe they can complete anything in under 2 hours or you may have a timid team of novices that estimate high. It doesn’t really matter. Let me explain…

Imagine there are two teams—team rabbit and team turtle. Both teams are asked to complete the same 100 work items using Scrum. Team rabbit is full of speedsters and they estimate each work item on the low side. When they add up all of their estimates they get the number 100. This estimate may be 100 hours, or 100 points or 100 unicorns…it doesn’t really matter. Only the number itself matters. Let’s use “points” as the unit just for the sake of argument.

At the end of sprint 1 team rabbit has completed 15 points worth of work items. In sprint 2 they complete 23 points and in sprint 3 they complete 17 points. Team rabbit’s average “velocity” is 20 points per sprint ((15 + 23 + 17) / 3 = 60 / 3 = 20) and management can reasonably conclude that they will complete all the work in approximately 5 sprints. If another 100 points of work are added to the project, management can plan for that work to take another 5 sprints to complete.

Team turtle on the other hand is not confident. They estimate each of the 100 work items and come up with a total of 1000 points. At the end of sprint 1 they complete 100 points, then 200 in sprint 2 and finally 150 in sprint 3. Their average velocity is 150 points and management can conclude that it will take them between 6-7 sprints to complete the work. Note that even though their average “velocity” is higher, their actual speed is lower. Average velocity is just a relative number for measuring a teams speed. You can’t really compare the numeric velocity of two teams because each team estimates in a unique way.

So as you can see, the actual estimate number doesn’t really matter. It’s only value is to determine a team’s average velocity which is in turn used to forecast when future work *might* be complete. I’ve found forecasting to be reasonably accurate out to about 3 months assuming the team’s average velocity is stable and they estimate in a consistent way. In my opinion, forecasting isn’t very useful.

Average velocity on the other hand *is* useful. Once you establish an average velocity, you have a pretty good idea of how much work your team can complete in a given sprint. If you commit to more work than what you’ve been able to complete (on average) in the past, you are probably not going to meet your commitment without working overtime and/or taking shortcuts and in the long run you’ll do more harm than good that way.

The best way to use estimation is to be as honest and consistent as possible in order to establish an average velocity. Then once that is established, use your retrospective meetings to come up with ideas to increase average velocity by eliminating bottlenecks and inefficiencies. Fudging the numbers in order to force the team to commit to more work in a given sprint will probably not change the actual amount of work that is completed, but it will certainly lower the quality of work, create resentment within the team, and even further reduce your ability to forecast accurately.

One last thing…sometimes a team will break sprint backlog items into tasks. For example, you might have a backlog item to add a new “report” feature. When you decided that you will work this backlog item in the next sprint, it is common to create tasks for all of the individual steps involved in completing the backlog item. You might have the following tasks.

  1. Create data structures to store the report data
  2. Create the report layout
  3. Create the report export process
  4. Write unit tests
  5. Deploy to QA environment
  6. Perform unit tests

Some teams will estimate these individual tasks in hours and use this value to determine how much work to commit to in a given sprint. For example, if your sprint lasts 2 weeks and you have 4 developers, then you have a capacity of 2*40*4 = 320 hours to spend on tasks in a given sprint. In my experience, this is a very poor way to estimate how much work you should commit to in a sprint. Average velocity is much more realistic. In the 3 years that I did Scrum, we always estimated our tasks in hours but used average velocity to gauge the amount of work to commit to. Most of the time the estimated task hours was 20-40% of our capacity and I worked overtime almost constantly.

Overall, I think agile is mostly about getting into an iterative mindset and working as a team. Check out the video linked below. It’s the best explanation of Scrum that I’ve seen. It might be worth to share it with your team.

The one true variable in our sprint is the average velocity. For the last 7 months, our average velocity is 280, but our hours worked always seems like we only worked two-thirds of a month. So, management keeps saying we need to increase our velocity since our hours worked shows we have extra time. I don’t know if I can get the under-estimators to be realistic since they get praise from management for estimating low numbers.

Agile is a great way for people to get promoted, because they can under-estimate, let other people do the work, and then get praise from management because they push for fewer hours.

It really takes a good team to make Agile work.

Agreed, difficult team members can really make life unpleasant. One thing that I did to increase the number of task hours was add tasks for every little thing…testing, deployment, planning, refactoring. I took a few days and wrote down everything I did to get an idea of what I was spending time on and then I started including it in my task estimates, even if it was just 15 minutes. I found that I was underestimating the time it took to test and debug code. For example, if something took 4 hours to code, I found that took about 4 hours of testing to get all the edge cases worked out, but I was only estimate 30 minutes.

Despite all that, we rarely committed to more than 50% of our team capacity. Fortunately, no one cared. If I ever work on a Scrum team again, I plan to suggest that we not even bother estimating task hours. It’s a waste of time.

Also, I should mention that capacity is typically calculated at 6 hours per team member per day (not 8 hours like I implied last email). So if your sprint is 1 week and you have 10 team member, the capacity is 5 days * 6 hours * 10 team members = 300 hours. When committing to work, most recommend that you not go above 75% of your capacity. But like I said, I think this is a waste of time.

It’s all just a bunch of nonsense anyway. The real trick is having fun despite all the difficult people. 🙂