The purpose of this chapter is to review the history, basics, philosophy, benefits, and a few interesting tidbits about Scrum. A well-balanced, effective ScrumMaster understands both the principles and the practices of Scrum. I have discovered over the years that people often realize the true nature of the ScrumMaster; unfortunately, the role is consistently misperceived and downplayed as a mere "iteration manager" or "Agile project manager". Along with your reading of the referenced source materials, this chapter will provide you with the foundational information necessary to navigate the murky waters of organizational change—the real task of the ScrumMaster.
In order to stay in business, the typical for-profit company must meet its consumer demand with an adequate supply of valuable services or products. The primary role of the ScrumMaster is to bring visibility to the development team's true capacity, so that the organization may balance that against its consumer demands. Unlike a manufacturing line in which the management measures units delivered per day, Agile software development is measured in terms of customer happiness based on the features or feature set delivered by the team. In this day of viral YouTube videos, tweets, and other such media, companies must stay focused on delivering today's most important demands quickly, while maintaining a flexible response to rapidly changing competitive and consumer landscapes.
A few decades ago, the waterfall response was, in fact, adequate because new markets weren't being created every day, the Internet did not exist as we know it today, and it took much longer for people to spread their messages and ideas around the world. The viral Gangnam Style or recent Harlem Shake are testaments to just how quickly words (and funky dances) can travel. There is simply no time to play around these days.
I've attempted to illustrate the opposing forces of supply and demand in the figure above. The business continuously demands new features and products from the technical team, whether it's the CEO with a new mandate or a product manager in need of a cool new feature. A professional ScrumMaster can help the business see when the balance is tipped too far in one direction; and most importantly how to best use its capacity to respond to demand with speed and quality. There is no perfect long-term solution; rather the ScrumMaster must help the organization find balance as the landscape on both sides constantly changes.
In the past two decades, companies have increasingly relied on Agile methods to keep up with growing demand and changing markets; today, about half of all companies that use Agile use Scrum. Over the years, we have observed Scrum's adoption at companies new to these ideas, as well as renewed/continued interest even in experienced Scrum/Agile organizations. We have also noticed a trend that companies have adopted subsequent processes—Extreme Programming, Lean, Kanban, and others—usually after initiating Scrum first.
Jeff Sutherland first used Scrum at Easel Corporation in 1993, and subsequently used it at companies such as VMARK, Individual, and IDX Systems throughout the 90s. Ken Schwaber, who worked with Jeff at Individual, 'formalized' Scrum at the OOPSLA Conference in 1995. At the turn of the millennium, Jeff famously applied Scrum at Patient Keeper and Ken helped scale Scrum at Primavera Systems, the latter whose case study was made popular by an online whitepaper and several anonymous mentions in Ken's second book, Agile Project Management with Scrum.
However, these early applications in the mid to late 90s weren't the first rumblings of Scrum. In 1986, Harvard Business Review published an article by Hirotaka Takeuchi and Ikujiro Nonaka entitled The New Product Development Game, in which the authors wrote that development organizations must extend their focus beyond scope, time, and cost to find ways to increase speed and flexibility of product delivery in order to win in the new competitive landscape. Instead of the relay race, "…a holistic or 'rugby' approach – where a team tries to go the distance as a unit, passing the ball back and forth – may better serve today's competitive requirements." This article was the first mention of Scrum as a new paradigm for product development—a thought framework for quick, flexible, and competitive product development. It's important for ScrumMasters to remember that Scrum practices—a set of work steps, outputs, and artifacts—are nothing without the underlying mind-set and concepts toward product development that Takeuchi and Nonaka set out to describe: built-in instability, self-organizing project teams, overlapping development phases, multi-learning, subtle control, and transfer of learning.
The concepts behind Scrum go even further back in time. In the 1950s, a management consultant by the name of W. Edwards Deming created the Plan-Do-Check-Act (PDCA) cycle as a framework for continuous improvement. PDCA, also known as the Deming or Shewhart cycle, had an early influence on Toyota's lean approach to manufacturing. These ideas map one-to-one to that of a Scrum's sprint, and even to a sprint's daily scrum, as indicated in the following figure and later in the book, but Deming didn't know he was doing Scrum. Or more appropriately, perhaps, is that today's Scrum teams don't readily realize that they're applying the Deming cycle!
Scrum's foundation is even older than Deming. Go back 1,000 years to when Alhazen, a Muslim scientist and mathematician, ran experiments with reflection, refraction, lenses, and mirrors, thus deriving some of the principles of optics (captured in his book, The Book of Optics). Alhazen is considered by some to be the father of the scientific method (http://www.wikipedia.org/wiki/Scientific_method), a process through which a scientist creates a hypothesis or asks a question, runs an experiment, discovers results or gains knowledge, analyzes the findings, and possibly modifies the hypothesis (or decides to run a subsequent experiment). This empirical, or evidence-based, process is one in which a person will use knowledge gained from one experiment to draw conclusions or influence the next experiment, like the process by which Alhazen discovered the basics of human eyesight.
Finally, if we go a bit further back in history, you could imagine that even a primitive tribe's hunting session looked a lot like a Scrum sprint: everyone gathered together, grunted about where to stalk the prey, gathered up their hunting tools, killed dinner, and then talked about the hunting experience, which improved the plan for next time. We see evidence of these primitive retrospectives in cave drawings around the world. Scrum a most natural way of working that relies on the collaboration and close work of people to achieve an outcome. It's been this way for at least 11,000 years. We'll explore in later chapters why, even though it reflects a natural approach toward work, Scrum is so challenging.
There are three categories of concepts that provide the foundation for Scrum. The first set of ideas resides in management approach for complex and chaotic situations, the second is a set of core values, and the third is a deep-seated focus on delivering product in a lean fashion. You should gain a thorough understanding of these concepts, not for passing a certification exam, but for explaining to others in the organization why and how Scrum is different than waterfall.
Complex adaptive systems are made up of varied interdependent elements whose relationships change as a result of experience. Complex systems adapt; as one variable emerges, others are affected. A simple example from biology is the Staph bacteria; in the 1940s, more than 99 percent of Staph bacteria were sensitive to penicillin. Today, Staph is resistant to penicillin; due to the introduction and exposure to the antibiotic, the bacteria evolved to survive.
In our world, requirements, technology, and people are intertwined; a change in one means a change in the other. When I was a kid, I loved to go to the movie theater to, and sometimes I would go just to play video games at the adjoining arcade (Ms. Pac-Man, Frogger, Pole Position, and Asteroids). My family couldn't afford a home system such as an Atari (and, having played at the arcade, I much preferred the arcade to an Atari home system anyway). Well, all that changed when Nintendo launched its Nintendo Entertainment System (NES). The kids next door got the system as a Christmas gift that year and next thing you know, we were over at their house many times per week after school playing Super Mario Brothers. Then along came the handhelds, Super NES, PlayStation, Sega to name a few, and now mobile phones and interactive play are all the rage. The advances in 16-bit and 32-bit consoles initially improved the user's experience at home with better graphics, competitive pressure resulted in more innovative home products from which a user could choose, and new technologies have allowed gaming companies to extend offerings into new arenas and find new markets. The technology, user needs and requirements, and developers in the gaming industry are all intertwined. A change in one necessitates a change in the others. Little did I realize, at that time, Mario and Luigi's importance to technology—much greater than just saving the princess!
Here's the deal: today's users change their minds about needs and desires minute to minute, developers continuously release new technologies and better versions of the old, and stakeholders and customers are never shy about saying how they feel about existing features. This emergence, that is, the evolution of new knowledge, needs, and desires about requirements and technologies during the course of a project, contributes to one heck of a complex situation. Complex projects cannot be adequately managed in a traditional approach, where all tasks are defined up front and then delegated to workers. In situations where the exact answer is not known, and must be discovered, an empirical process or evidence-based process is the best approach to use. (Take a look at the article A Leader's Framework for Decision Making, by David J. Snowden and Mary E. Boone, at Harvard Business Review). But it's more than just applying the right tool or technique to the matter; complex systems involve complex human interactions. While a complex project requires a process that allows for emergence, it also requires a leadership style that facilitates humans getting along together.
The following figure is the famous Stacey Matrix, adopted from Dr. Ralph Stacey, Professor of Management at the Business School of the University of Hertfordshire, who has spent many years exploring how the complexity sciences can be leveraged to understand organizations. You may remember hearing about his work during your ScrumMaster training. The Stacey Matrix looks at certainty and agreement, which correlate to technology and requirements, respectively, in our world. The farther from certainty and agreement a situation is, the more complex or even chaotic the situation; on the other hand, a simple situation is one with agreement and knowledge. Think about an easy project from your career. The requirements for the new feature were very clear and straightforward, and you wrote all the existing code a few weeks ago. It was easy for you to provide a time estimate since you figured that surprises weren't likely. This is a simple situation, rare in the software world. A complex scenario, on the other hand, is one in which you may not have been very familiar with the code, the customer often changed her mind, and the vendor team never met its deadlines. In this case, it was impossible for you to estimate beyond the tip of your nose as surprises (mostly bad) were guaranteed.
You'll notice that I overlaid the Scrum roles and control points onto the Stacey Matrix. ScrumMasters know that they do not control complex projects by acting as task masters and telling everyone what to do every day; rather, control is acquired by allowing a team to self manage through a series of sprints (or time-boxes) and by requiring the product owner to force rank the product backlog. Each role in Scrum has a specific and important set of responsibilities, each steeped in the idea of controlling chaos (in fact, Ken Schwaber's original Scrum website is called www.controlchaos.com). Many people ask me if there is a role called project manager in Scrum and the answer is a resounding no! The reason is that by giving control to only one person over what gets done, how it gets done, and why it gets done results in chaos and other bad effects. Three main responsibilities divided among three main roles creates checks and balances in our complex project systems; these three main Scrum roles, coupled with a simple process framework, control chaos. Each sprint, by focusing on a narrow selection of product backlog, an empowered team devises the appropriate technical solution bit by bit, thus bringing the project closer to the simple range sprint after sprint. If these controls are not protected, it is all too easy for the scale to unfavorably tip. Scrum is a very simple concept and extremely powerful once implemented (and yet rather difficult to do). You, my dear ScrumMaster, must keep that sprint to the simpler side of things by thwarting interruptions and helping the product owner keep the product backlog in order.
Even though Dr. Stacey's matrix and Snowden and Boone's article provide us with nice frameworks of thought to choose the right process for the situation, please do not oversimplify this information! Dr. Stacey didn't publish the matrix beyond the second edition of his book, and he's now up to six editions! What happened? Dr. Stacey stopped publishing the matrix because people oversimplified it, thinking that one can just choose a tool or technique depending on the classification of their project. Dr. Stacey warns us that it's not that simple, that, "what happens is the result of the interplay between the intentions and strategies of all involved and no one can control this interplay, hence the fundamental uncertainty and unpredictability of human life." This is another way of saying that Scrum doesn't fix the problems, people do; Scrum only exposes where the challenges lie. People must solve the problems exposed by Scrum and should craft different solutions for different scenarios. This book attempts to explore how the ScrumMaster may facilitate some of those challenges both inside and outside of the team.
Knowledge is the outcome of experience. In a traditional approach to managing projects, all the decisions are made at the beginning of the project. This has always been interesting to me because a person has the most knowledge about a project at the end of the project! Scrum puts urgency on teams so that they deliver value quickly; in doing so, knowledge about requirements and technology increases rapidly and team dynamics begin to emerge and stabilize. The sooner a team gets started, the better. When a team has knowledge its members can make better, well-informed decisions. Scrum teams delay decisions around lower-value requirements for a later point in time, as requirements in their fast-paced technology worlds are bound to change anyway.
There are four legs supporting the Scrum barstool that make it unique and effective in complex situations: prioritized product backlog, dedicated cross-functional team, time-boxes, and inspection and adaptation. These four entities form the necessary boundaries that allow us to call Scrum an empirical process. When a barstool loses a leg, it is imbalanced (ever try sitting on one?); even if one leg is slightly shorter than the others, the barstool is wobbly and annoying. The same idea applies with the Scrum concepts: when the team become lax on one or more boundaries, the project gets loosey-goosey; the experiment gets shaky as it loses its control points. For example, if for some reason the ScrumMaster allows interruptions during a sprint, the team runs the risk of not finishing anything by the sprint's end. Everyone must stop and start again—we call this chaos. Likewise, if the product owner does not rank features in the product backlog, and no thought is given to priority, the team may or may not deliver the right features to customers. A simple violation of these guiding principles can cause major downstream effects. If a team adheres to these four principles of Scrum, it will likely attain a higher degree of stability and knowledge.
Ken Schwaber and Jeff Sutherland designed these four "legs", or principles, of Scrum to enable empirical process control for software development projects (although the techniques are useful in just about any type of project). Scrum is an evidence-based way of progressing through a project or initiative. Without designated points in time around which people can provide feedback on features, like in waterfall projects, the team, management, and customers really have no idea what is being developed. Try to recount a status meeting for a traditionally managed project. What was that meeting like for you? I can say that, for me, it was often daunting. I would present a status report with a bunch of red, green, and yellow dots on it, some numbers and percentages of 'phase complete', and sometimes a list of risks. I can honestly say that as a traditional project manager, if I didn't truly know the status, I'd just put a yellow dot on the dashboard, reckoning I'd figure it out later. Or, even worse, I'd stay at work very late for a few nights ahead of the status meeting attempting to make the Gantt chart look something like reality. Scrum, with its emphasis on potentially shippable product increments each and every sprint leaves nowhere to hide. Sprint reviews replace status meetings; and the status is real—that is, the team shows real features that work! That is exciting and tangible! It's evidence, and we can work with evidence! Instead of a bunch of guesswork about what we think is the status of the project, in Scrum we can actually see, touch, and feel the status of the project by attending a sprint Review and interacting with the software or system as it exists at that point in time.
I recently watched a television program that featured George Cleverley and Company, a famous London shoemaker. Founded in 1958, Cleverley makes some of the most expensive handmade shoes in the world, shodding some of the world's most discriminating tootsies. In order to make such a beautiful product of unprecedented quality, the shoemakers must work very closely with the customer to understand their desires and requirements. After crafting precise shoe-lasts (forms around which the shoe is built) made from several measurements of a customer's foot (bunions and corns included), the shoemaker must then work with the customer's choices regarding various leathers, soles, dyes, laces, thistles, welts, and stitching in order to make the shoe that the wearer has envisioned and will love. It takes months for a customer's shoes to be finished, and at least three thousand U.S. dollars, but for 58 years customers have gladly paid and anxiously waited for the call that their shoes are ready. George Cleverley and Company involves its customers in the design and creation; there's simply no other way to build the perfect shoe. In reading about the craftsmanship, attention to detail, and customer satisfaction—the hallmarks of the Cleverley brand—I realized that the process in which Cleverley shoes are built is very Scrum-like: an iterative creation process where the customer is an essential part of the desired outcome. Even though the product backlog in this process is relatively fixed—that is, the order of the steps in shoemaking is fixed, the customer must be involved every step of the way.
In addition to the basics in complex adaptive systems theory, Scrum also has at its center five core values: commitment, focus, openness, respect, and courage. (Take a look at Agile Software Development with Scrum, by Ken Schwaber, Prentice Hall.) We will explore these in more detail in Chapter 8, Everyday Leadership for the ScrumMaster.
Teams commit to their goals for the sprint, product owners commit to ordering the product backlog, and ScrumMasters commit to removing obstacles along the way in order to steady the flow of product development. The Scrum team should do whatever is necessary in order to meet their goals, and it's important that they are empowered to do so.
Additionally, for a team to be able to complete its work, its members must be allowed to focus. The ScrumMaster does not allow changes in the sprint's commitment so that the team may keep its focus. When a team gives its full attention to the problem, its work is much more productive, predictable, and fulfilling.
As Scrum uses empirical process control to make progress through a project, it is essential that the results and experience of an experiment (that is, a sprint) are visible. For example, the sprint review provides visibility into how the product's features and capabilities have emerged, and the sprint retrospective engages the team members to discuss their personal and team successes and challenges experienced during the sprint. Once visibility exists, inspection and adaptation can occur. Again, evidence (information) is the heart of an empirical process; thus, openness is one of the five core values.
Respect is something that all humans should have for each other, but unfortunately it does not always exist. In order to be its best, a team's members need to respect for each other and, the knowledge that each brings to the table, experiences, working styles, and personalities, just to name a few. Respect doesn't come for free; it is earned. Scrum team members should be dedicated, cross-functional, empowered, and self-organizing—any team that is not provided this environment or structure will have a tough time achieving mutual trust and respect. We will discuss in subsequent chapters how a ScrumMaster can model (and thus create) trust and respect within the team.
It takes buckets of courage for a ScrumMaster to apply Scrum the way it was intended. Even though Scrum is very simple in design, people end up over-engineering its intended plainness. I often see management at various companies bring in Scrum as a way to do more with less (not always the case!), and they don't at the very least realize or acknowledge that the use of Scrum puts pressure on organizations to change. One of the primary responsibilities of the ScrumMaster is to help the organization identify its weaknesses so that it may improve. This takes courage. Sometimes, a team has to push back on the product owner when asked to take on too much during a sprint. It takes courage to say no to that sort of pressure. A product owner must have courage when communicating with other stakeholders about the reality of a project. These are just a few examples that require courage in order to not break the boundaries of empirical process or introduce more waste into the product or process.
An interesting tidbit: the Scrum core values sound a lot like the U.S. Army's seven values of loyalty, duty, respect, selfless service, honor, integrity, and personal courage. We might not find that too surprising as the grandfathers of Scrum have military backgrounds: Jeff Sutherland was a Top Gun fighter pilot in the U.S. Air Force, and Ken Schwaber attended the U.S. Merchant Marine Academy.
Lean software development is the result of the translation of lean manufacturing into the software and systems development space. Whether it falls underneath the Agile umbrella is hotly debated in the community, yet it really doesn't matter when it's all said and done. This is because the seven principles of lean are inherently reflected in Agile methods, but perhaps in different ways and with different lexicons or naming conventions.
The first principle of Lean is to eliminate waste. From unclear requirements, unused documentation, hand-offs, wait time, and so forth, lean practitioners seek to reduce or eliminate altogether these wastes from the process. The Scrum framework echoes this lean sentiment by providing the retrospective so that a team may discover and fix anything that's not working well. You'll often hear teams discussing subjects such as wasteful documentation, overbearing and/or manual processes, too many defects, and other such issues in the retrospective.
Lean thinking also stresses that everyone on the team gets the chance to learn. Scrum mirrors this by requiring that every sprint ends with a potentially shippable product increment. This quality threshold requires that team members code, integrate, and perform tests, of various sorts in order to learn what is defective, while the product owner learns about the emerging product. Additionally, members of a team work very closely together, which means that they often learn a little about the others' domain.
In any project, people know the most about the project at the end of the project. Scrum teams would rather make well-informed decisions; therefore, they do not make decisions about every requirement up front. This is a manifestation of the fourth value of Lean: decide as late as possible.
Lean says deliver fast; in Scrum we deliver at most every 30 days, while many teams deliver even faster than that. Lean says empower the team, and so does Scrum. Lean says that integrity should be built into the system; Scrum answers this by requiring a team to define done with a customer. Finally, Lean guides us to see the whole—how the entire value stream, or chain of events leading to customer value, operates. Any bottlenecks should be removed immediately, and teams should be staffed so that they can complete finished product increments. Scrum reflects this by guiding us to create dedicated, cross-functional teams that conduct retrospectives. Such retrospectives help us unearth bottlenecks (or obstacles as called in Scrum) so that they can be eliminated.
This section serves as a very light introduction to the three roles in Scrum: Scrum team, Product owner, and ScrumMaster . While the book focuses predominantly on the ScrumMaster role, we will touch upon the other roles in more detail due to the natural interconnectedness of all three. As you probably already know, people get very anxious about roles and responsibilities; we'll go into more detail about Scrum (and non-Scrum) roles in Chapter 9, Shaping the Agile Organization.
The Scrum team includes the product owner, ScrumMaster, and the team members, whereas the Scrum delivery team is a subset made up of only the technical team members. The entire Scrum team huddles around a problem (a requirement from an ordered list known as the Product Backlog) and innovates solutions. Scrum teams should be five to nine team members, dedicated to the life of the project (and perhaps beyond), cross-functional, empowered, and self-organizing. Scrum teams plan, estimate, and commit to their work, rather than a manager performing these functions for them. The end goal of the team is to deliver a potentially shippable product increment that meets an agreed-upon Definition of Done each and every sprint. The George Cleverly shoe craftsmen would be considered Scrum delivery team members while the craftsmen, customer, and shoe design consultant together would comprise the overall Scrum team (it takes everyone's participation to make the right shoe).
The product owner is responsible for the product's success. In other words, while the team is responsible for delivering a quality solution, the product owner is responsible for knowing his market and user needs well enough to guide the team toward a marketable release sprint after sprint. There are different types of product owners in different types of projects, but regardless of the technical situation or the desired outcome, there should be one (and only one) product owner who makes final decisions about the direction of the product and the order in which features should be developed. The product backlog, or list of items to be completed by the Scrum team, is ordered by the product owner so as to reflect his most valuable requests or changes for the product in development. The product owner, since he is representing the "what" and "why" of the system, should be available to the team to have regular dialogs about the requirements in the product backlog; additionally, the product owner must make the product vision clear to everyone on the team and regularly maintain the product backlog in keeping with the product vision. The product owner always keeps the next set of product backlog items in a ready state so that the team always has work in the queue for the next sprint. The George Cleverley shoe customer is the 'purest' product owner a team can get—the actual person paying for the product!
The ScrumMaster safeguards the process. He/she understands the reasons behind and for an empirical process, and does his or her best to keep product development flowing as smoothly as possible. This "servant leader" (take a look at Servant Leadership, by Robert K. Greenleaf, Paulist PR) protects team members from interruptions in order to keep them focused on their sprint commitments, as well as helps the product owner get the product backlog in order if he or she does not understand how to do so. Additionally, ScrumMasters facilitate all Scrum meetings, ensuring that everyone on the team understands the goals and that they share a commitment together as a true team and not just as a collection of individuals. ScrumMasters remove obstacles that prevent a steady flow of high-value features; many times, the obstacles are organizational in nature.
Think of the ScrumMaster as the Switzerland of the Scrum process—remaining neutral, helping both business stakeholders and development, interacting at the right times to create the most important and most valuable features first.
Scrum is simple; it consists of six time boxes (one of which is optional), three roles, and three 'official' artifacts.
A sprint, the first of the six time boxes, is an iteration defined by a fixed start and end date; it is kicked off by sprint planning and concluded by the sprint review and retrospective. The team meets daily, in a daily scrum meeting, to make their work visible to each other and synchronize based on what they've learned.
During sprint planning, the second time-boxed event, the product owner and the team discuss the highest priority items in the product backlog and brainstorm a plan to implement those items. The set of chosen product backlog items and their subsequent tasks collectively is referred to as the team's sprint backlog.
The sprint planning meeting is time-boxed to eight hours for a 30-day sprint, reduced proportionally for shorter sprints (for a two-week sprint, for example, one may reduce that time to a maximum of four hours). The meeting is comprised of two parts. The first segment of the meeting is driven by the product owner who presents the most important product backlog items—along with any helpful drawings, models, and mockups, for example—and clarifies questions from the development team about what he/she wants and why he/she wants it. The second segment of the meeting is driven by the Scrum delivery team who work together to brainstorm approaches and eventually agree on a plan. It's at the start of this second segment that the sprint actually begins.
Of course, teams are always searching for ways to make planning faster and more efficient. You will likely see sprint planning take less time over a number of sprints for a team whose members have remained consistent.
The result of sprint planning is a sprint backlog that is comprised of selected product backlog items for the sprint, along with the correlating tasks identified by the team in the second segment of sprint planning. We'll discuss many details and ideas for effective sprint planning meetings in Chapter 3, Sprint Planning – Fine-tune the Sprint Commitment
In the daily scrum meeting, the third time box of Scrum, team members make their progress visible so that they can inspect and adapt toward meeting their goals, sort of like a mini-Deming (PDCA) cycle every day! The meeting is held at the same time and in the same place, decided upon by the team. Even though a team makes its best attempt at planning for a sprint, things can change in flight. Tom finishes a task late, Beth finishes early, Ashish is stuck. In this 15-minute meeting, team members discuss what they did since yesterday's meeting, what they plan to do by tomorrow's meeting, and to mention any obstacles that may be in their way. The role of the ScrumMaster in the daily scrum is to facilitate, keep the conversation from delving into problem solving, and record any obstacles that team members feel they cannot fix for themselves. The ScrumMaster will attempt to remove said obstacles after the meeting. The daily scrum meeting represents inspection and adaptation at a daily level. The Scrum delivery team members, product owner, and ScrumMaster are participants in the meeting. Anyone else is welcome to attend but only as observers.
The sprint review, the fourth time-boxed event of Scrum, provides the opportunity for stakeholders to give feedback about the emerging product in a collaborative setting. In this meeting, the team, product owner, ScrumMaster, and any interested stakeholders meet to review the features and talk about how the product is shaping up, which features may need to change, and perhaps discuss new ideas to add to the product backlog. It is common for a ScrumMaster to summarize the events of the sprint, any major obstacles that the team ran into, decisions that were made in-flight with the product owner's help, and so on, and of course the team should always demo what they've accomplished by the sprint's end. This meeting is time-boxed to four hours (for a 30-day sprint).
During the final sprint meeting— the sprint retrospective—team members discuss the events of the sprint, identify what worked well for them, what didn't work so well, and take on action items for any changes that they'd like to make for the next sprint. The ScrumMaster will take on any actions that the team does not feel it can handle; likely broader organizational issues. The ScrumMaster reports progress to the team regarding these obstacles in subsequent sprints. This meeting is time-boxed to three hours. We will dive into more details about sprint reviews and retrospectives in Chapter 5, The End? Improving Product and Process One Bite at a Time.
The final time box in Scrum is release planning, an optional event, in which Scrum teams plan for long-term initiatives comprised of multiple sprints' worth of work. The product owner and team discuss product backlog items, dependencies, risks, schedule, and capacity, among other topics, in this meeting to settle on a forecast for upcoming work. Since the product backlog can be infinite, release planning helps the team and product owner understand what may be possible given a release deadline. This subset of the product backlog is called the release backlog and is a valuable output of the release planning meeting. Release planning should be time-boxed but the time required depends upon the scale of the program, the number of teams, team distribution, and how well the product backlog is prepared. Release planning meetings are often held at the beginning of the project, and teams will meet with their product owners throughout the project to re-plan based on emergent knowledge. It is not unheard of, however, for teams to postpone planning a release until they have a few sprints of work under their belts, as the experience and knowledge they gain in a few sprints results in greater confidence in a longer-term release plan (more details in Chapter 2, Release Planning – Tuning Product Development). Either approach is acceptable and depends upon the needs of the organization.
Ken Schwaber once said to me, "I don't know why people make this such a long, drawn-out event. It isn't really that hard." And with that, we completed our release planning in 15 minutes.
Scrum only has a small set of artifacts: the product backlog, sprint backlog, and the product increment. We will briefly review these artifacts here and dive much deeper into them in subsequent chapters.
The product backlog is the product owner's 'wish list'. Anything and everything that they (and other stakeholders) think they might want in the product goes in this list. It could be infinite as there are always new ideas about how to extend a product's features. The product owner maintains the product backlog, although other stakeholders (including the team) should have visibility of and the ability to suggest new items for the list.
The product owner prioritizes the product backlog, listing the most important or most valuable items first. That is, there aren't 10 critical items at the top of the backlog with equal value; rather, there are 10 critical items that are ranked according to their priority or urgency, and they appear at the top of the product backlog, one after another. This is because items at the top are next in the queue to be implemented. Once a team selects items for a sprint (or iteration), those items and their priorities are locked; however, priorities and details for any not-started work may change at any time. Through this mechanism, teams are able to focus on this sprint's work while the product owner retains maximum flexibility in ordering the next sprint's work.
Product owners have many ways of evaluating and thus prioritizing their lists. They may also attribute product backlog items with additional information such as improves brand recognition, allows scaling, infrastructure, contracts greater than 10,000 dollars, and so forth. Attributes are unique to a particular company and a particular product and help the product owner to keep the list in the proper order.
Owned by the team, the sprint backlog reflects the product backlog items that the team committed to in sprint planning, as well as the subsequent tasks and reminders. Team members update it every day to reflect how many hours remain on his or her tasks; team members may also remove tasks, add tasks, or change tasks as the sprint is underway.
The product increment is a set of features, user stories, or other deliverables completed by the team in the sprint. The product increment should be potentially shippable—that is, of high enough quality to give to users. The product owner is responsible for accepting the product increment during each sprint, according to the agreed-upon Definition of Done and acceptance criteria for each sprint deliverable. Without a product increment, the product owner and other stakeholders have no way to inspect and adapt the product.
A team must keep its progress visible at all times. It will create many additional artifacts in order to ensure visibility. Some common visibility tools are the release and sprint burndown charts.
A subset of the product backlog that has been identified for a particular release is called the release backlog. Even though release backlog can be defined up front, the product owner may remove items, exchange items, or negotiate scope depth for some items as he/she considers scope, time, and cost throughout the duration of the project. Therefore, the release backlog should be updated throughout the project. Chapter 2, Release Planning – Tuning Product Development, provides more detailed information about product backlogs, release burndowns, and backlogs.
The release burndown chart displays how much work remains in the release backlog at the end of each sprint. This provides the product owner with important information so that he/she may make well-informed decisions about scope, cost, and time. In the following diagram, you can see that the amount of work remaining at the end of each sprint is more than the work planned:
During a particular sprint, if everyone on the team updates the sprint backlog with the number of remaining hours per task every day, then the team can see if they will be able to burn down the number of task hours by the end of the sprint. In the following figure, you can see that the team did not complete all the tasks it had identified in sprint planning; approximately 50 hours remain.
This burndown concept is very important because, once set, the sprint end date does not change. Coupled with a daily Scrum, the sprint backlog and burn down chart can help teams visualize when they might be getting off track and turn the conversation to focus on what to do about a given situation. The sprint burndown "burns down" hours over days of the sprint, while the release burndown looks at units of work (often referred to as points) for a release, or number of sprints. Chapter 3, Sprint Planning – Fine-Tune the Sprint Commitment and Chapter 5, The End? Improving Product and Process One Bite at a Time, provide more details about burndowns and backlogs.
Scrum is based on the lean concept of turning an idea into a feature as efficiently as possible. The Scrum framework is designed to surface obstacles that get in the way of delivering value. The game rules of Scrum are in place to protect the team from chaos so that they may finish their commitments for the customer by the end of a given sprint.
Because of the short cycle time and the relentless pursuit of identifying obstacles, there seems to be a never-ending list of challenges that the ScrumMaster needs to fix. The team surfaces—on a daily basis—any interruptions or impediments to their work. The product owner and other stakeholders inspect the product in the sprint review, which frequently leads to adaptations in product backlog and thus the evolution of the product. Finally, the retrospective provides time for the team to focus on process improvements so that the experience is smoother in the future. In conclusion, there are three built-in mechanisms in the Scrum framework for discovering obstacles. As they say in Texas, "If you go hunting for trouble, you'll sure find it." Scrum can feel very challenging because of what it brings to the surface.
So how do we handle these tough challenges as ScrumMasters? We have to ask ourselves: is this issue/challenge/hardship a constraint within our organization or is it a dysfunction? An example of a true constraint would be a document that must be written for U.S. Food and Drug Administration (FDA) compliance. The team mentioned it in retrospective because they think it's wasteful, but the product won't make it to market if it doesn't achieve FDA compliance. Therefore, it's a true constraint that must be worked around.
On the other hand, let's say that a team member mentions in the retrospective that he is being pulled away from the team to work on another project for a different manager. Is this a constraint? Maybe. But perhaps it's a dysfunction. Why? Well, Scrum says that team members are dedicated so that they can focus on and finish the functionality they've committed to implement. When a team member gets pulled onto another initiative, this makes for an unhappy developer who now must multitask and probably feels inadequate because the workload is too much to bear. It is likely that he or she won't finish either of the teams, commitments. Without finishing features, it's impossible to inspect and adapt, which breaks the benefit of empirical process control. This scenario is simply not acceptable; team members are not to be pulled from their teams. In this case, the ScrumMaster should alert the product owner that the developer's commitments are now in jeopardy. The situation may have to be escalated to managers to put a stop to this behavior. Eventually, team members must learn to say no, but that is not likely to happen in the beginning.
While you're probably not going to have a perfect team in a perfect environment as you start or continue your Scrum practices, I've provided a short checklist to help you identify the most important elements to help you as you begin:
Do you have a team whose members are dedicated to the project? Do the members represent a cross-section of skills and talents—everything necessary to build features for the customer?
Do you have a product owner? If not, can you find someone to play this role so that the team can get started working on the most important items?
Does the product owner have a product vision and a product backlog? (See Chapter 5, The End? Improving Product and Process One Bite at a Time, for more details)
Can you establish—at maximum—a 30-day sprint? Shorter if possible?
Can you get participation from business stakeholders in the sprint review? (Not a requirement, but sure to drive urgency and visibility to your team).
Do you feel courageous enough to communicate obstacles as they arise?
Can you help the team create and maintain the sprint backlog? (See Chapter 2, Release Planning – Tuning Product Development, for more details)
Can you commit to protecting the team from interruptions, no matter the interrupter?
The professional ScrumMaster must understand the principles and the practices of Scrum—the historical perspective, foundational concepts, and the mechanics of meetings, artifacts, and roles. This knowledge will help you as you face mismatches of expectations, beliefs, and knowledge about Scrum and its intent. Applying Scrum will expose innumerable challenges, unique to the organization and team in which it's been applied. Categorizing a challenge as a constraint or a dysfunction is at the heart of the ScrumMaster role. As you continue reading, you'll discover why this is so difficult and why a ScrumMaster needs courage.
Nonaka, I. and Takeuchi, H. (1986), The New Product Development Game, Harvard Business Review
Royce, Winston (1970), Managing the Development of Large Software Systems
Schwaber, Ken (1 February 2004), Agile Project Management with Scrum, Microsoft Press
Schwaber, Ken; Beedle, Mike (18 February 2002), Agile Software Development with Scrum, Prentice Hall
Schwaber, Ken (2007), The Enterprise and Scrum, Microsoft Press
Snowden, David and Boone, Mary (2007), A Leader's Framework for Decision Making, Harvard Business Review
Stacey, Ralph D (2012), The Tools and Techniques of Leadership and Management: Meeting the Challenge of Complexity, Routledge
Stacey, Ralph D (1996), Strategic Management and Organizational Dynamics, Second Edition, Routledge
Jeff Sutherland's Scrum Handbook at http://jeffsutherland.com/scrumhandbook.pdf
Stacia Viscardi's Scrum website at www.helloscrum.com
