Business Case of Agility

Success stories and surveys support agile software development. In addition to anecdotal evidence there are little hard figures to guide our decision making. Here cumulative business value charts are used to describe the impact of agile and traditional choices to the bottom line. Focus is on visualizing the economic impact of individual agile practices and assumptions of their costs and profits.

A chart describing the cumulative earnings as a function of time is a basic tool for optimizing the return of investment, ROI. The line is higher all the time if we have multiple deliveries. It is still better if the more profitable increments are deployed the first. This advantage in time-to-market is clear also in the surveys. Double the results in half of the time assumes that earlier deliveries have business value and that the additional cost of more deliveries is small.

  • It is not trivial to find minimum viable products that create value especially when organizations do not improve their businesses continuously but use large projects with non-negotiable late deadlines. Waste of the inventory of the partially done software is not as visible as the waste of tangible inventories.
  • The cost of a delivery has been a show-stopper of agility. Business models were based on sales of new versions of software that were installed manually to each client computer. Each deployment package had to be tested and integrated manually. Without automation the costs can overcome the benefits of early deliveries.

The cost of change must be small if we proceed empirically and feedback directs our product and development process. Customers pay these changes when they create more value than the costs of delaying and implementing the the changes.

  • Extreme programming proposes a set of practices like test driven development, re-factoring and pair programming that change the cost curve, cost of change, such a way.
  • Waste of unnecessary features can be avoided by prioritizing product backlog items based on their business value and changing the order when appropriate.

Risks are an integral part of any design. In software development we do not commonly know what the users actually need and how we use  new technologies that create the results. The cone of uncertainty is high. Reports of more than hundred-fold effort differences are common. Sales, savings and profit estimates are so inaccurate that it is quite common to ignore them altogether.

  •  Whole investment is lost if  we do a wrong thing or fail in the implementation
  • Customers try to protect themselves by fixed price projects. Costs increase because vendors have to buffer their offers especially if penalties are used. Competitive bidding weight visible price or over quality and total cost of ownership. The bidding game sets customers and vendors against each other.
  • Incremental deliveries make risks visible and adaptation is easier.
  • Agile approach is a natural risk management system, but it often fails when appropriate action is not taken. Organisation cultures must allow cancelling and redirecting projects.
  • Queuing theory and theories of variation can lead to improved practices in handling unpredictable situations.

Traditional mass production uses specialized lowest costs work force, rigid processes and invests in tools. It locally optimizes the cost of the tasks that workers do and is often blind about the required administrative burden.

  • Cost of learning can be significant. It unavoidable when we do something new. Open workspaces, pair programming and shared responsibilities are agile practices that accelerate learning. It would be good if that could break Brook’s law.
  • Cost of teamwork is minimized in a cross-functional team in single location. Wastes of relearning, transfer of work, task switching and delays are minimized. Software development has dis-economics of scale because a lot of communication and coordination work is required.
  • Utilization rate of the specialist become easily low. To avoid that developers take part of multiple simultaneous projects which often leads to coordination chaos.
  • Cost of motivation is difficult to estimate but essential part of working life. Leadership style has an impact on the engagement of team members which may result in more value for the business.
  • Traditional big front end design tries to minimize the cost of rework but real life shows that a large amount of work is needed to correct the bugs, integrate the components and deliver what users actually need.

Traditional and agile approaches have different assumptions of the relative costs of the parts of software development. Agile assumes high risks, high integration and collaboration needs, low predictability and low cost of change.

 

Scrum Gathering Prague Focused on Agile Leadership

Keynotes of Niels Pflaefing “Organize for Complexity” and Brain Robertson “Holacracy: A Radical New Approach to Management” summarized the agile position on leadership. In a rapidly moving world old paradigms of management are not fast enough. Teams near the customer must do the decisions using the power that organization’s constitution assures them.

More information about their ideas can be easily found from their books: Organize for Complexity and Holacracy .These ideas of leadership are essential in understanding why agile transformations are so difficult. It is hard to be agile if the management structures are not changed.

Another hot topic was scaling. SAFe is not the only option and LeSS seems to get more support from the agile community than it, because SAFe’s top down approach is less capable of  changes.

Scrum Alliance has now more than 400 000 members and it is growing with about 6 500 new members every month. The number of trainers and coaches is now 188. Members will get more content in Scrum Alliance’s site and trough the learning consortium.

Scrum Gathering Berlin 22.9-24.9.2014

Scrum Global Gathering 22.9- 24.9.2014 in Berlin was sold out. Its 600 participants heard the good news:

1) The number of certified members of Scrum Alliance of  is now 350 000 and it will grow by 5 000 every month.

2) Certifications will be extended so that the number of the upper level of Certified Scrum Professional certificates will be determinately increased with the help of additional training courses.The 200 trainers and training companies  involved in the community will increase the range of advanced training courses in the  technical and the leadership areas.

3) Scrum is converging as the organizations in the area, Scrum Alliance and Scrum.org (8 000 members), agreed on a common definition of Scrum.

This means in practice that our offerings of current and forthcoming trainings at agile leadership, interpersonal skills, facilitation, agility, scaling, Lean, Kanban, Certified Scrum Developer, agile testing, agile architecture, etc, etc will get increasing global marketing support from Scrum Alliance.

Course development takes place in the spirit of the Lean Startup based on customers’ needs. We are able to offer advanced courses that our clients request as the course contents have already been prepared at the global level. Many of the MIF‘s interaction skills courses and others courses are inherent add-ons to basics of Scrum.

Reality over the Plan

Big front-end designs are bad, because they do not match the reality. The devils are really in the details. However, the cost of changes is considered too high and the illusion of predictability makes us close our eyes on reality.

In Scrum we say that we should focus on infrastructure and architecture in the first Sprints. Decisions about our software development environment, tools and architectures are not easily reversible. The situation is much worse in the construction business. If we are building a bridge or a tower of Eiffel we can’t start it again from the beginning if the base is not strong enough.

The case is not that bad in software development. We use software instead of electronic circuits just because it is easier to change software than hardware.

We need to have appropriate engineering practices to change Kent Beck’s famous cost curve. Extreme programming contains many practices that are needed to make the code easy to change. In addition to these we need solid architectural principles and rigorous attitude to quality.

PowerPoint architectures outlined at the first Sprints are not good enough. We need executable architectures and extensive testing with highest priory business functionality to make sure that the quality attributes, non-functional requirements, are OK, before we continue deeper.

We don’t stop to that. We require that the architectures are easy to change. Ease of refactoring and testing can be achieved with known design patterns.

My colleague has written a provocative blog entry about the current situation of agility.

Actually, we are promoting our new course: Agile Engineering Practices, which can be used as a part of Certified Scrum Developer curriculum.

Visual software design with themes and epics

My talk in Scum Gathering Amsterdam is now visible. See more about visual software design with themes and epics

The place: 10:00 – 11:00am on Tuesday, November 16 in Foyer

Synopsis: We have issues like user stories, themes, epics, UI mockups, business rules and acceptance tests that are used in creating our understanding of what to do and how. We groom product backlogs and have Sprint planning meetings and design tasks in Sprint backlogs. This IdeaCamp session pursues to tell us how to put these all together in real life projects.

The slides and result flip charts are now available at SlideShare. I like especially the idea of drawing users’ value stream with epics shown by one of the groups in the idea camp.

New lean and agile books

I bought some lean&agile  books to read during my summer holiday. The list is not complete because I have already read quite many of them. I got Mike Cohn’s new Scrum book freely from the publisher. Thanks about that.

“Scaling Lean and Agile Development: Thinking and Organizational Tools for Large-Scale Scrum: Successful Large, Multisite and Offshore Products with Large-scale Scrum (Agile Software Development)”
Craig Larman;

“Agile Testing: A Practical Guide for Testers and Agile Teams (Addison-Wesley Signature)”
Lisa Crispin;

“Leading Lean Software Development: Results are Not the Point (Addison-Wesley Signature)”
Mary Poppendieck;

“Agile Product Management with Scrum: Creating Products That Customers Love (Addison-Wesley Signature)”
Roman Pichler;

“Coaching Agile Teams: A Companion for ScrumMasters, Agile Coaches, and Project Managers in Transition (Addison-Wesley Signature Series (Cohn))”
Lyssa Adkins;

“Kanban”
David J Anderson

ReWork: Change the Way You Work Forever”
Jason Fried;

“Lean Architecture: for Agile Software Development”
James Coplien;

“Drive: The Surprising Truth About What Motivates Us”
Daniel H. Pink

“Switch: How to Change Things When Change is Hard”
Chip Heath;

Command and control with pair programming

I found a revealing discussion at a popular suomi24.fi-site titled Ketterä IT-helvetti (agile IT hell ). It is amazing how often daily Scrum is practiced just opposite of Scrum’s basic idea of a self-organizing team. This 15 minutes is just for the developers to say hello to each other and coordinate days activities. They need to know the situation after yesterday’s work because software development is not predictable. Sometimes tasks estimated to take 2 days take 5 days and sometimes they are ready after one day. So everyone needs to know where the team is and agree on what to do next. The metaphor of chickens and pigs is meant to make sure that managers do not use that for micromanaging the people.

Command and control is bad because it destroys peoples’ natural work motivation. People including developers want to achieve something and autonomy to do it – not that they are given orders and controlled tightly. Command and control reduces productivity and creativity which is really harmful in an art like software development. Psychological studies have shown that time after another.

Pair programming can be done in a way that does not decrease the motivation. As the code is owned by team and every team member is responsible of changing any peace of it the team needs a way to teach the secrets of the code modules to everyone in the team. That is pair programming. It prevents harmful specialization and at the same time decreases the number of errors in the code. Your pair is not watching and controlling what you are programming but you two work together to create great software. Two creative minds coding and discussing together is a way to learn from each other and do a great work. Pairs are changed normally on daily basis so that different angles on the problem can be thought.

Pair programming should not be a stressful situation like the one described in suomi24. The team should discuss about the situation in its retrospective and decide how to continue to find the the joy of work that agility gives when it is practiced at its best.

Less extreme Scrum

Agile software development has often seen as an foreign element leading to chaos in Virginia Satir’s model. Current situation of Scrum is indeed somehow chaotic. Key persons are arguing against each other and it has been very demanding to define what Scrum actually is? Ken Schwaber’s ScrumGuide is our current definition but it is not ready for a Scrum Alliance’s multi-choice exam. There are many different versions or variations of Scrum which may or may not be under a common framework. Common thing in these is that they are more realistic and less extreme.

It has been visible for a while that radical extreme Scrum has given space to real life. Some of these modifications clearly belong under the title “Scrum but” but some others are coming from Schwaber and Sutherland, who are the authorities who define Scrum. In the London Gathering Schwaber introduced product backlog refactoring meetings where team collaborates with the product owner to create actionable product backlog items. In Munich Gathering Sutherland emphasized that the user story must be ready for the Sprint, which means that a good enough specification exists to continue. The main difference here between Rational Unified Process and Scrum is in the collaboration between the developers and the product owner. In RUP there are strong roles and artifacts that are just handed to next group of people. Product owners’ role has got more content by the discussion of release planning – still obscurely defined time-box of Scrum.

We are now also openly talking about undone features and telling that team might not be able to complete a user story to the point that it can released. So we add a stabilization Sprint where undone features are completed before the release.

Situational leadership model was my favorite in the 80’s. In addition to Tucman’s model it was also well visible in the Munich Gathering. Actually that means that we have to admit that leaders can’t just assume that teams self-organize when they are just empowered to do so. Sometimes directive forms of leadership are needed though we understand the drawbacks of command and control.

Schwaber’s integration teams that he introduced in his book Enterprise and Scrum have not been heard much in talks of the Scrum people. The case studies of real life projects tell clearly that defined organizations are used, not just self-organizing flocks of business and technical people. The question of scaling agility is interesting and clearly not solved in a way that theoreticians and practitioners can both accept.

Actually, I would like to asses (or rather measure) different agile solutions instead of arguing about right or wrong Scrum.

Losing weight with Scrum

Scrum is simple and hard. So is losing weigth for the majority of people. The secret of losing weight is that you eat less. That is that simple, but doing it is hard.

We can apply empirical process control to control one’s weight. Let’s have a daily weighting every morning to create a burn down chart of your body mass. Then you tell your group of peers what did you actually eat yesterday and what are you planning to eat today. This is basically how weightwatchers do it.

Eating less is hard and so is software development. Deciding what do (that is called prioritizing), limiting work to capacity under pressure and keeping the commitment are tough things to do in real life.

Disclosure: My own weight is normal and has always been.

Scrum Team availability

If you read XP or Scrum books or online sources you find that team is a stable group of developers who work full-time to accomplish the goal. The recommended way of estimating its capacity is to use historical velocity and team members report effort remaining in hours to have daily updates of the burn down charts.

In agile estimation the estimate of the complexity of the features, often expressed in story points, is the easy part though not trivial. In my thesis I have a small experience report about the variability of the feature sets, but I will leave that now. Instead I want to talk about the variability of the team’s capability to implement what they have promised.

In real life the organizations are not so lean that they have dedicated teams for the development. Instead, each developer has responsibilities in multiple projects and also critical maintenance work that must be done as soon as possible. An of course, they are also vacations, trainings, corporate administration and common meetings.

Some the capacity variation mentioned in the previous chapter is known or can be estimated in the Sprint planning. Using just velocity averages assumes that this variability is irrelevant and averages out in a large number of Sprints. But if we know that most of developers have their vacation in July, then it is just reasonable to take that into account and adjust the promises accordingly. In my own Sprints I have asked every developer individually how many person days is he/she available for Sprint work. That is best guess of the capacity at the moment. An then the team limits the work into that. Naturally, there is a mathematical relation between velocity in story points and capacity in person days. So you can still use story points as your unit if you want.

We could argue about the best guess of the capacity. In historical average velocity we take into account the average changes of the availability of the developers, because we count only the story points of realized features. It could be better estimate of the capacity if the developers have optimistic bias in the Sprint planning. But in averages we don’t have known capacity deviations as we have if we ask the capacity from the developers.

The real variability of the capacity occurs during the Sprint. It is not enough that team members report the effort remaining of each task. Suppose that I have promised to be available 15 person days in the Sprint and I get an urgent need to put 10 days to something else. Then it is quite natural that I tell that as an impediment in the daily Scrum as soon as I know about it. The consequence might be that a feature is taken away from the Sprint. Obviously the burn down chart would have told the same thing but not so early.

I think that it is relevant to take availability variations into account in Agile projects, because they have impact on the stress level of the developers with all of its consequences. The variability has also consequences in releases especially when we work in a project which has many teams that depend on each other. Realistic daily view of situation is needed.

That’s it today. A long story of a small but important detail.