Business Case of Agility

Success stories and surveys support agile software development. In addition to anecdotal evidence there are little hard figures to guide our decision making. Here cumulative business value charts are used to describe the impact of agile and traditional choices to the bottom line. Focus is on visualizing the economic impact of individual agile practices and assumptions of their costs and profits.

A chart describing the cumulative earnings as a function of time is a basic tool for optimizing the return of investment, ROI. The line is higher all the time if we have multiple deliveries. It is still better if the more profitable increments are deployed the first. This advantage in time-to-market is clear also in the surveys. Double the results in half of the time assumes that earlier deliveries have business value and that the additional cost of more deliveries is small.

  • It is not trivial to find minimum viable products that create value especially when organizations do not improve their businesses continuously but use large projects with non-negotiable late deadlines. Waste of the inventory of the partially done software is not as visible as the waste of tangible inventories.
  • The cost of a delivery has been a show-stopper of agility. Business models were based on sales of new versions of software that were installed manually to each client computer. Each deployment package had to be tested and integrated manually. Without automation the costs can overcome the benefits of early deliveries.

The cost of change must be small if we proceed empirically and feedback directs our product and development process. Customers pay these changes when they create more value than the costs of delaying and implementing the the changes.

  • Extreme programming proposes a set of practices like test driven development, re-factoring and pair programming that change the cost curve, cost of change, such a way.
  • Waste of unnecessary features can be avoided by prioritizing product backlog items based on their business value and changing the order when appropriate.

Risks are an integral part of any design. In software development we do not commonly know what the users actually need and how we use  new technologies that create the results. The cone of uncertainty is high. Reports of more than hundred-fold effort differences are common. Sales, savings and profit estimates are so inaccurate that it is quite common to ignore them altogether.

  •  Whole investment is lost if  we do a wrong thing or fail in the implementation
  • Customers try to protect themselves by fixed price projects. Costs increase because vendors have to buffer their offers especially if penalties are used. Competitive bidding weight visible price or over quality and total cost of ownership. The bidding game sets customers and vendors against each other.
  • Incremental deliveries make risks visible and adaptation is easier.
  • Agile approach is a natural risk management system, but it often fails when appropriate action is not taken. Organisation cultures must allow cancelling and redirecting projects.
  • Queuing theory and theories of variation can lead to improved practices in handling unpredictable situations.

Traditional mass production uses specialized lowest costs work force, rigid processes and invests in tools. It locally optimizes the cost of the tasks that workers do and is often blind about the required administrative burden.

  • Cost of learning can be significant. It unavoidable when we do something new. Open workspaces, pair programming and shared responsibilities are agile practices that accelerate learning. It would be good if that could break Brook’s law.
  • Cost of teamwork is minimized in a cross-functional team in single location. Wastes of relearning, transfer of work, task switching and delays are minimized. Software development has dis-economics of scale because a lot of communication and coordination work is required.
  • Utilization rate of the specialist become easily low. To avoid that developers take part of multiple simultaneous projects which often leads to coordination chaos.
  • Cost of motivation is difficult to estimate but essential part of working life. Leadership style has an impact on the engagement of team members which may result in more value for the business.
  • Traditional big front end design tries to minimize the cost of rework but real life shows that a large amount of work is needed to correct the bugs, integrate the components and deliver what users actually need.

Traditional and agile approaches have different assumptions of the relative costs of the parts of software development. Agile assumes high risks, high integration and collaboration needs, low predictability and low cost of change.

 

Better Profitability with Lean Principles

Lean thinking is particularly well-suited to today’s battle of margins in a weak economic environment because it comes from Toyota’s innovations in post-war market, where demand for cars was occasional, sparse, intermittent and difficult to predict.

Customer Driven

In customer driven business model it is important to sell what the customers need and reserve production assets just after the customer’s order. Sales and marketing should emphasize customers more.  That sells better than pushing what we happen to offer. Listening to customers is an important way to find tacit signals necessary for developing the products. We can meet the needs by preferring simple and fast solutions and avoid the tendency to invest heavily and slowly. Consultation business can begin lightly without extensive materials and finalized products.  Consultants learn by doing, working together with the customers.

Sales should not be overly concerned about the availability of the experts and resources. There are always more quotations than deals. Reservations should not be done before the commitments are clear.  Transparency helps everyone to adapt to rapidly changing situation in available consultants and speculations and realizations of the deals.

Flexibility levels loads

The ideal situation in which the company acquires the resources only when needed is not possible. Prices are higher if you cherry-pick.  Transaction and purchasing costs must be accounted and availability of the resources in peak demand is not a sure thing.

Profits as sales increase

Profit with fixed costs (green) and without (red)

Profit calculation is more difficult when the equation includes fixed costs. Annual rent of a typical agile team room could be 10 000 €. The charge for the corresponding space in Technopolis is 400 € per day.  If the rented space would be used 10 days a year, it would cost € 1 000 a day. If we could use it 100 days a year, it would cost € 100 per day. It would save a lot if the same space could be used as the office space for the team, internal meeting room and meeting room for the customers. You have to invent how to profitably change the configuration of the space to fit the needs of the moment. It is probably cheaper than having separate rooms for each purpose or renting more space each time.

Businesses are  crazy about premises in Finland. They have always too much space and in too costly locations. Home offices have a clear competitive advantage.

The situation is the same if we look at the profitability of the sales.  If the profit of a consulting day after variable costs like accommodation, travel and the consultant’s fees and the avalanche of the general expenses is € 200 to cover the seller’s costs, each salesman must sell  500 days a year to reach € 100 000 gross margin paying a typical 5 000 € monthly salary for him. If sales are lower keeping the margin by increasing the selling price makes it more difficult to sell in a tense competition situation.

ICT professionals have full-time employments because getting good people in bad terms is not possible. There is a long learning curve for each task and there is no visible end of the demand of their skills. Vendors’ success factor is a small cross-functional team in which all members are able to do virtually any of the tasks in the backlog. In addition, the team members could be responsible for a product development, support, sales support and marketing (especially brochures, blogs and public appearances). This ensures useful work throughout the year despite of the typical seasonality.

If an organisation is heavily based on specialized roles in its business model the weakness in demand is immediately reflected in the bottom line. It can not guarantee adequate income to its partner network and which fades away focusing on other customers.

Flat organization

Lean organization does not require a strongly specialized and multi-tiered hierarchy of managers because teams will make virtually all operational decisions. Maintaining and calculating profit margins is easy because organization’s general expenses are not significant. To get any work done creates a huge amount of coordination work if every task has its own manager. It is a slow and expensive. Trade of squirrel skins, purely speculative budgeting, performance management, etc. will add operations its own heavy share in the bottom line.

Economic monitoring is simple. Lean measures throughput times in addition to sales and gross margins.

Scrum Gathering Prague Focused on Agile Leadership

Keynotes of Niels Pflaefing “Organize for Complexity” and Brain Robertson “Holacracy: A Radical New Approach to Management” summarized the agile position on leadership. In a rapidly moving world old paradigms of management are not fast enough. Teams near the customer must do the decisions using the power that organization’s constitution assures them.

More information about their ideas can be easily found from their books: Organize for Complexity and Holacracy .These ideas of leadership are essential in understanding why agile transformations are so difficult. It is hard to be agile if the management structures are not changed.

Another hot topic was scaling. SAFe is not the only option and LeSS seems to get more support from the agile community than it, because SAFe’s top down approach is less capable of  changes.

Scrum Alliance has now more than 400 000 members and it is growing with about 6 500 new members every month. The number of trainers and coaches is now 188. Members will get more content in Scrum Alliance’s site and trough the learning consortium.

Scrum Gathering Berlin 22.9-24.9.2014

Scrum Global Gathering 22.9- 24.9.2014 in Berlin was sold out. Its 600 participants heard the good news:

1) The number of certified members of Scrum Alliance of  is now 350 000 and it will grow by 5 000 every month.

2) Certifications will be extended so that the number of the upper level of Certified Scrum Professional certificates will be determinately increased with the help of additional training courses.The 200 trainers and training companies  involved in the community will increase the range of advanced training courses in the  technical and the leadership areas.

3) Scrum is converging as the organizations in the area, Scrum Alliance and Scrum.org (8 000 members), agreed on a common definition of Scrum.

This means in practice that our offerings of current and forthcoming trainings at agile leadership, interpersonal skills, facilitation, agility, scaling, Lean, Kanban, Certified Scrum Developer, agile testing, agile architecture, etc, etc will get increasing global marketing support from Scrum Alliance.

Course development takes place in the spirit of the Lean Startup based on customers’ needs. We are able to offer advanced courses that our clients request as the course contents have already been prepared at the global level. Many of the MIF‘s interaction skills courses and others courses are inherent add-ons to basics of Scrum.

Example Driven Development

Example driven development is the best name to cover acceptance test driven development, behaviour driven development and specification by example approaches.

The concepts of acceptance test, testing and a tester have been defined long ago. I have found that changing their meaning is difficult. Even when we talk about test driven development we have to spend a lot of effort to explain that it is not done after programming and not by the separate testers.

The same happens if we talk about specifications. The old school uses user stories as a new way of writing specifications, which could be then send to a separate development team. Collaborative design is not happening as it should.

Though I like the idea of programming with natural languages, I think that they are too abstract and too prone to interpretations. Examples are a good way to explain things unambiguously and in a way that both the users and programmers see useful. Behaviour is something that programmers talk about, but for users that might not be as clear as examples.

With example driven development I want to emphasize collaborative design where users and developers work together using examples to clarify how users interact with computer programs to achieve business benefits that pay the return on investment.

Traditionally we have specifications, tests and code written to different purposes. They should not just be consistent but the same. In agile tests are specifications and in behaviour driven development code is moving closer to the tests.

The ultimate goal might be that we have only one description of the program that can be understood by both the users and the compilers. That is one goal of visual programming. The lack of programmers has been tried to be solved by moving the work to users. This approach has not been very successful because building large programs requires a lot of logical thinking and solutions to huge amounts of details. Software designers are not endangered species but programmers might be if the abstraction level of programming goes up.

 

 

 

 

Another booklist

In this list I have a lean and management focus:

Hope, Jeremy: “Beyond Budgeting: How Managers Can Break Free from the Annual Performance Trap”

Denning, Stephen: “The Leader’s Guide to Storytelling: Mastering the Art and Discipline of Business Narrative (J-B US non-Franchise Leadership)”

Denning, Stephen: “Radical Management”

Reinertsen, Donald G.: “The Principles of Product Development Flow: Second Generation Lean Product Development”

Reinertsen, Donald G.: “Managing the Design Factory: A Product Developers Tool Kit”

Goldratt, Eliyahu M.: “Theory of Constraints”

Ries, Eric: “The Lean Startup: How Constant Innovation Creates Radically Successful Businesses”

Gojko Adzic: “Specification by Example: How Successful Teams Deliver the Right Software”

Fowler, Martin:”Domain Specific Languages (Addison-Wesley Signature)”

 

Reality over the Plan

Big front-end designs are bad, because they do not match the reality. The devils are really in the details. However, the cost of changes is considered too high and the illusion of predictability makes us close our eyes on reality.

In Scrum we say that we should focus on infrastructure and architecture in the first Sprints. Decisions about our software development environment, tools and architectures are not easily reversible. The situation is much worse in the construction business. If we are building a bridge or a tower of Eiffel we can’t start it again from the beginning if the base is not strong enough.

The case is not that bad in software development. We use software instead of electronic circuits just because it is easier to change software than hardware.

We need to have appropriate engineering practices to change Kent Beck’s famous cost curve. Extreme programming contains many practices that are needed to make the code easy to change. In addition to these we need solid architectural principles and rigorous attitude to quality.

PowerPoint architectures outlined at the first Sprints are not good enough. We need executable architectures and extensive testing with highest priory business functionality to make sure that the quality attributes, non-functional requirements, are OK, before we continue deeper.

We don’t stop to that. We require that the architectures are easy to change. Ease of refactoring and testing can be achieved with known design patterns.

My colleague has written a provocative blog entry about the current situation of agility.

Actually, we are promoting our new course: Agile Engineering Practices, which can be used as a part of Certified Scrum Developer curriculum.

Visual software design with themes and epics

My talk in Scum Gathering Amsterdam is now visible. See more about visual software design with themes and epics

The place: 10:00 – 11:00am on Tuesday, November 16 in Foyer

Synopsis: We have issues like user stories, themes, epics, UI mockups, business rules and acceptance tests that are used in creating our understanding of what to do and how. We groom product backlogs and have Sprint planning meetings and design tasks in Sprint backlogs. This IdeaCamp session pursues to tell us how to put these all together in real life projects.

The slides and result flip charts are now available at SlideShare. I like especially the idea of drawing users’ value stream with epics shown by one of the groups in the idea camp.

New lean and agile books

I bought some lean&agile  books to read during my summer holiday. The list is not complete because I have already read quite many of them. I got Mike Cohn’s new Scrum book freely from the publisher. Thanks about that.

“Scaling Lean and Agile Development: Thinking and Organizational Tools for Large-Scale Scrum: Successful Large, Multisite and Offshore Products with Large-scale Scrum (Agile Software Development)”
Craig Larman;

“Agile Testing: A Practical Guide for Testers and Agile Teams (Addison-Wesley Signature)”
Lisa Crispin;

“Leading Lean Software Development: Results are Not the Point (Addison-Wesley Signature)”
Mary Poppendieck;

“Agile Product Management with Scrum: Creating Products That Customers Love (Addison-Wesley Signature)”
Roman Pichler;

“Coaching Agile Teams: A Companion for ScrumMasters, Agile Coaches, and Project Managers in Transition (Addison-Wesley Signature Series (Cohn))”
Lyssa Adkins;

“Kanban”
David J Anderson

ReWork: Change the Way You Work Forever”
Jason Fried;

“Lean Architecture: for Agile Software Development”
James Coplien;

“Drive: The Surprising Truth About What Motivates Us”
Daniel H. Pink

“Switch: How to Change Things When Change is Hard”
Chip Heath;

Origin of waterfall

A short history of waterfall

As experienced people remember the eve of software development was agile. Waterfall emerged because the prevailing management thinking embraced it. There was a quest for more rigor and predictability and we started to talk about software engineering which would improve our quality and productivity. More planning was an obvious solution when plans were failing.

When the systems grew the developers began to specialize, which was based on the prevailing mass production paradigm. Functional organizations were the norm everywhere. Because it was difficult to find skillful programmers, the tasks were divided so that cheaper and available labor could be used for defining, testing and documenting. Promotions to project managers enabled the traditional corporate ladder hierarchy.

Outsourcing was tried to solve the software crisis by decreasing the unit cost of the huge amount of work that was required for writing each single line of code only to find out that the distribution created another layer of complexity.

Massive process guides tried to catch every possible view of software development to make it predictable and repeatable. 80s was the era of methodologies. After that we got quality initiatives like CMM which created a quagmire of documentation and turf wars between them. It was still very difficult to tell how to create software because we have so many variations of it and its development. You had to write either something that is right and so generic that it does not help or something that must be applied or something which fits only to certain kinds of software development.

CHAOS reports told that the majority of software development project were not successful. Success factors were extracted and we found that the requirements and capability of making decisions are the keys to success. Systems thinking explains the failure of old straightforward initiatives. Specialization, outsourcing and rigorous processes answered well to one visible view of software development but they did not optimize the whole. Big front-end planning did not create better plans because the plans were frozen prematurely without appropriate testing and feedback.

In the 70s computers were less powerful than they are today. The programmer wrote the code to a paper based on which punch cards where created. Then the cards were read, compiled and finally executed. If anything went wrong the programmer had to make corrections and try again. The length of the development cycles might have been several hours if not days. Barry Boehm’s famous paper published in 1981 about software development economics was based on studies of the projects earlier than that. So it is natural that it concluded that the cost of a change is so huge that errors should be prevented at practically any cost. Reviews and careful table testing were the ways to prevent errors.

Waterfall was born due to the management paradigms but it is useful to consider the feasibility of modern iterative development using the technology of the past. Essential practices like continuous integration and automated testing were difficult but not outright impossible in all of the projects. Programming cycle times have not prevented agility after the punch card. The “mythical man-month” of Fred Brooks was written in 1975 but this was unfortunately not enough to change the history of software development.