Another book list

In this list I have a lean and management focus:

Hope, Jeremy: “Beyond Budgeting: How Managers Can Break Free from the Annual Performance Trap”.

Denning, Stephen: “The Leader’s Guide to Storytelling: Mastering the Art and Discipline of Business Narrative (J-B US non-Franchise Leadership)”.

Denning, Stephen: “Radical Management”.

Reinertsen, Donald G.: “The Principles of Product Development Flow: Second Generation Lean Product Development”.

Reinertsen, Donald G.: “Managing the Design Factory: A Product Developers Tool Kit”.

Goldratt, Eliyahu M.: “Theory of Constraints”.

Ries, Eric: “The Lean Startup: How Constant Innovation Creates Radically Successful Businesses”.

Gojko Adzic: “Specification by Example: How Successful Teams Deliver the Right Software”.

Fowler, Martin:”Domain Specific Languages (Addison-Wesley Signature)”.

 

Reality over the Plan

Big front-end designs are bad, because they do not match the reality. The devils are really in the details. However, the cost of changes is considered too high and the illusion of predictability makes us close our eyes on reality.

In Scrum we say that we should focus on infrastructure and architecture in the first Sprints. Decisions about our software development environment, tools and architectures are not easily reversible. The situation is much worse in the construction business. If we are building a bridge or a tower of Eiffel we can’t start it again from the beginning if the base is not strong enough.

The case is not that bad in software development. We use software instead of electronic circuits just because it is easier to change software than hardware.We need to have appropriate engineering practices to change Kent Beck’s famous cost curve. Extreme programming contains many practices that are needed to make the code easy to change. In addition to these we need solid architectural principles and rigorous attitude to quality.PowerPoint architectures outlined at the first Sprints are not good enough. We need executable architectures and extensive testing with highest priory business functionality to make sure that the quality attributes, non-functional requirements, are OK, before we continue deeper.We don’t stop to that. We require that the architectures are easy to change. Ease of refactoring and testing can be achieved with known design patterns.

My colleague has written a provocative blog entry about the current situation of agility.Actually, we are promoting our new course: Agile Engineering Practices, which can be used as a part of Certified Scrum Developer curriculum.

Command and control with pair programming

I found a revealing discussion at a popular suomi24.fi-site titled Ketterä IT-helvetti (agile IT hell ). It is amazing how often daily Scrum is practiced just opposite of Scrum’s basic idea of a self-organizing team.

This 15 minutes is just for the developers to say hello to each other and coordinate days activities. They need to know the situation after yesterday’s work because software development is not predictable. Sometimes tasks estimated to take 2 days take 5 days and sometimes they are ready after one day. So everyone needs to know where the team is and agree on what to do next.

The metaphor of chickens and pigs is meant to make sure that managers do not use that for micromanaging the people. Command and control is bad because it destroys peoples’ natural work motivation. People including developers want to achieve something and autonomy to do it – not that they are given orders and controlled tightly. Command and control reduces productivity and creativity which is really harmful in an art like software development. Psychological studies have shown that time after another.

Pair programming can be done in a way that does not decrease the motivation. As the code is owned by team and every team member is responsible of changing any peace of it the team needs a way to teach the secrets of the code modules to everyone in the team. That is pair programming. It prevents harmful specialization and at the same time decreases the number of errors in the code. Your pair is not watching and controlling what you are programming but you two work together to create great software. Two creative minds coding and discussing together is a way to learn from each other and do a great work. Pairs are changed normally on daily basis so that different angles on the problem can be thought. Pair programming should not be a stressful situation like the one described in suomi24. The team should discuss about the situation in its retrospective and decide how to continue to find the the joy of work that agility gives when it is practiced at its best.

Visual software design with themes and epics

My talk in Scum Gathering Amsterdam is now visible. See more about visual software design with themes and epics The place: 10:00 – 11:00am on Tuesday, November 16 in Foyer

Synopsis: We have issues like user stories, themes, epics, UI mockups, business rules and acceptance tests that are used in creating our understanding of what to do and how. We groom product backlogs and have Sprint planning meetings and design tasks in Sprint backlogs. This IdeaCamp session pursues to tell us how to put these all together in real life projects.

The slides and result flip charts are now available at SlideShare. I like especially the idea of drawing users’ value stream with epics shown by one of the groups in the idea camp.

New lean and agile books

I bought some lean&agile  books to read during my summer holiday. The list is not complete because I have already read quite many of them. I got Mike Cohn’s new Scrum book freely from the publisher. Thanks about that.

Craig Larman: “Scaling Lean and Agile Development: Thinking and Organizational Tools for Large-Scale Scrum: Successful Large, Multisite and Offshore Products with Large-scale Scrum (Agile Software Development)”.

Lisa Crispin: “Agile Testing: A Practical Guide for Testers and Agile Teams (Addison-Wesley Signature)”.

Mary Poppendieck: “Leading Lean Software Development: Results are Not the Point (Addison-Wesley Signature)”

Roman Pichler: “Agile Product Management with Scrum: Creating Products That Customers Love (Addison-Wesley Signature)”.

Lyssa Adkins: “Coaching Agile Teams: A Companion for ScrumMasters, Agile Coaches, and Project Managers in Transition (Addison-Wesley Signature Series (Cohn))”.

David J Anderson: “Kanban”.

Jason Fried: “ReWork: Change the Way You Work Forever”.

James Coplien: “Lean Architecture: for Agile Software Development”.

Daniel H. Pink: “Drive: The Surprising Truth About What Motivates Us”.

Chip Heath: “Switch: How to Change Things When Change is Hard”.

Origin of waterfall

A short history of waterfall

As experienced people remember the eve of software development was agile. Waterfall emerged because the prevailing management thinking embraced it. There was a quest for more rigor and predictability and we started to talk about software engineering which would improve our quality and productivity. More planning was an obvious solution when plans were failing.

When the systems grew the developers began to specialize, which was based on the prevailing mass production paradigm. Functional organizations were the norm everywhere. Because it was difficult to find skillful programmers, the tasks were divided so that cheaper and available labor could be used for defining, testing and documenting. Promotions to project managers enabled the traditional corporate ladder hierarchy.

Outsourcing was tried to solve the software crisis by decreasing the unit cost of the huge amount of work that was required for writing each single line of code only to find out that the distribution created another layer of complexity.

Massive process guides tried to catch every possible view of software development to make it predictable and repeatable. 80s was the era of methodologies. After that we got quality initiatives like CMM which created a quagmire of documentation and turf wars between them. It was still very difficult to tell how to create software because we have so many variations of it and its development. You had to write either something that is right and so generic that it does not help or something that must be applied or something which fits only to certain kinds of software development.

CHAOS reports told that the majority of software development project were not successful. Success factors were extracted and we found that the requirements and capability of making decisions are the keys to success. Systems thinking explains the failure of old straightforward initiatives. Specialization, outsourcing and rigorous processes answered well to one visible view of software development but they did not optimize the whole. Big front-end planning did not create better plans because the plans were frozen prematurely without appropriate testing and feedback.

In the 70s computers were less powerful than they are today. The programmer wrote the code to a paper based on which punch cards where created. Then the cards were read, compiled and finally executed. If anything went wrong the programmer had to make corrections and try again. The length of the development cycles might have been several hours if not days. Barry Boehm’s famous paper published in 1981 about software development economics was based on studies of the projects earlier than that. So it is natural that it concluded that the cost of a change is so huge that errors should be prevented at practically any cost. Reviews and careful table testing were the ways to prevent errors.

Waterfall was born due to the management paradigms but it is useful to consider the feasibility of modern iterative development using the technology of the past. Essential practices like continuous integration and automated testing were difficult but not outright impossible in all of the projects. Programming cycle times have not prevented agility after the punch card. The “mythical man-month” of Fred Brooks was written in 1975 but this was unfortunately not enough to change the history of software development.

Unit testing EJB 3

My colleague at Tieturi, Arto Santala, has written a series of three blog entries about unit testing in EJB 3 environment. It is in Finnish but you might get the point through the code examples without our language.As an idea unit testing has been known and used for decades but examples in blogosphere focus on simple situations instead of complex cases of real life. When we do unit and acceptance testing in industrial scale, we need to go deeper considerations about maintainability and performance of the test harness.

Innovation requires skill, motivation and courage

Innovation is not just brainstorming with fancy techniques. Truly new business ideas need well grown skill, deep motivation to struggle through all the forthcoming difficulties and courage to put enough effort to make it successful.

A stereotype of an inventor is a mad man who works to create a perpetual motion machine. He has limited education to base his inventions but decades of hard work is a kind of substitute of that. Chances of success are limited because it is easier to be creative at the spearhead of science. Motivation is a required because innovation is not just heureka moments in brainstorming sessions but persistent work to try different things, ideas and variations.

Many of the ideas we get are indeed doable, but not commercially feasible or the manufacturing process is not feasible.I believe in intrinsic motivators, but money is not totally out of the table, because we need money to fund the materials and inventor’s time for a long journey of trials and errors – inventors use empirical process control. Many useful inventions are out of reach of individuals, because of expensive equipment that is needed. On the other point view, governments typically overspend because too much money is chasing too few ideas. Just think of ITER.

Courage is important because new innovations break our barriers of thoughts. Something that we believed, was found to be not true. Saying that is not easy. It is easier to let it be. So, employees have no reason to innovate. They get their salaries without and hunger overcomes the desire to create something great. People have ideas but they do not go ahead with them, because that will do harm. Remember that the majority of the ideas are not that great when studied more. And if it happens to be a success story, it is not just fair that your employer gets it for free. You might get a 100 € bonus 🙂

Universities are thought to be sources of innovation. Ph.D’s have the theoretical knowledge to synthesize new ideas but current world is very eager to make sure that the big companies get the results. Tight research funding narrows the focus of the researchers. You must complete your Ph.D in four years. To do so, you take part of a large team that is sure of about the results when they start.

BTW. I found an academic study of this topic. Creativity and innovation in the Systems Engineering process [PDF].

Merry Christmas and Happy New Year

I wish you merry Christmas and happy new year 🙂 It has been surprisingly strong upturn after the economic downturn in 2009. I have  been running CSM trainings one after another – the public ones are just a peak of the iceberg.

Based on the statistics this blog is not dead though I have not written anything for a while. The backlog of topics that I would like to write grows larger and while I read a lot – web, books and discussion forums I get still more issues to comment. I don’t make a promise for the new year to be more active even though I feel that way now in my Christmas leave.

Less extreme Scrum

Agile software development has often seen as an foreign element leading to chaos in Virginia Satir’s model. Current situation of Scrum is indeed somehow chaotic. Key persons are arguing against each other and it has been very demanding to define what Scrum actually is? Ken Schwaber’s ScrumGuide is our current definition but it is not ready for a Scrum Alliance’s multi-choice exam.

There are many different versions or variations of Scrum which may or may not be under a common framework. Common thing in these is that they are more realistic and less extreme. It has been visible for a while that radical extreme Scrum has given space to real life. Some of these modifications clearly belong under the title “Scrum but” but some others are coming from Schwaber and Sutherland, who are the authorities who define Scrum.

In the London Gathering Schwaber introduced product backlog refactoring meetings where team collaborates with the product owner to create actionable product backlog items. In Munich Gathering Sutherland emphasized that the user story must be ready for the Sprint, which means that a good enough specification exists to continue.

The main difference here between Rational Unified Process and Scrum is in the collaboration between the developers and the product owner. In RUP there are strong roles and artifacts that are just handed to next group of people. Product owners’ role has got more content by the discussion of release planning – still obscurely defined time-box of Scrum.We are now also openly talking about undone features and telling that team might not be able to complete a user story to the point that it can released. So we add a stabilization Sprint where undone features are completed before the release.

Situational leadership model was my favorite in the 80’s. In addition to Tucman’s model it was also well visible in the Munich Gathering. Actually that means that we have to admit that leaders can’t just assume that teams self-organize when they are just empowered to do so. Sometimes directive forms of leadership are needed though we understand the drawbacks of command and control.

Schwaber’s integration teams that he introduced in his book Enterprise and Scrum have not been heard much in talks of the Scrum people. The case studies of real life projects tell clearly that defined organizations are used, not just self-organizing flocks of business and technical people. The question of scaling agility is interesting and clearly not solved in a way that theoreticians and practitioners can both accept. Actually, I would like to asses (or rather measure) different agile solutions instead of arguing about right or wrong Scrum.