Move fast, travel light

Traditionally developed software does not age gracefully. In fact, on average, it only takes about five years for newly developed software to deteriorate to a state where it is no longer economically feasible to maintain it. I.e. the cost of adding new features is larger than the income that the new features would generate.

Why does this keep happening? The answer is that through the years, business people have learned that developers are lazy. When developers whine about impossible deadlines it is just a matter of telling them that it needs to get done, or else… and then they will do it.

Developers, on the other hand, have learned many powerful ways to “speed up” the development when presented with impossible deadlines that “have to” be met. They can skip making the code readable, skip the unit testing, skip refactoring, skip the load and security testing, skip the code review, skip the documentation, … In this way they can easily double the speed at which they produce crap.

Of course the drawback of these techniques is that they do not really produce usable software. So a lot of time will have to be spent on fixing trouble reports. Typically 15-60 times as much as it would have taken to build quality in from the start. Also, over time, the code base gets more and more messed up and harder to modify. Thus, development will get slower and slower.

Naturally a project like this will overrun its initial schedule & budget. After a few years it will no longer be possible to maintain the system and the company will be in serious trouble.

The way to make this madness stop is to shift to iterative, value-driven development. I.e. each release is split into a number of iterations. Each iteration a number of features of value to the users will be completely developed. The most important thing with this is to establish a “definition of done”, a detailed specification of what it takes for a specific feature to be considered as “done”. If the definition of done states that quality shall be built in from the start, it will no longer be possible to “speed up” development by reverting to produce crap instead of working software. Business people will not be able to tell developers to work faster anymore, so instead they will need to start selecting the features for each iteration/release that give the maximum return on the investment.

One interesting thing about that is that an average requirements specification contains 64% features of little or no use to customers. These features will never be selected for implementation when doing value-driven development and thus the project time may be cut in half.

Here are a few ideas on what to put into your definition of done:

  • Code shall be covered by automatic units tests with at least 90% branch/condition coverage
  • The design shall have been refactored to integrate the new features in the cleanest possible way
  • There shall be no duplicate code
  • Coding standards shall have been met
  • The code shall be easy to read and understand
  • Documentation required for future maintenance shall be done
  • Cycliomatic complexity of the code shall be < 10
  • Maximum levels of nested code (if statements etc) shall be 3
  • The code passes some lint-type tool at some specified warning level
  • All automatic tests have been run with a dynamic checking tool such as Purify
  • The new feature shall be covered by automatic customer acceptance tests
  • All non-functional requirements applicable to this feature shall be covered by automatic tests
  • User documentation shall be done
  • Release documentation shall be done
  • The feature shall have been accepted by the customer
  • The feature shall have been installed at the customer site

Pair programming is very useful to make it possible to maintain extreme dicipline like this under pressure. For the same reason, whenever possible, try to cover your definition of done with automatic tests. Some useful tools for this are BullsEye, Lint, Purify, CloneDetective, …

The list above is just a quick example. You will be able to add a lot more to your list, and yet most teams will not be able to meet the example list initially. But just get started with whatever you can manage today and keep working on it. With time you will be able to expand your definition of done so that there is really nothing left to do when the code is marked as done.

When you get to that point, you will always be operating at maximum speed since there will be no old baggage slowing you down. At this point, few of your competitors will be able to keep up with you. That is a good place to be…

– Henrik

Co-located teams

One of the best ways to get started with an effort to get more agile is to co-locate teams, creating “war-rooms” for them to work in. Studies like this show that co-locating teams at least doubles productivity. In the study above, the teams that used the new facilities after the study was finished actually did even better, doubling their productivity again, performing four times better than the company baseline. Also, both the teams and their sponsors were in general very satisfied with the experience.

How can such a large improvement be possible? Being co-located makes very high bandwidth communication possible, enhances learning and supports creating teams that really jell together. Remember, the bottleneck in software development is learning and the best work is carried out by small self organizing teams.

In the research on co-locating there are also some really interesting findings on what happens when people are not “at hand”. Communication drops of significantly it seems, right when people are first out of sight. Being 30 meters apart is essentially the same as being truly remote. One would think that use of email, phone calls, and video conferences would make up for distance in today’s high tech workspace, but in practice this is not so. Professor Thomas J. Allen puts explains it like this: “We do not keep separate sets of people, some of which we communicate in one medium and some by another. The more often we see someone face-to-face, the more likely it is that we will telephone the person or communicate in some other medium.”

So, if you feel up to trying out some co-located teams, here are a few tips:

  • Provide facilities for teams to work in, but don’t co-locate a team until they request this. Remember, the basic rule for all change is that it does not happen unless everyone involved individually takes the decision that the new way working is of direct personal benefit to them.
  • People may feel that giving up a private office for co-located space is a loss of status. To handle this, a co-located space must be made an attractive option. Working in a well designed space like this should be made feel like a privilege, not as a cheap alternative to offices. So use your best spaces and don’t crowd them. Provide windows, air, plants and lots of first class work equipment like whiteboards and computers.
  • Have just one team (i.e. ideally < 10 people) in each room. Agile teams get noisy and that is the way it should be. Provide noise isolation to prevent disturbing other teams/persons.
  • Supply adjectant conference rooms for meetings and hotelling cubicles for work away from team, making private calls etc.
  • Make sure desks support pair programming by having space for two people working side by side, shifting keyboard back and forth.
  • Encourage each team to write down and maintain its own set of rules on how to work together. This could include work hours, handling phone calls outside of war-room, no speaker phones, phones on vibrator mode, which type of discussions to hold in the war room and which should be taken to conference rooms, signals for indicating that a person wants some privacy etc
  • When co-locating and shifting focus from individual work to teamwork, remember to also update your performance evaluation systems. Bonus goals and other performance metrics need to be measured at team level (or in general, one organizational level up) to prevent sub-optimisation.

So, get right to it and tear down those office walls now…for people working in solitude refining the latest statistics on the wheat production in the EU, offices may be great. For software development it is simply not good enough anymore…

– Henrik

Lean contracts and cooperation

The last part in our three part series about agile development and contracts will provide a glimpse into the future by looking at how thought leaders have managed contracts to foster cooperation between companies and to create superior performance in complex environments:

Are rigid contracts needed for protection so that companies will not take advantage of each other? Actually there is another way of looking at cooperation between companies. Once again, lean pioneer Toyota provides the example to follow.

When Toyota started to cooperate with suppliers in the US in 1998 they naturally did have contracts, but contracts were not the main instrument protecting the interests of the suppliers. In a short time the suppliers developed trust in Toyota and in 1998 Toyota was ranked as the most trusted automaker by all suppliers. Toyotas ranking was twice that of GM’s. Trust in this case meant that the supplier could trust Toyota to treat them fairly and not to take advantage of them.

Toyota worked with their suppliers, making them more efficient by improving their processes, but did not demand that they would cut prices as GM is known to do. Toyota realizes that long term relationships and partnership is more important than short term gains that can be realized by pressuring suppliers.

Toyota uses target-cost contracts composed of a base cost + a calculated cost for change. If the cost is exceeded, the parties negotiate on who should pay what. But Toyota usually pays the bigger part. This gives engineers at both companies incentive to keep cost down. It also keeps transaction costs down and makes it possible for them to work together with little bureaucracy and complete transparency. Very little money is spent on negotiating and managing contracts. That is important since transaction costs dominate most buyer/seller relationships according to “Dyer, Collaborative Advantage, 91-96”.

Toyota buys 75% of their parts in the US from suppliers. US automakers buy less than 50%. Yet Toyota spends only haft as much time and money on purchasing activities. In addition to that, suppliers have better quality and are more productive in cells devoted to serving Toyota.

Another interesting application of lean thinking was when the British Airports Authority, BAA, built the new T5 terminal at Heathrow. T5 is Europe’s largest and most complex construction project to date. To avoid the catastrophic performance and the 1000 % budget overdraws of other previous similar big construction projects; BAA did not try to sell the risk to subcontractors by using fixed price contracts. Instead they used cost-reimbursable contracts with incentive for contractors to improve performance. The T5 contracts were designed to get everyone working towards a common goal. BAA took on the complete risk and the responsibility for managing and running the entire project, including all subcontractors as if it were all one big virtual company.

BAA used cost information from other projects, validated independently, to set cost targets. If the out-turn cost was lower than the target, the savings were shared with the relevant partners. This incentivized the teams to work together and innovate. It was in fact the only way to improve profitability!

In return for its goodwill BAA demanded absolute transparency in the books of its suppliers; this created an approach in which all team members were equal and which encouraged problem solving and innovation in order to drive out unnecessary costs, including claims and litigation, and drive up productivity levels.

So, it seems that there can be trust between companies after all. The strategies outlined above fits perfectly with the goals and methods of agile software development. So software buyers, no time to waste, pick up the phone and give your purchasing department a call. Tell them that you cannot afford fix price contracts anymore and that you want them to do some research on a new type of contract for your next project…

– Henrik

Turning fixed price contracts into a win-win situation

Continuing our three part series about contracts and agile software development, here is part 2 about how to handle the fixed price environment that most of us work in today:

If you are an agile development organization, but you do actually have to submit a bid for a fixed price, fixed scope contract anyway, there is no way around adding an ugly up front requirements phase in front of your shiny agile process.

Doing this you can use the same estimating techniques here for an agile project as if you were doing a traditional project. E.g. check out Steve McConnel’s “Software Estimation, Demystifying the Black Art” for some good coverage of this.

But even if you are forced to add a traditional requirements phase (and thus waste a lot of money since on average you will be spending 64% of that time analyzing features of no value to the customer), you can still get significant benefits from using agile development:

  • An agile team will have higher productivity than industry average. You will be able to complete the project in half the time and have a nice profit.
  • An agile team is skilled in estimating since they work with estimates every day. They will also know their historic velocity. This gives you a better position to bid from.
  • In a high risk situation you can select to do what a really big and successful telecom supplier does. Before bidding they run 3 iterations on each big new project, taking the cost themselves. After this they have a measure of the velocity of the teams involved and can make bid with high confidence.
  • When you are awarded the contract, you can offer the customer to add an optional, risk free, agile clause to the contract. This clause is sometimes called “Money for nothing, change for free”. The clause will invite the customer to cooperate in an agile manner, making changes at no cost as they see fit as long as the total work is not changed. It will also allow the customer to stop the development whenever they want, paying a fee of 20% of remaining work as compensation for loss of opportunity. Usually 80% of the value of a system comes out of 20% of the features. So, when the customer has received 80% of the value, it can stop the development and not waste anymore time and money on developing low value features. The customer will pay 20% + a termination fee of 20%*80% = 36% of the initially agreed cost and will have the project delivered early. If the supplier was counting on a 15% profit for the time spent, it will instead get a 95% profit counting the termination fee also. And so the evil fixed price contract has been transformed into a win-win situation!

Once the customer has experienced the ease and power of being an agile customer, it is likely that they do not want to work in any other way again. Probably this means that there will be more business coming your way also. Also, the next time you work with a customer like this, there will be trust between the companies and you may not have to bother with the up front requirements waste and the fixed price contract.

In the next and final part in this series we will cover other types of contract that has been successfully used to encourage more productive cooperation between companies.

– Henrik

What’s wrong with fixed price contracts?

When presenting the agile approach to audiences new to the ideas, one comment is by far the most common. It is: “How can we do this, we have to work with fixed scope, fixed price contracts?”

We will cover this question in a three part newsletter, here is the first part “What’s wrong with fixed price contracts?”:

When practicing agile software development, the focus is on delivering customer value every month, sometimes even more often than that. Every iteration we do completely develop a few of the highest priority features. At the end of the iteration, ideally, the feature is finished and already deployed at the customer site. No work remains and the feature is actually done when we call it done. The customer knows exactly what the status is and may be able to start profiting from the partly finished system very early. The team benefits since it can forget about the previously developed features and move on at full speed to do new tasks. We also embrace change by allowing allow our customers to change priorities and add and remove requirements at any time as they wish. As long as we did not start working a feature already we allow this at no cost, no fuss. This is currently the most efficient and proven method to develop software known.

So, how does this relate to contracts? Traditional thinking is that by specifying and controlling scope you can protect yourself so that the other part will not take advantage of you. Trying to fix and control scope leads to some severe unexpected consequences though:

  • Trying to limit scope, actually makes scope grow. Customers tend to put everything they can think of in their fixed scope requirements specifications. Then they put some more stuff in, just in case. They know that if they do not get every possible feature into the requirement specification, the supplier will try to squeeze out lots of money for changes later. Thus, on average 64% of the features in the requirements specification are things that will be seldom or never used. I.e. compared to a project focused on delivering value, trying to fix and control scope has more than doubled the project cost by wasting time on documenting and developing features of no value.
  • Another thing that buyers try to do by using fixed price contracts is to transfer the risk to the seller. For this the buyer has to pay a premium. But who owns the risk really? If the project fails to deliver, the buyer usually is the one that will be in trouble anyway. So, in practice the buyer is always the one owning the risk. Even if there is a fixed price contract with penalties for delays etc, this is usually of little value if a system needed for the customers business is not delivered.
  • Another aspect with fixed price contracts is that controlling scope and managing change takes a lot of resources. According to “Dyer, Collaborative Advantage, 91-96”, these types of costs dominate most vendor-supplier relationships.
  • Fixed price contracts are often awarded to the lowest bidder. Bidders know this and make bids that barely break even. The profitability of the project is then saved by charging big sums for changes. On average, 35% of all requirements change during a project, so this is a safe strategy from the seller’s point of view. The problem with the strategy is that it fights change and puts strain on the buyer-seller relationship. Change is legitimate and there are numerous reasons why change will always be needed. Without change, optimal value will not be produced. Yet, fixed price contracts makes changes hard and thus it does not create the best possible customer value for the money invested. The reason behind this problem is that the contract type does not align the goals of the seller and buyer. Thus it creates a non-productive, inefficient, climate for cooperation between companies.

So, perhaps you agree by now that fixed price contracts are pretty useless tools if you care at all about getting giving your customers value for their money? Or perhaps you don’t care about value being delivered to your customers? In that case you better hope that all your competition feels the same. All it takes it that one of them starts delivering value, and then your offerings may not be that attractive anymore.

In the next part of this newsletter we will talk about how to turn fixed price contracts into something workable and mutually profitable by using agile development. So stay tuned, and maybe wasting time by arguing over contracts can soon be history…

– Henrik

The bottleneck of software development

What do you think the one activity is that takes up the most of the time for software developers today?

In a draft of a new book, Amr Elssamadisy presents this hypothetical experiment that allows us to figure this out:

Suppose I was your client, and I asked you and your team to build a software system for me. Your team proceeds to build the software system. It takes you a full year, 12 months, to deliver working, tested software. I then thank the team and take the software and throw it out.

After that, I ask you and your team to rebuild the system. You have the same team, requirements, tools, and software. Basically, nothing has changed. It is the same environment. How long will it take you and your team to build the system this time?

Common answers to this question by developers, many of them with 20+ years of experience, is that it will take between 20% and 70% of the time. I.e. Instead of one year it would take 2,5 – 8,5 months. This is a huge difference!

So, what was the problem the first time, what has changed so much? The difference is that team has learned about the requirements, about each other, about the tools etc.

Thus, learning is the bottleneck of software development.

This insight can be used to better evaluate your work methods. Instead of asking: “Will pair programming slow us down”, a more relevant question is “Will pair programming speed up or slow down our learning?”. Another example: “Will we learn faster if we show the system to our customer every other week or if we show it once a month?”

Keep an eye on the bottleneck!

– Henrik

Pair programming

A very powerful, but one of the most non-intuitive agile practices is pair programming. Before one has tried to get good at this, it seems unbelievable that it is actually is better and more efficient for two people to work side by side on the same problem than having two developers working individually.

If you want to get some inspiration and courage to try pair programming, check out this often referenced scientific study of the topic:

“Abstract: Pair or collaborative programming is where two programmers develop software side by side at one computer. Using interviews and controlled experiments, the authors investigated the costs and benefits of pair programming. They found that for a development-time cost of about 15%, pair programming improves design quality, reduces defects, reduces staffing risk, enhances technical skills, improves team communications and is considered more enjoyable at statistically significant levels.”

There you have it: The extra time of 15% that is spent when pair programming is paid back 15-60 times by reduced test costs, depending on the organization. So, pair programming is profitable, good for quality, increases work morale and builds teams.

So, if you have not done so already, go ahead and try now.

– Henrik