Frugal production

Let’s think about this enterprise philosophy hit song:

  1. I want to satisfy the needs of users, buyers, managers, stakeholders.
  2. Therefore I want features.
  3. Therefore I want to optimize the production of features.
  4. In other words I want to maximize the production of features.

Imagine that big bank. It wants to solve problems for its advisors, or even its customers. With some help from specialists at every step, the bank formalized requirements in a backlog, organized RFI and RFP, selected a software package, started a project, set up constraints to make sure everything is delivered on time and budget, mounted and unmounted every corresponding team.

At every step, the bank relied on the result of previous investments, results of which were validated, and thus considered right. If everything done before is right, we only have one thing to measure when developing: the speed of production according to specifications. I may be user stories, man.days, lines of code, green tests in a campaign, whatever.

We maximize production of features because we don’t want to come back to what was already validated, and because we don’t know how to do all the steps at the same time. In other words, because it is easy. As a consequence, we measure the accumulation of things on top of accumulated things. This logic is the cause many a failed project.

Jeff Patton teaches us that the goal is to optimize outcome (i.e. changing behavior) to improve impact (i.e. consequences for our organization), while minimizing output (i.e. production).

How to do that concretely? As said earlier, we maximize output because it is easy. Therefore solutions will be hard. This is good news: it is an occasion to take some advantage over your competitors.

Validation

We must move quietly. Take the time to verify that what we throw out the door is useful. Consolidate fundations before adding new floors.

As Jim Benson says, “Software being ‘Done’ is like lawn being ‘Mowed”. Software gets interesting when it gets into users hands. It is finished when it is decommissionned.  Make this official, by adding a validation/understanding/learning/feedback step at the end of the value stream.

A feature to release is not code to produce. It is an outcome hypothesis. John Cutler insist on talking about bets. Once the code is released, we need to evaluate its consequences, and decide where to go from there:

  • It’s perfect we can stop iterating.
  • We should try modifying this or that.
  • We need more info.
  • Let’s deactivate or remove it.
  • And so on.

By the way, if you doubt, like you should, of features usefulness, you should limit the number of experiments you run in parallel. An experiment takes time to reveal its secrets. This delay is actually an interesting topic to think about. Among other things, it helps understanding why our so-called experiments are not scientific. Cynefin rather talks about probes.

Pull system

Limit your WIP (Work In Progress, i.e. the number of things being worked on), in all steps of the stream, including studying/prioritizing. By doing so, you will avoid preparing items when the next step is not ready to take it. Your backlog will thus remain under a reasonable limit.

  • The first step of the stream is a prioritized list of problems to solve, ideas, wishes, unverified asumptions. You study these topics by priority, when you have the capacity to take them. That is to say, when it’s useful to think about them.
  • Then you can think further about them: “what for”, split, explicit the goal and constraints, share understanding…
  • Then development if needed, test, deployment, and so on.
  • And then, validate the need, gather feedback to iterate.

So, step by step, you push features from the last step: impacting the world.

Of course, it only works if items are small. How small? If you’re beginning, it’s never small enough. If you have more experience about flow, and you know why it’s too small, then improve your process to decrease the transaction cost, and then make items smaller.

Wrapping up

Let’s fix the logic described above. Instead of:

  • Users have needs.
  • So we must produce features.
  • So we must optimize feature production.
  • I.e. we must maximize feature production.

Let’s try:

  • Users have needs.
  • We might satisfy those needs with features.
  • So we must optimize feature production and impact.
  • I.e. we must minimize feature production while maximizing impact.

When you use a product, you are delighted when you can see right away the feature you need. It’s a nice surprise to use it feature fluidly, the way you expected. It’s a change, compared to those products crawling under menus of sub-menus to propose all potential options, endless forms to support every possibility, just in case.

Propose the product you love using:

  • Verify features usefulness.
  • Do less things in parallel, and finish them.
  • Focus on users.

It’s never too late to do things properly. It’s always time to validate the hypotheses induced by upfront investments, however huge they are.

Now let’s go with code.

Advertisements

Enough is enough

This series of article is translated from French articles on my employer’s blog, Arolla. The French articles may or may not be released at the time you’re reading this.

We consume too much. We eat, throw away, heat, send e-mails, spend, earn, too much. We need to learn how to do more with less.

We have limitless backlogs. We look for ways to produce more faster. Production is our main indicator. As a company or a country measuring its income growth to invest more to grow more, we measure our production to release more to have more features to earn more. Simple isn’t it?

We miss two parameters here:

  1. Complexity doesn’t grow linearly with size. It tends to grow in a chaotic and explosive way. Sometimes independantly of growth. And surely in an unpredictable way. The worst news is, complexity doesn’t have a maximum. It is not capped by your capacity, anyway.
  2. You can’t predict how the system you’re creating will evolve. It is a kid, living its growth in a chaotic way. You can’t predict consequences of the evolution of your system, as soon as it gets a little bit complex.

I can only see one way of keeping that under control: move slowly, carefully, checking how the system evolves while you touch it. In other words, evolve frugally.

Because frugality diserves loads of ink, and I’m paid by the article, I’l try exploring this topic in 4 steps:

  1. Let’s start with backlog.
  2. What about code?
  3. Did you forget the process?
  4. Let’s get back to serious work.

People are better in the unknown

We are currently discussing about adding a team in another country to our product (single collocated team in Paris). This is the kind of discussion we’re having:

Program Manager: “We need to deliver Feature A in 6 months, and I don’t want to allocate more than 40% of our throughput to it. 40% need to remain for the expedite, and 20% for the support. Feature A alone would take about 80% of your current throughput, so we need a plan to double your throughput.”

Dev Director: “We have issues hiring here. And even if we can, it takes about 6 months from opening the job offer to having the dev in the office in Paris. So we think we’d rather open the job offers in Romania where it takes about 2 months instead of 6.”

The first thing that strikes me in this kind of conversations is that we consider people as resources (I didn’t coin the term in the discussion to avoid offending these virtual personas). When I say resource, I mean a linear resource, with a predictable and somehow linear behavior. I’m not offended being compared to a keyboard or a chair when I’m referred to as a resource. I’m just always surprised argumenting with someone with experience and responsibility, who never realized that 1+1 is not 2 when talking about people’s throughput. 1+1 might be 1, it might be 3, or 5, or sometimes 2, it depends.

A team has a complex behavior, and it’s very hard to predict it. If you add X% people to a team, you won’t add X% throughput. In general, you’ll even lose Y% (Y not being necessarily <100) for a given time. Even if you consider interactions, you won’t be able to come up with an equation that can predict a team’s output.

While I was thinking about it, I bumped into two awesome [series of] articles: people are resilience creators not resources by Johanna Rothman, and the principles not rules series, by Iain McCowatt. They helped me put words on my point of view.

People don’t have a linear behavior. They learn, they socialize, they create bounds, interact, and create together. We are not predictable. Let’s just realize it and deal with it. And you know what? That’s what we are great at! We are great in the unknown, at adapting collectively.

We won’t be more predictable or efficient by following a process or a precise plan. Or at least not very often. Actually only in cynefin’s simple quadrant. And when we’re adapted to bound processes and predictability, we’re a good candidate for being replaced by a machine, which would do better than us. Think about automated tests being great, for example, but mostly for known knowns. The organization’s interest is not to get predictable behaviors out of people. At best, you may get somehow predictable throughputs out of stable teams, but you don’t want to have more.

Back to our original discussion. Whatever the plan, we don’t know what will happen. Given the time frame, the business context, and the code base we’re working on, we are quite sure creating a team in a different country, in a different language, will have a negative effect for the release. But we’re not sure about it. We think creating a team in Romania will have a more negative impact than growing the team in Paris, but we don’t know about it. We think it might have a positive effect on throughput after some ramp up period… I could go on for pages.

The thing is, the system doesn’t want to be sure about any of these assumptions. It’s not the system’s interest. If the system could predict these assumptions, then people would have predictable behaviors, and it would be a bad thing for the organization.

So let’s start with a belief (e.g. “we can’t hire in Paris”), a value (e.g. “face to face collaboration”), a hypothesis (e.g. “a remote team could improve the throughput of this project”), a strategy (e.g. “scale that team”), and experiment/iterate on it.

When should I…

Most agile posts I see are about finding the right bargain about this question:

When should I gather requirements/specify/test/merge to main line/document/integrate/deploy/communicate progress to customers/<insert any phase gate you’re used to and concerned about when you think about iterative/continuous software development>?

The answer is: always!

If it hurts, do it often. It there is risk, do it often.

I won’t go through every possible question, you’ll find them in every consultant’s blog. There are two short answers to all of them:

  • It depends: from your situation to the ideal one, there is a gap. You must deal with it and find the right trade-off.
  • Do everything more often. Every time you delay something, like working on a branch, you don’t decrease risk: by delaying risk, you increase risk.

These answers come together It’s an and, not a xor.

The ideal situation is: every character you change in your code works, brings value, is crystal clear to developers and users, and instantly available to your stakeholders. This is ideal. But we can be ambitious: it is the goal. Everything we do must tend towards this goal.

A veto killed the team

You’ve worked for several days on a complex problem with a couple of teammates. You are proud of your approach and your code, and the three of you high-five when the CI server and testing confirm everything works fine. But the guy shows up, has a look at your code, and tells you that this detail is not the way we work here. You ask him for detail, try to convince him your solution is OK, but he doesn’t agree, and he tells you how to rewrite that part of the code. You know he’s considered right, so you just do as he says. The consequences are huge. You get 2 additional days of refactoring to comply to the rules, and you end up with a code you’re not so proud of.

Next time, you know you will ask the guy first. You will wait for his approval up-front, take his instructions into account, and do as he says. As you ask him, he will tell you all the details about how he would have done it. You know you’d better take notes of every detail, cause you don’t want to rewrite your stuff next time. And if anything doesn’t work as expected, you know you’ll ask for his advice.

You got a veto.

Veto is a smell: it happens once something is somehow decided, discussed, or done. Which means waste.

Veto tends to prevent teams from empowerment. If you need the approval from someone who is likely to put a veto, the result is very quick: people wait for detailed specifications, or micro management, from this guy. He will be the only brain in the team, and the bottleneck.

Counter-measures:

  • Allow mistakes. A team may be wrong when it decides something, which is OK. If you accept mistakes, you may have good surprises. What you consider as a mistake up-front might be a great solution.
  • Limit batch size. Small batch size means lower risk. Lower risk means higher resilience to mistakes.
  • Make sure vetoers coach before instead of using the card after. With great power comes great responsibility. Everybody’s job is to get useless. If you have the veto power, ask questions, like “what if”, “how will you know”, “how should we mitigate”… Get along with people, coach them.
  • Make sure the vetoers put vetos on actual issues with high risk and high impact, not potential issues, not actual issues that will be detected right on, not actual issues with low impact, not actual issues with known counter-measures.
  • Make sure your team is made up of opposite powers. This is the only guaranty to balance. If the vetoer shuts a silent minority, you have a smell. Only with attention can you detect it. If you have as much power as the vetoer, pay attention, so that veto only occurs in extreme situations.
  • If alignment is needed between teams, explain why and discuss about it. Note that alignment is far less needed than you may think.
  • Limit veto power.
  • Include people with veto power from the start of decisions (which is rarely possible, as powerful enough people are not available).
  • If nothing else can be done, remove the vetoer from the team. Every time I saw a necessary person get out of team, willingly or not, the outcome was surprisingly better. At least, you can run the experiment and see how it works.

The veto is a very dangerous tool. If you use it in your team, you must use it with extra care, only if all other tools failed, and only if the consequences of not using it are too serious. If you couldn’t convince people of your perspective before, or make sure they were able to take what you consider as the right decision on their own, you may consider it as a failure. But you shouldn’t use dictatorship to cheer up. Your team mates don’t have this tool.

Everybody has great ideas, and their own perspectives. Take benefit of that. Help the teams become autonomous, you’ll get great results.

Sweet WIP limit

Like many teams we adopted WIP limit. And like most of them we could see the benefits of this practice right away.

WIP stands for work in progress. Limiting WIP thus means limiting the number of items you work on at the same time within the team. When your team reaches the max number of items it can work on, it just stops starting new items.

By limiting the WIP we create slack. Which is exactly how I could finally introduce it in the team: “what if, when we reach the limit, we just did nothing?” Though it’s not as simple as this, fear of the void is the main issue to overcome when you want a team to adopt WIP limit: ultimately, when you reach the WIP limit, you stop working on the top priority items. Instead, you help to keep the items you started flowing towards their done doneness. But when you can’t help anymore, you just do nothing. Actually, the rules, when we reach the WIP limit, are:

  • You don’t start any additional item.
  • You help finishing started items.
  • If nothing can be done, you can do whatever you want, as long as it can be interrupted anytime to work on the project’s priorities.

So we created slack. And, as often described, the benefits to slack are huge:

  • Started items are done in a quicker way.
  • As a consequence, there is less uncertainty.
  • More reactivity.
  • Less interruptions.
  • Less WIP means less dependency issues within the team.
  • We have a tendency to pair more. Thus more quality, more tests, more refactoring.
  • More knowledge sharing because we help more each other.
  • We facilitate more other team’s flow.
  • We develop more tools that help us speeding up our flow.

Let’s look at the concrete stuff. We did it in a slightly different way than the academic way, or other teams on the project. Our flow is quite standard for a dev team.

Flow

Usually, you would have a WIP limit on Dev and Dev Done, and another one on Test. We tried something else. As we are a team of 4, we set a WIP limit of 3 around Dev, Dev Done, AND Test (actually it’s 2 backlog items + 1 bug at least major, but it doesn’t matter here).

Flow with WIP limit

We did this for several reasons:

  • As we have no dedicated tester in the project, we often test our backlog items ourselves. 
  • When a backlog item is rejected in test, it goes back to dev directly.

A WIP limit works because it’s applied to a production unit: a team. It’s normal to apply the WIP limit to everything the team covers. And even when testing isn’t in our scope, we are still supposed to take it into account, as the item being tested can still come back to dev. Testing doesn’t always answer yes. Kanban comes from manufacturing. In manufacturing, when an item isn’t valid, it is usually dumped, or sent to a specialized team. In our case, when an item isn’t valid, it consumes a slot to the dev team that produced it.

After a couple of iterations with this rule, I can confirm that the fear of the void is still the main impediment to overcome. We are cheating the WIP limit here and there, especially when managing bugs. We are still tweaking the WIP limit in order to make it fully functional.

I’ll keep you posted when I have some significant news. In the mean time, if you have any experience to share, please feel free to help.

Known or unknown unknowns we know we don’t know or not

In this video Adrian Howard talks about how to replace academic user stories with hypotheses to validate. In short, he proposes to ask more questions about what you need to do. You even need to question why, as this role, Bob wants that so that blah blah, or if he really wants or needs it. This is a great proposal. He justifies this by the fact we need to focus more on what we know we don’t know: the know unknowns. We tend to consider user stories as a specification. He wants us to ask more questions, even what the user story asks the team to implement.

The hypothetical user story is not in contradiction with my previous post about iterative experiments. I actually thought about this possibility, without getting as deep as Adrian Howard did. But I thought it wasn’t enough for our case, where we don’t know what we don’t know. Anyway, thinking about it, and putting it parallel with known/unknown knowns/unknowns, helped me understand something about what we do. When we tackle a topic, we get through several phases:

We explore. In this phase, we need to put our brains around why we need to do something, what we want to do, and how we should do it. We mainly don’t know. Even what we don’t know. We need to experiment and learn, about anything: the scope, the user, the stakeholders, the technology, the constraints… As we don’t know what we don’t know, we can’t have a precise goal to reach. As we can’t rely on the result, we timebox. As we want to learn, we focus on what we learnt at the end, and re-assess where to go from there.

We validate hypotheses. In this phase, we know that we don’t know stuff. We need to clarify what we know we don’t know. The best way we know to validate a hypothesis is the lean startup way: as cheap and scientific as possible.

We know what to do, how to do it, and why. Yes, this might also occur. In this case, a user story is the best when there’s a user with a story. Acceptance tests are also great. A specification might also do the trick.

Note that these phases are not sequential. We may start with a hypothesis to validate or technical backlog items to install a CI server. Neither do these phases describe the lifecycle of a project. A project is a sequence of releases, epics, topics, and each of these require exploration, hypotheses, and more linear implementation phases. Finally, you may need to experiment about what to do and be more confident about the technology. In other words, the why, the how, and the what may be tackled in different ways. As a consequence, these phases will be interlaced in a project, from the beginning to the end (I’ve never seen an end, but I heard it happens).

Now we know we need these three types of backlog items, so we need to know when to apply them. The problem is that we can’t know when we don’t know enough. One of these categories is about UNKNOWN unknowns. You may always pick the iterative experiment, but it’s not really the more efficient. And not the more predictable at all, in terms of planning. So you need to pick the right tool when you start with a backlog item. And this is were I will leave you with your guts. I don’t have a formula for choosing what a backlog item will mainly be about. After all, if you make the wrong choice, you’ll learn from your mistake. This is the beauty of doing small iterative things.

So what do you think? What do you know I don’t know I don’t know, or don’t know I know? Please share your experience.