Run experiments

In the previous post, we saw why we couldn’t apply user stories or spikes for a big technical epic we’re working on: developing an efficient engine in addition to the default one for a subset of what the default one supports. With user stories, we have no control on what we’re doing, and we can’t commit on what we’ll deliver. Ultimately, it does not allow to show progress, because it doesn’t correspond to what we do.

The key here is that we are progressing. We know it. We could implement a basic engine, and validate that the performance improvement is what we could expect from it. We have an overall idea of big chunks of stuff that will need to be done to make it functional. We just can’t make it real according to agile by the book. But there is a solution: as this is what we do, let’s make iterative expirements come true.

I’d like to introduce a new type of backlog item: an experiment iteration. It’s a kind of spike slice, or lean startup applied to technical stuff:

  • Define an objective, that you may or may not achieve.
  • Timebox it. This timebox is the main acceptance criteria. The backlog item is over when the timebox is over, period.
  • Just do it until the timebox is over.
  • Validate non-regression.
  • When it’s over, the key is that you check what you learned. This is what you do in the validation/test/acceptance column. What you learned might be:
    • Ideally, a backlog.
    • Worst case, give up and rollback. You must always keep this option open.
    • We added this and that tool or feature and it’s ok.
    • We should add this to the non-regression campaign.
    • We couldn’t support that case.
    • This or that piece of code must be re-designed.
    • That part will take time.
    • We need to check what we want to get regarding this.
    • You should check if this works as you expect.
  • Decide where to go next, based on what you learned.

Note that code branching or conditional compilation are not valid ways to avoid risks of regressions. They are only ways to delay risk, and thus to increase consequences. All experiments should be implemented in the trunk, using feature branching if necessary.

The main difference with a spike or a user story is that we focus on learning. It is transparent to everyone. You will be expected to implement the objective, because it is not the priority. We also make discoveries about the product and the code base more transparent, because there is no limit to what you might declare as learned. It might also save you some time during retrospective or andon meetings, because you already introspected on many technical topics.

Iterative experiments should be run when large chunks of software are to be implemented without knowing where to go in detail. It’s not too big to fail, i.e. too big to succeed, but too big to know all details in advance. They should lead to more deterministic work like an actual backlog.

What do you think? Have you ever focused on what you learned? Do you have some feedback about it?

Advertisements

Why technical user stories are a dead-end

As said in my previous post, our team works in the intermediate layers of the product. I didn’t get into details, but one of the reasons why we need horizontal slices is that we have a huge job to do to improve performance of a 4-year old very technical product. The task will take several months. We have a direction to go, but we don’t know how to get there in detail. We need to iterate on it anyway, as a tunnel is the biggest risk of development. The common way of doing it is to iterate on small user stories. This is where problems begin.

First of all, user stories start with a user, and a story. We have none of these. We just have a big technical tool to implement. If we wanted to write user stories for each iteration, with a relevant title, each of them would be something like “as any user, I would like the product to be more reactive, because fast is fun”. Maybe we could suffix titles with 1, 2, …37 to make a difference between them. Some would prefer stating technical stuff like “as Johnny the developer of XYZ module, I would like more kryptonite in the temporal convector, because I need a solar field”… and would call it a user story. You see my point.

And we need a good title for a user story… sorry… a backlog item. Because it’s the entry point for the PO, the customer, the
developers, the testers,… to understand WHAT is to be done and more important, WHY it is so (which is why I prefer beginning with the why in the title). Stating how we do stuff is irrelevant, because it’s only a dev matter. It doesn’t help understanding each other in any way. Which means we don’t have any clue about how we will agree on its doneness. In other words it’s a non-sense to commit on such a story. So we have a first dead-end: we need good descriptions , but we can’t provide them in a “user story” way.

The performance project is big because the new engine, that “caches” the default one, needs to implements a lot of rules of the default engine. And releasing it is not relevant until it implement enough of these rules. Dividing the work in rules has no sense, first because we need to build the basis of the new engine, which is already a big work. Secondly because implementing a rule in the new engine is not equivalent to implementing it in the default one. The algorithms, the engines, the approaches, are different. What is natural in one environment is a lot of work in the other one, and vice-versa. In addition, the work is so huge no one can wrap his brains around the godzillions of details we need to implement. Finally, we don’t know in detail the rules implemented by the default engine. Said another way, we continuously discover what needs to be done, along the way. Even in each backlog item, we are discovering so much we can’t estimate precisely enough at the beginning, or even make sure we agree on the same thing to be done. Not to mention longer term planning (which we will see in another post). Why denying it? Discovery is our everyday reality.

So we have no user, no story, no clear short-term scope, but we agree on the fact we need small items to iterate on. But wait! Agile has the solution: spikes! We timebox a short study, which we might iterate once or twice, and we know how to go on. But that’s what we already did for 3 or 4 iterations with a team of 4 developers, we are progressing, and we still don’t know clearly what we will do in the next iteration. So spikes might sound nice, but when you are at the bottom of a technical Everest, it needs to be taken to the next level. We are not picking a logging library among 3 candidates.

So following the basic agile rules, we’re stuck. But there is a solution we’ll talk about in the next post.

Code re-use can be as bad as code duplicate

Code duplicate is one of the top evils in software development. But as strange as it may sound, code re-use can be a pain in the eyes too. Let me explain.

Our team works on intermediary layers of the product. Yes, sometimes horizontal slices are the way to go. Our product is about monitoring. It’s made up of a home-made database, an intermediate layer that defines and computes analytics, a framework to define specific applications, and the applications built on top of the platform. Our team works on the analytics. The main thing here is that we build frameworks. It’s actually what I always did. The problem with frameworks is that you never can anticipate every use case. They are here to provide tools, not end-user features.

For this kind of code, BDD-like tools are difficult to apply. BDD needs you to know precisely enough what you’re going for when you commit on a backlog item. And it requires to come up with a limited set of features to test. With such technical code, though, you can hardly state the new required behaviors as user stories. And the main thing you want is the whole to remain stable and efficient. Before validating new features, you want to test non regression. After a while, it becomes difficult to test it manually. And automated tests are not perfect by nature: the more stuff to test you embed in the automated test, the less maintainable and fast the campaign will be. You thus make trade offs, which test is all about. You can hardly test a re-used tool correctly.

We can even get down another step. A piece of code that is re-used by a flip load of modules is a nightmare to maintain because we never really know how it’s used. Even if you run a set of unit (or automated in any way) tests you rightfully trust, you are not sure the piece of code is used the way you test. It’s particularly true for code that’s been here for a while: when an additional module uses your code, it’a miracle if the guy implementing the calling code thinks about adding the automated test that guarantees the called code does what he expects. And please allow me to look somewhere else with a strange face if you tell me that 100% code coverage guarantees your test base is nuclear bomb proof.

A few years ago, when we were having regression issues on our code base, I was surprized to realize we wouldn’t have such problems if we had not shared a particular piece of code. It hurt a basic rule I had believed in for years. And then I realized I was only experiencing the need to think about another trade-off we always need to balance: sustainability vs sustainability.

Perfection is a verb

Ok, this sentence sounds like a fortune cookie. Maybe I’m turning buddhist, but I love it. And it  applies especially well to software development. As a craft based on human interactions, software development is a set of compromises. There’s no such thing as a best practise, a definitive process, or a methodology that works for all contexts. I don’t even understand how words like agile and process or methodology can have ever been put together in any mind.

Agile and lean are approaches that work the best according to what we know today. But they are just containers of values and practises that constantly evolve. They don’t tell you what words to use in discussions, or what lines of code to write for every single situation. They need to be adapted to every context. Scrum and lean startup by the book might be good enough to design a simple shopping cart, but not necessarily to design a car, a distributed NoSQL database, or a nuclear plant software complying with all required regulations.

You always need more than what you can learn in trainings or books. Your situation is always unique and new. If it’s not new, why bother doing it? If you didn’t need to adapt the framework, why would scrum define the basic team around the opposite powers of dev, backlog, and test? Why would lean be based on kaizen in the first place? A set of definitive rules, a process, would be enough.

In the end, you always need to adapt the way you build, collaborate, understand each other, find solutions, analyze gaps, code… and to get better at it. Always. There’s no end to getting better. Because you can always be better, and because the context keeps evolving. And that’s what I love about it. You always can and must do better. Everytime you’re not trying to get better, you’re getting a bit closer to the death of your team, project, organization, company. The world won’t wait for you.

So perfection is the goal, but you can never reach it. How exciting! That’s what the best approaches known so far explain to us right from the beginning. You need to find the right compromise, for every situation: test coverage vs test campaign maintainability, code performance vs readability, taking the time to discuss and understand each other vs saving meetings, writing just enough documentation vs focusing on working software, self-organizing teams vs facilitating their progress, releasing features vs stabilizing the product… There are hundreds of parameters you need to balance. And you always need to tweak the buttons as the situation evolves.

In the next posts, I’ll try to talk a bit more about the tradeoffs we are faced with day by day. What the books don’t say, and the concrete problems they don’t give solutions to. I’ll try to describe the approach we’re taking to oil the wheels. I hope it will save me the shrink.

Estimating, what is it good for?

We just had a very interesting discussion about estimates. We started this discussion with re-assessing the weight of our story points, but it lead us to talking about why we need estimates.

I think the ideal goal of any framework or methodology, as for managers, is to disappear and lead to anarchy (but not chaos). Therefore, I look for any occasion to get rid of iterations, estimates, follow-up, or any other method or tool. I’d like to get to a minimum process focusing on communication, and then make it disappear as it becomes natural.

Estimates can be useful, but they are expensive. And we weren’t clear about where they were useful. So we tried to think out of the box, and rebooted our thinking about the reasons why we needed estimates.

In a recent post about transitioning to agile, Dean Stevens said:

[We can] explain some benefits of an Agile Transformation this way:

  • Business wants predictable and improved throughput.
  • Customers want quick lead times for quality product.
  • Teams want a good working environment where they can contribute and succeed.

I thought the ultimate goal of estimating was to come up with a long term planning (few months) with some confidence. From my experience, customers demand some planning for their requirements.

But I was explained that getting to that end to sell something was a commercial failure. Long term planning is never reliable. It is not when it comes from the provider, and it is not when it comes from the customer! I know you rarely deliver exactly what you planned a few months ago. And does your customer still exactly needs what he requested 6 months ago? Is he still ready to use it now that you’re delivering him?

Negotiating such plannings in a precise way is a poker game, where everybody looses.

You can still have a high level long term planning, but be clear on the fact that it is nothing more than an unclear wish-list based on the knowledge you have today. Don’t constrain your development process for this objective. Manage long-term planning outside of the development process, using tools like portfolio management or epic/theme backlog, where you estimate your epic/themes in comparison with other [done] epic/themes, and not development items like user stories.

The agile community agrees on the use of the discussion around estimates. I totally agree with it. Estimating is a great way to make sure we have a discussion. It’s a great way to detect comprehension gaps between team members (devs, qa, business,…). It also allows devs to agree on smaller chunks of the story (tasks) to run in parallel in order to swarm. By having this discussion, and the objective of a consensus around estimates, we have a better chance to align our comprehension of the why, the what, and the how.

But estimating is only one way to get there. We could get to the same point with a different objective, like colorful or timely drawings, a list or a number of tests, mock-ups,… So again, if the goal of the estimate is only the discussion, if it’s a game among others to run during the iteration planning, we should not make the rest of the process heavier for it.

What about the process? Through estimates, you get a  throughput. By making this throughput predictable, you make the process and the backlog predictable. By optimizing your throughput, you optimize your whole process. By detecting deviations in the throughput, you detect exceptions to your standard process (i.e. deviant deviations), and thus optimize it. And so on…

But there are other ways, like following up the cycle time, the WIP, and thus bottlenecks. You can monitor your throughput as a number of items, and not a number or story points. If your items are within a reasonable size, the standard deviation of your throughput and cycle time should be acceptable (compared to estimating error or your constraints). Agreeing whether an item is of a “reasonable size” is estimating. But I believe it is a lot lighter than formal methods like planning poker.

It leaves us to the short-term planning. Estimates are useful to know what you’ll do in the next few weeks. Like for long-term planning, I’m not clear on the goal of this. Making sure you respect due dates is definitely a fair objective. Motivating teams with an objective to reach is another one. Prioritizing considering constraints and slack is also important. Having a prioritized backlog where teams can pull work without bothering a PO is great. Fine, we’ll suppose you need to know what you’ll be able to produce in the next few weeks. And drum roll…

BUT is estimating useful to have a prioritized development backlog and a short-term planning? It is if your throughput and cycle time standard deviations are too important. If they are, you need to sort your items by categories where these standard deviations are acceptable.

As a conclusion, what you need is buckets where cycle time and bandwidth occupation are close enough for all items. If you only have one bucket, it’s perfect. If you have more than 4 or 5 buckets, you should really think about it. Estimating is knowing which bucket fits your item, or splitting an item so that all sub-items can fit one of your buckets.

For different levels of plannings, you can have different buckets. For example, you may have development buckets for the development backlog, and release buckets for the long-term backlog. Those buckets are not of the same nature. Neither are the items they contain. Don’t try to fit development items in long-term backlogs, or long-term items (epics/themes) in the development backlog, without further analysis.

As a conclusion conclusion, monitor your throughput, your cycle times, and their standard deviations, identify your buckets, and make them the least possible.

What do you think?

My stories are too small

We want to have backlog items that are as small as possible. It protects us from the unknown. It prevents us from having to re-work too much code if rejected, and limits the effect of wrong estimates. It allows us to be predictable.

In the same time, backlog items must have a meaning. We want to be able to release something valuable. Done-done also means useful. If it’s not, how can you adapt your backlog to actual needs? What’s the value of your high-level estimates? If something considered as done is in fact half-done, how can you release it? Splitting stories also means you can stop at the end of the first part.

I’ve never found the right recipe. And I just bumped into this situation again. The need is something like “I want to use an object of type A in screen B”. To use this object, I must first select it, so I need a selection screen. Then I need a select button that actually allows to update screen B given the object selected. The thing is… just implementing the selection screen doesn’t fit in the iteration.

Note: we are in an iterative mode but the issue would have been the same with a flow. The story is too big, and we couldn’t split it.

Only implementing the selection screen doesn’t make any sense. If we had to release at the end of the iteration, the user would have a selection screen with a disabled select button that says “to be continued”.

We chose the smallest issue: split it and deliver something that is not usable or useful at the end of iteration, but deliver something anyway. We prefer feedback than nothing, partial waste than entirely in stock. But still I’m not satisfied, like always in this situation. And finally we have this ugly story: “I want to use an object of type A in screen B, part 1”. So…

Feedback is warmly welcome. Have you encountered this situation? What do you suggest?

As an employee and a developer, I don’t want to

Work with a direct manager (or sole decision maker) I never see.

Follow a plan that is not adapted [anymore], because stakeholders agreed on it.

Work on a project that is never validated against reality (users, actual needs, integration, deployment, tests…).

Spend lots of time writing or maintaining doc that is never read, will become irrelevant as soon as it’s written, or duplicates existing information like the code.

Face impediments that are never removed… ever.

Work for a company that doesn’t allow to take any micro decision because:

  • Of blaming.
  • It doesn’t have other goals than “increase money”, thus no direction regarding the actual work.
  • Mistakes are not allowed.

Spend hours in meetings managing problems I would have solved by working.

Spend hours in meetings sharing, or providing tools with, information that is already publicly available, or data that could be directly derived from it.

Put hands back on code I left days ago because of bugs/regressions.

Work on a born-dead project, because it was budgeted 18 months ago, or because it was budgeted even earlier and budgets are based on history.

Not having resources that would save us money, because it’s managed by another department which sole objective is to minimize its budget (e.g. IT).

Work for a company that encourages fire-fighting instead of anticipating problems.

Be slowed down by the organization I work for.

Optimize things that don’t get in the way.

Work for a company that manages human resources.

Hear that a decision that gets in the way is strategic/the rule.

Not have a dream to make come true with my company.

As a human being, I want to have the opportunity to succeed, and make it happen.