Dear Mattias and Jeff.
I am writing to you as I can not sleep and I fell to thinking, or perhaps it was the other way around. Anyway, I remembered Jeff asking me to elaborate on my definition of success idea. And I can imagine you would like that too Mattias.
Questioning, distilling and trying to see a bigger picture have intrigued me for some time. At Lavasoft this drove me to discover and learn many things. This is one of those things.
At one point, trying to do things as smart and effective as possible no longer seemed good enough; there was simply no way to keep everybody happy, there were fewer of us and seemingly more work to do. A new approach was due. Taking a step backwards I asked myself with sincerity: Why are we doing these things? What is the intention? What are we trying to accomplish?
At the time I wrote the following:
I’ve been thinking a lot lately about how we almost never evaluate the end result of our work. We complete stories then we demo to evaluate the end product, then we have a retrospective to evaluate how we worked to realize the product. We almost never evaluate the effect or the success of our projects. In the end this is what really matters. I also find it very likely that doing this will be how we learn what kind of projects to accept, question, investigate, initiate and so forth.
Self-quoting, isn’t it sweet? ;)
At the time I called myself a Product Owner, however I found no guidance from agile in how to deal with these matters. Instead I found inspiration in the idea of Retrospectives and a concept: Definition of Done. The later is essentially an aid in the question “What are we making?” Asking in the same way: “Why are we making?” and I got Definition of Success.
The quick and simplified answer to these questions were usually: We are trying to make money (for the company). The particular answers turned out to be of no interest, asking the question however was paramount.
This is how to use it
in order to better understand Why to build What When.
[ TODO Big bad advice here, something like: “Or don’t. Don’t follow other people’s advice - it is too often harmful...” It's turtles all the way down! ]
Whenever someone would ask us to put something in our backlog we would talk to them to make sure that we and everybody involved understood why this should be done. In some extreme cases the person with the request was “stuck in the middle” and there because a third party wanted something that none of us really understood. Walking over to this third party generally made things clear for everybody, futile efforts was avoided and everybody had cake. There is nothing new to learn here.
Often however, the people involved came straight to us, and usually we all quite quickly understood why something seemed like a good idea. A “good idea“ isn’t good enough if you have too many. So this is where we would try to understand the intention or the desired effect more clearly…
Wow! Hi Mattias! I forgot that I’m writing to you and Jeff. I think I’ll keep this style though, the sparse usage of Software Development and Agile lingo forces me to think.
- Fixing a problem in an internal tool might have the desired effect “Reduce hours wasted by support personnel circumventing this problem”.
- An alteration to the web store might have the intention “Increase number of upsells” or “increase conversion rate after user adds products to cart”.
- A mailer might have the desired effect “convert free users to paid”.
- Updating outdated information on the front page might have the intention “make people trust us.”
(I know, this is scary; we ask our colleagues and ourselves to summarize complex things in simple words and recognize these as the truth. Though very helpful to catch the most crazy of things and the least questioned assumptions …if done too far or too systematic I’m sure you will arrive at a soulless place with no business.)
Wait… There is still nothing new to learn here.
This is usually where we make estimations from the gut. I would estimate the time required from us and the other party would put a measurable outcome, like: I expect to…
“Eliminate 2 hours wasted by support every day.”
“Increase number of upsells by 20/week, can be checked after one month.”
“Convert 2 000 free users to paid (can be checked 8 days after the campaign has gone out).”
“Make people trust us more(!?).”. This one obviously doesn’t fit the framework. I think this is where you should see that measuring things needs to live side by side with sense, emotion and instinct. (Don’t be a scientist).
Sometimes this is enough information to agree if the thing is a “Go” or a “Gone”. For example: The time required is small and the effect is really good. Or the time required is substantial and the effect is unknown and unmeasurable.
Often though, the discussion has to be delayed until more accurate estimations can be gathered.
After everybody agrees that something should be done the task/story/project may be updated to include the most relevant information, like this for example:
Fix problem with cake getting stuck in cakemaker tool
in order to reduce hours wasted by support.
Definition of Success: Support no longer waste ~2 hours per day on this.
Restructure the web store
in order to increase upsells.
Definition of Success: Increase number of upsells by 20/week, to be evaluated after one month.
Perhaps you can see the bigger idea here:
Build a habit that encourages everybody involved to talk about and understand what they are going to do/make and why. Encourage people to think and talk about what kind of outcome can be expected.
Furthermore, build a habit that encourages reflection and discussion on the effect of projects after they have made a dent in the business and the world. “Has it been a month since the web store updates?” – “Yes, it has and we have seen a trend in the number of upsells. We now have around more 100 per week!” – “OMG that is amazing!” …Or: “The campaign to convert free users to paying, converted 400. That is great … we expected five times that but it was still a success.” And everybody has cake.
If you use a board, such as Scrum, Kartoffel or Kanban, you can make sure that cards/stories have a “Definition of Success” before they move into “Ready” (unless too small to bother investigate or somebody used a “veto card”). Furthermore you can add a column: “To be evaluated” after “Done” on your board. Cards and their effect may then be evaluated together with stakeholders before being archived (the cards, not the stakeholders).
Understand that your Definitions of Success will be off. Like crazy. This doesn’t matter, the point is to talk about and recognize why we do things before and after.
I didn’t feel like I saw this through. Good things came of it for sure and I had a strong feeling that more good things would come had I not submitted to …fear?
Most of this information is probably only potentially useful in less than ideal situations. In a better scenario I think there is room for less science and more Power of the Flower.
This post grew into a crooked. And I can no longer see clearly. Posting now before this gets further out of hand.