Musings on Holistic Software Development
It’s pretty pointless building things that your customer doesn’t need or want.
The first steps necessary in understanding how to deliver value to the customer:
1. Identify the value. – This must be done from the customer’s perspective.
2. Identify how that value is actually created.
Lean thinking describes this as identifying and mapping the value stream – the activities that are performed to create customer value.
The value stream must be understood from end to end – from customer need to customer use. Taiichi Ohno described this as understanding the entire process of product development from customer order to receipt of cheque.
In large organisations it can be difficult for a team to gain direct access to, or even identify its customer. I’ve been working with teams in a large systems of systems environment. In this organisation the work that a team does is passed successively downstream to the next development team in the value creation process, each team in turn adding their part to the system.
In this organisational model it seems to many in the development teams that their customer is just the next team downstream. An additional effect of this organisational pattern is the explicit handover points in the value stream which cause transactional behaviour invoking Conway’s Law which influences the resulting system architecture, interfaces and communication paths.
For most teams, in the linear handover pattern, the actual customer who uses the system is invisible to all but the last team in the chain, it’s no wonder teams can’t identify their end customer or indeed see the value they create. This scenario is often made worse by the role of the customer being represented by a technical authority figure who is, with all best intentions, trying to coordinate the efforts of the teams to create the technical solution. This is a compromise that misses the point of directly engaging the customer in defining the solution.
A better way to organise to deliver complex systems of systems is to use Integration Streams which not only makes the customer explicitly visible to the development teams but provides the mechanism to scale complex agile delivery.
Begin examination of the value stream from the customer end where actual business value is realised. This is important as it avoids the problem of mistakenly identifying downstream teams as spurious customers in an extended and complex value stream.
Be certain to ask your consumers the “4 tests of value”.
Be wary of anecdotal evidence of working practices and product usage, including metrics. No matter how convincing these metrics look there is no substitute for actually going to see for oneself. There is no better way of understanding the value of your product than seeing it in use by the consumer.
When you have an better understanding of the value stream there are some hard questions to ask regarding the activities and processes you’ve identified:
Activity that contributes directly to customer value is valuable and should be maximisedActivity that contributes indirectly to customer value should be questioned and minimisedActivity that does not contribute to customer value is waste and should be eliminated
Value must only be considered from the real customer’s point of view, determination of value from any other perspective will result in local optimisations at the expense of the Value Stream, the customer and your organisation.
A customer focused mindset, continually questioning the value of each work activity is key to getting the most out of the feedback loops in continuous improvement methods, driving an organisation to maximising delivery of real customer value.
How much of the work you do is actually delivering customer value?
We’re often asked the question “How do I quantify value?”
For many businesses this is a matter of balancing risks against (Estimated Sales – Cost of Development).
It is the quantification of the risk element that poses a problem, but if the potential reward significantly outweighs the cost to develop then the decision to proceed is at the discretion of a Business Leader and can be judged based on the risk vs. outcome value.
For non-profit based organisations the financial element is eliminated and it seems as if it is impossible to place a value on the product, however, value does not always equate to a purely financial gain.
- A problem of theirs is solved or minimized
- An opportunity they desire is seized, maximized or enabled
- That they “look good” to significant others
- That they “feel good” about themselves
We have applied the above statements to many different scenarios in the public and private sectors and they always provide a way to consider whether any endeavour is providing value for its customers. The decision to proceed is again at the discretion and judgement of a Business Leader based on the risk vs. outcome value.
Value for the Customer, not part of the business.
It is important to remember that Value should only be defined from the end customer’s point of view of specific products and specific outcomes.
Value cannot be determined by intermediate stages in a value stream, only end consumers of a product/service can truly indicate value. This causes difficulties in large organisations where products and services are created and supported by large numbers of teams, many of these teams work in an environment that precludes direct contact with their consumers. In these situations it is very difficult to quantify the value of a product or service as the team in question only sees the boundaries of their particular sphere of influence, this means that they treat the downstream recipient of their part of the value stream as their “customer”. The follow-on team is simply part of the value stream and is a recipient of another team’s contribution to the product.
Focusing on the sub-parts of a value stream prevents true identification of Business Value and will lead to local optimisation of the sub-processes involved in product delivery, often local optimisation has a detrimental impact on the flow of the whole value stream.
To avoid this we advise examination of entire value streams with the perception of value to be taken from the end customers’ perspective.
Understanding Value is the foundation of Lean principles and key to understanding why and how anything is done in a business.
A project team must be able to identify and articulate the value it delivers if this cannot be done the project should simply be stopped.
There’s endless debate in the industry around contracting models and which is the best model, to further confuse the subject there’s always a “Yes but” in the mix, over the last 10 years its been “Yes, but what about contracts with agile” before that it was “Yes, but what about contracts with iterative development”
In a software context a contract is a written or spoken agreement that is intended to be enforceable by law between a customer and a supplier on how a product or part of the software development process will be delivered.
Holistic Software Engineering – Contracts
I’ve recently noticed a backlash by some customers against the Time and Materials contracts used by some suppliers using “agile” teams. This negative customer reaction is founded on the customers belief that their suppliers are failing to deliver and simply burning their money. I’ve seen the suppliers do little to help themselves in this situation by deploying ever-increasing resources onto these projects simultaneously increasing the burn rate and making delivery less likely.
The end game of this customer reaction is a reversion to “good old” Fixed Price contracts in an attempt to get the project portfolio and spending under control as the trust relationships between supplier and customer have broken down.
Long term fixed price contracts are unsuitable for high risk projects; risks are a source of variability that a fixed price model doesn’t easily cater for, we have seen many attempts to incentivise fixed price models, often resulting in negative behaviours by both supplier and customer.
The Big Secret
In reality long term Fixed Price contracts tend to be broken down into shorter fixed price phases. We don’t often see a 20 month fixed price contract, it’s normally broken down into a framework commercial agreement and then 6 or 3 month fixed price periods.Similarly a successful Time and Materials model is normally staged (review points every 3/6 months) with a cap placed on the maximum spend.Both models have the option to continue or quit at the phase/stage boundaries. Where both models fail is the inability of supplier and customer to make informed decisions on whether to continue or quit at the stage boundaries.
“Ultimately the best incentive for good performance is continuation of work.”
A continued revenue stream is of more value to a supplier than a paid for change request, or an extension of a couple of months to finish a project. Incentives in contracts cause negative measurement driven behaviour and are typically gamed dysfunctionally. Instead a successful supplier builds trust and has a track record of delivery, making them a strong option for future work. A supplier that fails will damage their relationships and track record, they’re less likely to be picked again.
This idea of using supplier history can even be formalised as the Weighted Fixed Price model, although that might be too formal in some cases.
A simple solution is to shrink the fixed price phases to a small amount of time (e.g. in line with a release cadence if there is one or every quarter if not). If using T&M then insert review points at the same points. These review points give a point for evidence based decision making where the evidence must be:
- Open and Honest communication between supplier and customer
- Working software, delivering some Features or Integration Scenarios
- Delivery of Business Value
The H-Model of Software Engineering combines the various different concerns in the nexus of Release Planning – from a requirements, architectural, planning, integration and quality perspective. But also, from a contractual perspective.
In either Fixed Price or Time and Materials the levers of control are already there, regular review points based on Releases – just use them.
I’ve been involved in process improvement for around twenty years, I think I’ve had some successes.
Death March projects are those projects that everybody has heard about where everyone on the team knows the project is pointless yet all the team members persist despite the feeling of impending doom.
The team may make attempts to correct the problem but are usually thwarted in their efforts to change. Death March projects persist for many reasons, to appease the instigators, contractual, economic, political etc but the delivery team can predict the outcome, often even at project kick-off meetings!
My first Death March was in 1992 (ouch) I was a C++ geek on a project looking for a purpose. It was the brainchild of the company Business Leader and had nothing to do with the companies core business. All the bad signs were there from the start, we had a free reign with the technology, but our usual customers didn’t know what we are working on or why – they were bankers, the product was not a banking system. From our normal customers point of view they had half of the developers in the IT department pulled from their normal duties, sent off on a program of training and were never seen again for the next 12 months – some of us they never saw again.
What we’d now call the Business Customer wasn’t interested in how his “pet dev team” were creating his “thing” nor did he give us much of a clue about it’s content. I think I saw him twice in his two year tenure with the company, although he did give his son a job in our team. There was clearly no purpose to the project, it wasn’t part of the Business Strategy and it was costing the business a huge amount of money at a challenging time for the finance sector, there was no visible customer, and nobody apart from our immediate Project Manager seemed to care about what we did on a daily basis. So why did we carry on for over 12 months?
Why didn’t someone in the team speak out?
Why didn’t the other board members or senior managers ask what was going on? We were all on the same site so there was no excuse for poor communication. We were spending huge amounts of money, we even got a Texan OO guru over – Putnam Texel (what a brilliant name) and paid her travel expenses and accommodation. I did loads of training, anything I fancied, learned loads of stuff played esoteric academic exercises in C++, read Grady Booch, Ivar Jacobson – “Hey I just learned use cases let’s try that!” Ed Yourdon, Pete Coad and many others.
So I guess I stuck with this Death March as long as I could because it was my job and I was being told to. In my inexperience I didn’t challenge that assertion from the people who paid me and so I tried to make the best of a bad situation by increasing my technical skills so I could actually help, but ultimately the futility of the project bore down on me and I couldn’t take it any more, I started looking in Computer Weekly (remember that anyone!) at the jobs market and I jumped to a shiny new job with a Systems Integrator – my Second Death March started the following Monday morning!
But what about the other team members, my mates, we’d worked together for about 5 years, we socialised and laughed and did stuff together. Well every single one of them either left the company or bailed from the project and then left the company, I can’t remember the exact figures but I’m sure over 50% of the team directly exited the company.
We didn’t object during the Death March because we thought somebody else knew better than us, surely the IT managers knew what was going on, surely the senior management team was concerned about the lack of governance and wanted to know what was really happening. It wasn’t our responsibility to question, it wasn’t our problem, and even if we said anything nobody listened or could do anything anyway. And if we kept our heads down nobody would notice. Some of my colleagues tried to escalate the problem, but there was a fear that it would expose us and they’d kill our project, we all had mortgages, some had kids, I had a car fetish.
My colleagues and I tried to apply some of the things we were learning to fix the problem, my motivation to acquire knowledge wasn’t just for the sake of it, all of us were absorbing any new idea that the industry was generating and we tried most of those ideas to address our problems, OO Analysis and Design to fix our product problems, Use Cases to try and fix the requirements problem, we used OO techniques to help with testing our components, this was all new stuff to us back then.
But none of it ultimately changed a thing, there was no tangible improvement because the whole thing was wrong from the outset, and we’d known it all along.
How do you identify an impending Death March?
- Project requirements are flaky,
- Technology is misunderstood or inappropriate
- Team is incorrectly or under resourced
- Sales team sold a product that can’t possibly be delivered in time,
- The customer isn’t, can’t or won’t engage with the team.
A combination of the above list, and other contributory factors create a feeling of nervousness and anxiety, doom even, but nobody publicly challenges the situation.
Why didn’t anything we did change things?
A complex combination of communication difficulties, a culture that discouraged negative reporting, the team’s fear of exposure, management’s lack of understanding of the teams problems, and our management’s own fear of questioning the objectives of a project which was so misaligned to the company’s strategic goals.
What would I do now?
“STOP!, escalate, protest, or even abandon ship!”
This may seem to be a high-risk approach but your actions could save the organization. It is difficult in such circumstances especially when one has financial and family commitments to put one’s head above the parapet and speak out.
Ultimately you need to ask the question
“Do I wish to work for a dysfunctional organization that cannot align its workbook with a valid strategy?”
If the answer to this is No, then your choice is either to change the organization or leave it; by trying to effect a change at least you’ll have done the right thing. Consider the future of organisations that fail to deliver to a valid strategy, ultimately their fate is sealed by their customers or shareholders.
What should you do if you’re in, or about to start, a Death March?
Check-out Holistic Software Engineering we’ve developed HSE to prevent your Death Marches.
My thanks to Ed Yourdon for his 1997 book “Death March” that resonated so strongly with me, thanks also to all of my colleagues especially if any of you recognize “the project”.
I’m a lover of history, a follower of technology old and new, I love the fells of Northern England.
My professional life is almost exclusively focused on helping organisations large and small operate more effectively. I’m a coach, mentor, and a listener, thinker and sounding board for ideas. My greatest pleasure is in seeing the people I work with succeed.
I’ve spent around 25 years in the software industry, starting out as a C++ developer, subsequently during my career I’ve performed most of the roles involved in building software. For the last 20 years I’ve been driven by a desire to make software development organisations, teams and individuals more effective at what we do, this has involved everything from studying new technology and methods through to training and leading development teams.
I’ve worked in many different business sectors from energy management and distribution, finance, motor and aviation, law enforcement, to telecommunications. I’ve extensive experience in both public and private sectors and delivered training to literally hundreds of IT professionals around the world.
Having observed the evidence of the power in self-organising teams I am an advocate of agile methods, but I am also mindful of the challenges that agile raises in large organisations as they struggle to adopt agile and reap the benefits of modern iterative methods.