Software quality in the context of value chains and evolution

Markus Harrer

Dieser Blogpost ist auch auf Deutsch verfügbar

TLDR;

The importance of qualities can change with the evolution of a software system. It also becomes more difficult to find good arguments that convince users and product owners of the value of qualities that become more important later. I explain my view on this in this blog post. After reading this blog post, you will be able to understand this visualization and its impact on software development activities:

Introduction

A few years ago, a colleague and I helped a company solve their performance problems for a smartphone app (including the backend). We worked with the team in charge of the system to find out, what the current technology stack looked like, and where the problems might be. My colleague and I prepared ourselves with a few questions to better understand the system later on. On-site at the company, we started as usual with a small kick-off. One of the goals in these meetings is to find out if the team or teams know what the business model is that they are developing the software system for. Here I always like to ask, “How do you make money with the system?”. But this time, the answer came as a bit of a surprise to me. It was, “we aren’t making any money yet.” That threw me a bit off in the meeting. Later in our support, we also retained many of the questions about production-readiness (scalability capabilities, monitoring, error recovery and so on) we had prepared. After the initial workshops with the development team, we saw that such issues were not yet on the team’s agenda. There were different problem areas, which the users of the system (and thus the company) were more interested in, such as too slow queries, usability problems, or error-prone functions. What fundamental aspect had I missed here?

Change of scenery: A few years ago, I founded a small start-up with some former fellow students from my computer science studies. In our free time, we developed a web application for organizing and conducting bar-hopping tours (for readers that might not be familiar with the subject matter: A group of people gets together in a town and then they try to visit some pubs to have some fun and beers). We had very high expectations for the development of our software system. One of our most important goals was to prove that a start-up that used on clean-code principles from the start could be successful as well. Therefore, we built our application architecture according to the Onion Architecture. It allowed us to effortlessly put different frontend technologies on top of our business logic (which we did twice) and change the persistence mechanism (which we changed four times because it was so easy). Also, from the beginning of our project, we paid attention to extreme scalability of our software system by using various cloud services. We also fantasized about an ecosystem of plugins to make our modular software attractive to third-party developers as a platform. Ultimately, however, these were completely the wrong goals for our system. Our web application had never offered the features we would have needed to go live. What we had was also terrible to use. We gave up on our idea (we had kind of “luck” anyway, because of the COVID-19 pandemic, we would have been broke with an app that organizes such social events like bar-hoppings now).

For years, I couldn’t completely understand these situations. Today, these scenes are much clearer to me: of course, we had paid attention to the wrong quality goals in both cases. But in the case of the start-up, we also knew that the quality goals we had in mind would become necessary at some point to make the software systems attractive to customers and users in the future. So why wasn’t it wise to address all the qualities that a software system would eventually need as early as possible?

An important model that has given me more insight into when which qualities become important is Wardley Maps. Using Wardley Maps (invented by Simon Wardley), we can create many new perspectives on software development. This blog post uses Wardley Maps as a communication tool to better understand specific quality characteristics and their interplay with value and software evolution. No prior knowledge of Wardley Maps is necessary. I introduce Wardley Maps step by step below in the context of quality attributes.

But if you can’t wait to learn more about Wardley Maps right now, feel free to visit my recommended entry points into this topic and come back later.

Quality in context

To talk about quality in the context of software systems more specifically, I use another model that provides us with a common vocabulary to better talk about quality: The ISO standard ISO/IEC 25010:2011.

We also focus only on the main quality characteristics of the standard, including a lean description from me:

• Functional suitability: The system provides functions that meet stated and implied needs.
• Performance efficiency: The system delivers adequate speed with the resources provided.
• Compatibility: The system can exchange information with others.
• Usability: Specific users can use the system to achieve specific goals efficiently.
• Reliability: The system performs certain functions under given conditions.
• Security: The system protects information and data.
• Maintainability: The system can be modified to improve, correct, or adapt to changes.
• Portability: The system can be operated in different environments.

We also need to clarify what we even mean by quality in the context of software systems.

A definition, which is accurate in terms of content, but a bit too wordy IMHO, can be found in Helmut Balzert’s German classic “Lehrbuch der Softwaretechnik” (“the software engineering textbook”) which translates in English to “Software quality is understood as the entirety of characteristics and characteristic values of a software product that relate to its suitability to fulfill specified or presupposed requirements.” For me, this means that a software system is qualitative if the demanded quality is equal to the quality of the existing software system.

The first example showed a shortage of quality: good performance was required, but the system could not meet these requirements. We, therefore, had to find suitable measures to make the system perform better. In the second example, we exaggerated it with quality because they were not yet relevant to the current situation. However, we ignored other qualities that potential users and customers were expecting. Both types were quality deficiencies.

Ranking quality from a value’s perspective

We can use a Wardley Map to illustrate both situations (slow smartphone app and the useless bar-hopping software). In its simplest form, a Wardley Map consists of two dimensions: One dimension shows the visibility of components (systems, capabilities, knowledge, and more) in a value chain from some point of view (such as a customer or user of a software system). This dimension allows us to see, for example, how close and thus visible certain parts of a shopping app are to a user who wants to perform a value-adding activity such as “order a new graphics card” via an online store. We can list all the components, for example, to show what we need to do to complete this activity. “Need” is also an important keyword in this case, because the elements in the following list are in a “needs” relationship to each other to ultimately fulfill the user’s needs:

The list is read like “A user needs a new graphics card, which needs the shopping app, which needs app services, which needs REST services,” and so on. Components that are at the top of the list are more visible to the user. The user herself directly uses the smartphone app’s frontend. Components further down are getting more and more invisible to the user, which means she doesn’t need (and want) to care about them. But components further down are still relevant to realize the value-adding activity.

However, we now want to map individual qualities rather than individual components. We can create a similar list here as well. However, we cannot directly map a value chain since qualities are something less concrete. But we can list the importance of certain qualities again from a certain point of view. Let’s take the example “user wants to buy a graphics card” again as a starting point for our list. But this time, we list the qualities that the user expects according to the perceived importance:

To make this list more understandable, I add rough scenarios after each quality, explaining why I ordered it this way.

1. Functional suitability: If the shopping app can’t correctly process orders for graphics cards, we lose the user as a customer.
2. Usability: The user wants to find her way around and not be frustrated by a complicated user interface.
3. Reliability: The app should be usable when the user wants to order something to satisfy demand.
4. Performance efficiency: The user does not want to wait forever for feedback from the smartphone app.
5. Security: The user wants her personal data to be protected.
6. Maintainability: The user also expects new features from time to time that make the sales experience even better.
7. Portability: The user wants to be able to use the same app on different devices.
8. Compatibility: The user wants to use the payment method that suits her best.

This list does not claim to be 100% correct. Some qualities can be listed here from the user’s point of view very straightforwardly. I refer to these qualities as “external software qualities” because they are directly visible to outsiders. But for the qualities that are further away, an exact order is more difficult to define. I am talking about “inner software qualities” here, as these tend to be found under the hood of the software.Nevertheless, I have tried to argue the above order in a comprehensible way. In other contexts, however, deviations are undoubtedly possible here.

Nevertheless, this list already helps us to classify one of our previously mentioned examples better. The web application for organizing and conducting bar-hopping set completely different priorities than those that would have been important to a user, for example:

1. Maintainability
2. Performance efficiency
3. Portability

This list makes it easy to understand why this application was not a resounding success: it completely missed the quality needs of potential end-users.

A quote from Peter Drucker fits nicely here:

“There is nothing so useless as doing efficiently that which should not be done at all.”

The first example (system with the performance problems), on the other hand, had a more pragmatic approach: the essential functions of the application were already in place, and the team had a usability expert on board. Better scalability (“capacity” according to ISO 25010 and thus a sub-characteristic of “performance efficiency”) was now next on the agenda.

By now, at the latest, the doubt should have arisen that the model here is perhaps too oversimplified. You might have the impression that the same set of quality goals must always be achieved regardless of any specific needs. However, we know that quality goals are strongly dependent on the context (business models, constraints, capabilities, etc.) and therefore, almost every software system has a different selection of qualities to achieve. And not every software system has the luxury to build a 100% correctly functioning application that is also extremely easy to use. That is all true. But we look at quality from a very special point of view: From the software users. In addition, we have left open how far we turn up the knobs for a certain quality. Even with a minimal set of functionality and not too confusing user interfaces, we could already have met the needs of the software users.

However, I would like to make a completely different point with the “value chain” dimension: Don’t forget to think about the most visible qualities from the user’s perspective. This view will help us better understand why we sometimes do just great in developing software and have a easy life. It’s most likely the case that we’re delivering enhancements to these qualities. The dimension of value creation also shows us why software developers sometimes can have a hard time: When it comes to the “lower” qualities, the chances are that you constantly have to convince users (or people like product owners) that work in these areas is needed. From these viewpoints, they are too distant, invisible, and difficult to grasp for end-user-facing roles. As software developers, you need to know that this demands a lot of tact, like asking the right questions at the right moment for the further development of the software system.

Another critical point is that we also need to keep other points of view in mind. Often, software developers might have completely different sequences of qualities in their heads (see web application for pub crawls). The operations team, in turn, also has other quality expectations. Knowing these differences between the expected and visible qualities also helps us make any conflicts more transparent and communicate them better.

Quality and evolution

Our mapping of qualities does not end there, however. With Wardley Maps, we have a second dimension at our disposal, which we can use to take a closer look at qualities in the context of evolution. For this, we must undertake a small excursion into this “mechanism”. Components in the context of Wardley Maps can evolve. How much and how far they do so depends primarily on two factors: Demand competition (i.e., how much a component is demanded) and supply competition (how well a component can be provided). A Wardley Map maps these aspects with four evolutionary phases on one dimension. For mapping qualities, a rudimentary understanding of these phases is sufficient for us here, which I roughly characterize below:

1. Genesis: It is entirely unclear how to make the component or if anyone needs it at all.
2. Custom Built: Manufacturing processes are still volatile and not very repeatable, but there is already an initial need.
3. Product: The manufactured product is reliably produced with controllable processes, and the market buys it in large quantities.
4. Commodity: A strong mass market obtains what is offered essentially directly without any obstacles.

In Wardley Maps, components move through these different phases if there is corresponding supply or demand competition. Let me make a small example of how this can illustrate the evolution of computing capacity (inspired by a tweet from Simon Wardley):

1. Genesis: It is unclear whether automated computation is possible and who could use automated calculations. Nevertheless, there are first ideas and prototypes of calculating machines up to the first digital computer (e.g., Zuse Z3, the first computer in the world).
2. Custom Built: Automated calculations have shown to be quite valuable. The first digital calculators for commercial applications such as payroll accounting are created. These are made only on special customer requests because they are still difficult to manufacture, and there is little demand for them (e.g., LEO I, the first computer for commercial business applications).
3. Product: The first computers that separate the customer-specific computing processes (= software) from the executing computing units (= hardware) are available. Thus, the high demand for computing capacity can be met, and computer hardware can be produced in large quantities (e.g., IBM System/360, the first computer system that allowed to writing of machine-independent software).
4. Commodity: Computing capacity is now available as a matter of course, and the demand exists in almost every application area. It’s now possible to obtain computing capacity directly in the best quality at any time immediately without much effort (e.g., AWS Lambda, which manages the computing resources on-demand and is completely automated).

We can guess where our journey is going with the evolution axis from this and the other examples. Again, we can list the qualities one by one according to their relevance. This time, however, in the context of evolution. Here we need, however, no special point of view. We can roughly assign the characteristics of the evolution phases to the qualities, which are probably most relevant to evolving components further:

1. Genesis
   • Functional suitability: Without function, there is no use for anyone.
   • Usability: Without accessibility for users, there is no value.
2. Custom Built
   • Reliability: Without an available system, no increased demand for the offers.
   • Performance efficiency: Without fast response and scaling, there is no expansion of the user base.
3. Product
   • Security: Without data protection, there is no acceptance by the majority of users.
   • Maintainability: Without continuous development, there is no satisfaction of the increased demands.
4. Commodity
   • Portability: Without coverage of different operating platforms, no independence.
   • Compatibility: Without good access to service offerings via interfaces and standards, no mass demand occurs.

This sequence is partly subjective and gained from experience, but it also helps us gain some new insights. In particular, software developers need to know in which of the phases they are currently. In most cases, it makes no sense to talk about maintainability if we are still in an exploratory phase like Genesis. Measures for better maintainability, like introducing clean code principles and modularity, could even be a disadvantage. The probability is far too high that we will be wrong and then, thanks to the sunk cost fallacy, unnecessarily hold on to carefully created designs or an incorrectly modularized structure, thus blocking us even more in our daily work. We first have to find a suitable business model and transfer it into software that attracts potential users. The dimension of evolution, therefore, helps to recognize unnecessary work and premature optimization. But we also see that we always have to address new qualities when our software system is evolving.

Especially when we move to another phase, it may well be that the views on qualities fundamentally change. Suddenly, documentation becomes more critical to the success of our software as a way to improve maintainability as we move out of the phase of Custom Built by Product. At some point, API design will become necessary to address compatibility issues in a platform we provide if our software is to become a Commodity. Roughly classifying qualities along the evolutionary axis prepares us early for the challenges. This enables us to foresee which kind of work needs to be done soon if we would like to evolve a software system.

The assignment of qualities to evolutionary aspects also helps us when we are not evolving a new software system incrementally from scratch. Often we already know that there is a market for our software system by an existing demand competition. We also know the basic conditions that our software system must fulfill to be successful and to top possible competitors (supply competition). Thus we see that if we enter directly into a more advanced phase of evolution, we need to consider a more widespread set of qualities from the beginning. Otherwise, we will be without a chance against our competitors or will not meet the needs of our customers and eventually users.

Quality in value creation and evolution

Finally, we can also go one step further and bring both dimensions together. To achieve this, we use the value creation dimension from the user’s perspective and the ranking of qualities according to their relevance per evolution phase from above. The order of the qualities is the same in both cases because the sequence of qualities in the evolution axis also reflects the demand of potential users. Both dimensions brought together result in a Wardley Map: The dimension for evolution on the x-axis and the dimension for the value chain on the y-axis.

The two examples mentioned at the beginning are also included here as vertical bars in the graphics above. The bars show where I would place the systems these days. “A” stands for the smartphone app and system with performance problems. “B” stands for the bar-hopping web application that focused too early on the wrong quality goals.

System A was a system that had already made it into an app store as a product. But the complete system now had to do something for more performance. Otherwise, the system would run into decreasing demand from users because the system was too slow to use. Qualities like maintainability and security were also starting to become more necessary.

On the other hand, System B had taken care of things that were entirely out of place. For example, my co-founders and I worked on high-performance efficiency (the limit was only the credit card limit) without having users. We also implemented the possibility to change between various frontend and persistence technologies to be portable to multiple environments. The Wardley Map illustration immediately shows that we completely missed the target here. We were performing premature optimization in its purest form! Fortunately, we learned a lot during this time. We can still draw from this experience today and talk about it with very mixed feelings.

Summary

With the projection of software quality on a Wardley Map, we gain new insights into our software development activities. We have seen that during the lifetime of a software system, the quality demands can change. Software developers must not miss these points. Otherwise, the evolution of software systems will stagnate or even degrade. This situation leads to the risk that the competitors' software systems take over because they address the right qualities. However, software development teams must also understand that not every quality is necessary from the beginning. If we ignore this, there is the risk of too much additional effort that we can no longer justify with economic arguments.

In the end, keep in mind that this model simplifies an otherwise very complex matter (that’s why it’s called a model). However, mapping quality characteristics on a Wardley Map can help us better understand quality in a given situation. It also lets us think about when we need to address which qualities next in which evolutionary stage to succeed.

Overall, however, the model presented is itself still in the Genesis phase. Therefore, I would be pleased to receive questions, feedback, discussions through comments, e-mails, letters, or faxes!

Header image by Johannes Plenio from Unsplash.

Thanks to Anja Kammer for her feedback on an earlier version of this article!

Update 18 May 2022: Order of qualities adapted to feedback collected and new experiences gained as well as summarizing visualization placed at the beginning.