Dieser Artikel ist auch auf Deutsch verfügbar

In various communities, several methods for the collaborative modeling of business requirements have been established in recent years. Well-known examples are EventStorming or Domain Storytelling. These approaches are based on achieving a better shared understanding of the business requirements in an interdisciplinary way. But what about the requirements for the quality of the software being developed? Especially here, a collaborative approach is immensely important in order to avoid chasing after imperfect ideals that cause the costs and complexity of products to explode. This is where the workshop format Quality Storming comes in, which I would like to introduce in the course of this article.

It depends! On what?

In our work as consultants, we repeatedly experience situations in which teams passionately discuss the advantages and disadvantages of specific solution options. Popular in these discussions are technical topics, such as HTTP Feeds vs. Apache Kafka. However, such arguments are also becoming more and more common when it comes to slice the software functionally. A popular answer to the question “which solution is the best” is then often “it depends”. This answer indicates that we are dealing with a decision that implies a trade-off. The second question, “on what?” is therefore automatically asked. Possible factors may involve:

The first three drivers for such decisions are taken more or less seriously in most organizations. With the last point, however, I regularly find that most teams lack resilient requirements. Most of the time, these are so vaguely defined that they cannot be used as a basis for decision-making. Typical examples are “the system must be scalable”, “all common browsers must be supported” or “all input must be immediately visible everywhere at all times”. It goes without saying that over the course of time, best practices have established themselves that make suggestions as to how the requirements for the quality of a software product should ideally be documented. An example of this is the formulation in the form of quality scenarios, which are also used in the ATAM process (Architecture Tradeoff Analysis Method) [1] for the evaluation of software architectures. There are two steps in the ATAM process that address the definition of quality requirements for a software architecture: “5. Generate quality attribute utility tree” and “7. Brainstorm and prioritize scenarios”. This raises the question of how we, as a team, can get to these requirements. At this point, there are always similar challenges in many organizations:

As a result, the quality requirements are too superficial and are primarily used to fulfill internal governance checklists. Software architects and developers can rarely make real design decisions on this basis.

Therefore, the following quote from Alberto Brandolini is also fully correct with regard to quality requirements:

it is not the domain expert’s knowledge that goes into production, it is the developer’s assumption of that knowledge that goes into production

Alberto BrandoliniInventor EventStorming

Quality Storming: Collaborative Modeling for quality requirements

The quote just mentioned was not chosen at random. Alberto Brandolini is the inventor of a collaborative modeling method called EventStorming [2], which has caused quite a stir in the Domain-driven Design community and is now known far beyond that community. EventStorming, like most methods in the field of Collaborative Modeling, is based on the following principles:

Motivation

This is where Quality Storming comes in, trying to bring together a heterogeneous set of stakeholders of a product or project to collect quality requirements. The goal is to gain a shared understanding of the real needs for the quality characteristics of a product. To achieve this goal, Quality Storming uses some techniques from the highly acclaimed book “Game Storming” by Dave Gray [3], which also had a significant impact on EventStorming.

It is not the claim to produce perfectly formulated quality scenarios with the help of Quality Storming. Instead, the method aims to create a well-founded, prioritized basis for later formalization, which is understood across different stakeholder groups. The more often teams work with the technique, the better the quality of this basis becomes over time. Advanced teams are quite capable of creating very well-formulated scenarios within the framework of such a workshop.

Preparation

As with many other methods in the field of Collaborative Modeling, a solid preparation is an important success factor for the actual workshop. Make sure to take the following aspects into account:

The most important thing is undoubtedly the selection of the participants. We must select a heterogeneous and diverse group of people. All relevant stakeholders of a product or project should be present in order to get a holistic opinion and, above all, to achieve commitment for the results. Nothing would be more unfortunate than an influential group not participating and calling the overall result into question after one week. When selecting the circle of participants, think for example of the following groups of people:

Experience has shown that 16 or 24 people are ideal, especially when working with the widely used ISO/IEC 25010 quality model. I will leave a few comments on quality models further below.

When inviting the participants, it is vital to make sure that no false expectations are stirred up. It is important to emphasize that at the end of the workshop, there is a binding commitment to the quality requirements. The recipients of the invitation should understand that based on the results of the workshop, design decisions will be made in the areas of software development and architecture, user experience, and operations. However, the impression should not be given that a perfectly formulated document or a formally perfect quality tree is the result of a Quality Storming. The creation of such artifacts is done afterward, as needed. Make sure that the participants in the workshop are present for about 4 to 6 hours. The invited persons should plan the time exclusively for attendance. A telephone conference or a meeting in between is counterproductive and disturbs the process unnecessarily.

Already during the preparation, the organizer of a Quality Storming should think about the selection of a quality model. A quality model can be described as a rough outline for a quality tree. The latter can quickly be recorded in the form of a mind map.

In some companies, such a model for the description of software quality requirements already exists. In this case, it is an excellent starting point, and it is recommended to start with it. Fortunately, if no quality model exists yet, there is no need to come up with one of your own, because there is a standardized proposal for a quality model in the ISO/IEC 25010 standard.

ISO/IEC 25010 as a quality model
ISO/IEC 25010 as a quality model

ISO/IEC 25010 provides a total of eight main categories. As shown in Figure 2, these categories, in turn, have subcategories.

The choice of the quality model is essential because it will have an impact on the number of participants. I always prefer the following formula for determining the perfect amount of people: quantity of the quality model’s top categories x 2 or 3. In the case of ISO/IEC 25010, we would be at 16 or 24 people, both figures I mentioned earlier regarding this paragraph.

For the room selection, I strongly recommend to prefer rooms that have either no tables or movable tables. Traditional meeting rooms with wired tables are counterproductive. The room has to be large enough for the number of participants plus 1–2 facilitators as determined earlier, and should still offer enough freedom of movement for the participants when placing two flipcharts and up to eight movable pinboards. If there are not enough movable pinboards available, the room should have two free long walls on which plotter paper can be attached.

The last preliminary measure is the organization of the room equipment, the moderation material, and the rough documentation of the quality model. Ideally, the room should be equipped as follows:

The following list moderation material is useful for a good Quality Storming workshop:

Finally, I recommend the preparation of signs for the main and subcategories of the underlying quality model. I like to use DIN A4 for the top categories and DIN A5 for the subcategories. Each sign contains the name of the category in large, easily readable letters and a short description of the respective category. When describing the categories, please make sure that you use words that really every person in the room can understand. I also recommend the wording of a few simple examples for the respective categories. A comprehensive collection of such samples can be found in a subproject of arc42 on GitHub.

Shortly before the actual workshop, you need to setup the room. Each of the top categories of the quality model gets its own pinboard. Prepare this as follows:

Preparation of a pinboard for a main category of the quality model
Preparation of a pinboard for a main category of the quality model

Place the sign for the main category of the quality model at the top of the pinboard and line up the signs for subcategories vertically downwards on the left side. Especially with the subcategories it is advisable to pay attention to good descriptions. Furthermore, I recommend placing one or two examples of quality requirements or scenarios for each subcategory, especially during the first sessions of a Quality Storming workshop. The colors used in figure 3 are not to be understood as binding color codes.

Afterward, the mobile pinboards are placed evenly in the room. Please make sure that up to six people can stand and discuss on the pinboards. In addition, place two flipcharts and ideally stand-up tables with moderation material in the middle of the room. The room setup should look something like the following picture:

Preparation of the room for Quality Storming
Preparation of the room for Quality Storming

Running the actual workshop

Once the room is prepared, the workshop can begin. A Quality Storming consists of the following phases:

  1. Introduction
  2. Broad Collection
  3. Consolidation
  4. Prioritization
  5. Outlook

Phase 1: Introduction

This phase can usually be very short and should not take longer than 10 - 15 minutes. It is essential that the workshop facilitator briefly outlines the motivation and the approximate procedure. As a rule, I also briefly present the underlying quality model and provide a few practical examples of possible quality requirements. Especially teams that are conducting a Quality Storming for the first time will benefit immensely from the above-mentioned presentation and examples. As a facilitator, make sure that the participants have a rough idea of how to formulate quality requirements and which aspects have to be considered. You should also briefly present a code of conduct. In this context, I refer, for example, to desired and undesirable behavior.

As a facilitator, make sure that you have addressed the following points:

Phase 2: Broad collection

After the introduction, the first team building for the initial collection of quality requirements takes place. Depending on the number of participants and the number of main categories of the quality model, groups of 2 or 3 should be formed. Please make sure that the persons in the individual teams belong to different stakeholder groups. Avoid, for example, that two software developers join together to form a team. The teams will eventually be divided up in such a way that there is a heterogeneous group of two or three people on each pinboard.

After the team building, we start the collection of quality requirements. Each team stands at a pinboard that represents a top category of the quality model. The first step is to collect requirements for this category. The teams collect ideas, write them on the sticky notes provided and stick them on the pinboard to the respective subcategories.

Collection of requirements in phase 2
Collection of requirements in phase 2

After ten minutes, it is time for the groups of two or three to change the main category of the quality model and thus also the pinboard. After the change, the requirements are collected again for ten minutes, written on sticky notes and stuck to the pinboard. We repeat this process until it was the turn of each group. This ensures that every person present had the opportunity to make requests for each of the main categories. Furthermore, a large number of possible requirements can be collected in this way. The facilitators of the workshop should also emphasize several times that it is absolutely ok to have conflicting requirements for the quality of the software. In this early phase, we want to explicitly document different views. In order to reach a consensus, it is crucial to understand where different opinions differ and to what extent.

The following picture shows the rotation model of the second phase:

Team rotation in phase 2
Team rotation in phase 2

At the end of the second phase, all participants deserve an extensive (20 - 30 minutes) break. The workshop should have lasted about two hours now. During the break, the facilitators prepare the third phase.

Phase 3: Consolidation

The just mentioned preparation of the third phase consists of the identification of double or competing quality criteria. Exact duplicates are removed from the pinboards by the facilitators and put aside or stuck to the frame of the respective pinboard. Group competing quality criteria together. A competing criterion is understood to mean different requirements for the identical subject matter. Here is an example:

All these requirements relate to the identical facts, the required number of calculations of the mortgage lending value of a property. However, the quantity differs. All these sticky notes are grouped as follows:

Transition from phase 2 to phase 3
Transition from phase 2 to phase 3

When the participants come back from the break, new groups are formed, which are larger. Ideally, the new teams should consist of four to six people. Here, too, as with group formation in phase 2, care should be taken to create teams that are as heterogeneous as possible in terms of stakeholders. With eight pinboards for the top categories of the quality model, for example, four groups are ideal.

After the group formation, the actual consolidation of the quality requirements identified so far and grouped by the facilitators takes place. Each group of four or six persons then positions itself at a pinboard and discusses the grouped, competing requirements with the aim of finding a decision or a compromise. This can be a requirement that is already stuck to the wall or an agreement between the existing requirements. In any case, it is recommended to mark the decision made. This can be done, for example, by a new sticky note in a different colour or by marking with another, smaller Post-It.

Selection of quality requirements in phase 3
Selection of quality requirements in phase 3

Experience shows that each group needs about 15 - 20 minutes to consolidate the results. After this time, each team moves on to the next pinboard and starts consolidating from the beginning. Usually, it is sufficient if each pinboard has been visited and processed by only one consolidation group. However, heated discussions can arise during consolidation in individual groups. At this point, there are several possibilities for conflict resolution:

  1. Mark several sticky notes as potential candidates with regard to quality requirements and, at the end of phase 3, have them voted on in the entire plenum by majority vote. This is undoubtedly the fastest and most time-saving option, but there is a risk that legitimate concerns may be overlooked by individuals.
  2. The second option is to allow other groups to look at the pinboards with divergent opinions. This is a variant that takes more views into account but can prolong the workshop.
  3. The third option is primarily used when one of the other two options does not produce a result: mark the relevant area as “we need more information” and date the decision backward. Experience shows, however, that this option opens the door to political stalling tactics after some time and should, therefore, be used with caution.
  4. As an alternative to the third option, it is far better to go out with a hypothesis from the workshop and then verify it with appropriate metrics. At this point, I always like to define the metrics that need to be captured in order to make a better decision later.
Team rotation in phase 3
Team rotation in phase 3
Phase 4: Prioritization

In the preceding phases, numerous requirements for the quality of software were collected. However, when making architectural decisions, there is often the possibility that these must be weighed up between different quality requirements. Thus, an architecture decision can have a positive effect on certain quality requirements, but on the other hand, it can also have a negative effect on other quality requirements. Prioritization of requirements gives software architects and developers a more solid basis for their decisions. This prioritization is the last step in the actual quality storming.

Before prioritization is carried out, the quality requirements sorted out in the consolidation can be put to the side. In this phase, only the requirements marked in phase 3 count.

Dot voting has proven to be a suitable method for carrying out prioritization: Depending on the number of collected quality requirements, each participant in the Quality Storming workshop receives a certain amount of small sticky dots. The amount of sticky dots handed out per person should be about 15 - 25% of the consolidated quality requirements. Afterward, the workshop participants are asked to mark their most important requirements with the help of the sticky dots. As a rule, a person should only stick one dot on a sticky note marked in the consolidation. Another option would be to allow participants to stick up to two dots on requirements that are particularly important to them.

Dot-voting for prioritizing the requirements in phase 4
Dot-voting for prioritizing the requirements in phase 4

By the end of the fourth phase, we have gathered a significant number of prioritized quality criteria.

Phase 5: Outlook

The last phase, the outlook, is primarily a summary of the results obtained. It is always advisable to remind the participants that they now have a lot of requirements that have been collected and discussed across all stakeholders. Most people probably enjoyed the workshop, as well. Now the facilitators and the technical participants have to give a short outlook on how the results of the workshop will be used in the future. In addition, they should also briefly point out which tangible artifacts will be created in the follow-up work based on the workshop results and where the participants can find them.

Follow-up work

In principle, it is recommended that the results obtained be transferred to architectural documentation. Possible options are:

Summary

Quality Storming is a workshop for the identification of quality requirements based on a quality model, for example ISO 25010, using methods and ideas of Collaborative Modeling, which is popular in the domain-driven design community. Important here is a silo-spanning collaboration of different stakeholders and skillsets.

Quality Storming is based on three pivotal phases:

These phases are accompanied by a preparation / introduction and an outlook or a follow-up.

Overview Quality Storming
Overview Quality Storming

Readers of this article can purchase Michael’s book “Hands-on Domain-driven Design - by example” with a rebate coupon by clicking here

  1. https://en.wikipedia.org/wiki/Architecturetradeoffanalysis_method  ↩

  2. https://leanpub.com/introducing_eventstorming  ↩

  3. https://www.amazon.com/Gamestorming–Playbook–Innovators–Rulebreakers–Changemakers/dp/0596804172  ↩

  4. https://arc42.org  ↩

TAGS