The next post in this mini-series discussing overloaded terms: This time, I’ll cover evaluation, a term that’s often used in the context of systems or architecture reviews.
The Origin
The dictionary.com definition states:
evaluate: verb (used with object), evaluated, evaluating.
- to determine or set the value or amount of; appraise: to evaluate property.
- to judge or determine the significance, worth, or quality of; assess: to evaluate the results of an experiment.
- Mathematics. to determine or calculate the numerical value of (a formula, function, relation, etc.).
Of French origin, the word evaluate, contains the term value as its primary constituent. Value refers to a number or numerical value.
Evaluate to Determine Value
The (numerical) value is something that e.g. mathematicians and accountants need to determine. If we think about numbers in software architecture, we care about:
- timing, throughput, numbers-of-concurrent-users or similar runtime metrics
- source code metrics, like size, coupling, test coverage
- effort estimation
However, when we talk about these metrics, we don’t speak of “evaluation” but of measuring or estimation.
Evaluate to Find Problems
In his book Evaluating Software Architectures, the author Paul Clements uses the term evaluate to identify potential (quality) issues or problems in software architectures. He presents a few methodical approaches to finding such problems, primarily the ATAM method: architecture tradeoff analysis method.
Yes, you read that correctly: an analysis method in a book about evaluation.
I always wondered why Paul Clements didn’t call his book “Analyzing Software Architectures”.
Let’s conclude: When speaking of evaluating, what software architects actually mean is analyzing.
Conclusion: Did you mean “Analyze”?
I propose that we (as software developers and architects) better distinguish between analysis and evaluation. In the aim42 method for improving software architectures, these distinction has been made very prominent:
- The analyze phase of aim42 identifies problems and risks
- the evaluate phase determines how bad/severe these problems are in business terms. The outcome of this evaluation are numbers (e.g. cost).
Side note: Numbers and Nanoseconds
The famous and awesome inventor of the compiler, Grace Hopper, gave a talk about numbers and nanoseconds – every developer should see and understand this talk :-).
Proposal: Focus on Analysis
Analyze (or assess) your system and situation to identify problems, risks, symptoms and root-causes. Only later, you may evaluate these issues with respect to the amount of damage, pain or havoc they can cause.
Thank you
Thanks for reading – and may simplicity and effectiveness be with you and your architectural decisions!
Sources and Further References
Software Architecture Glossary by members of the iSAQB association. Extensive (and free!) collection of software architecture terminology, oriented towards the various iSAQB curricula - available as an eBook and online.
Evaluating Software Architectures: a (often quite theoretical) and very early book on identifying problems in software architectures. The primary source for the well-known ATAM method.