Module 3 — Quality Attributes and ASRs

Vocabulary

• Quality attribute — a measurable or testable property of a system used to indicate how well the system satisfies the needs of its stakeholders.
• Non-functional requirement (NFR) — defines the criteria used to evaluate the whole system rather than a specific behaviour; also called quality attributes and described in detail in architectural specifications.
• ASR (Architecturally Significant Requirement) — includes the most important requirements for architecture, whether functional or non-functional. An ASR directly impacts architecture design, whereas an NFR may or may not. That is why an ASR has more relevance than an NFR when referring to architecture requirements.

Only a subset of functional requirements, QA requirements and constraints are architecturally significant — the three sets intersect. In practice QA requirements and constraints are almost always ASRs; functional requirements more rarely. A typical functional ASR concerns integration: "integrate with a very old version of SAP."

Solid arrows mark the sets that are almost always architecturally significant; the dotted arrow marks the one that only sometimes is.

"Significant" is ultimately measured by high cost of change — monetary or not (time, resources, reputation, opportunity cost). What counts as high is invariably project-specific: a $10,000 cost can be high for a small-budget project and low for a large one. Some cost measures can be entirely qualitative yet still identifiable in their ability to distinguish ASRs.

A trade-off point is one at which no solution satisfies all involved requirements equally well, so the architect must select a design option that compromises some requirements to meet others.

The ASR characterisation framework

From Characterizing Architecturally Significant Requirements (Chen, Ali Babar, Nuseibeh — IEEE Software, March/April 2013). An empirical grounded-theory study of 90 practitioners across four countries (58% US, 28% India, 13% Netherlands, 1% UK), who had collectively worked for 500+ distinct organisations, with more than 1,448 years of accumulated software development experience and 761 years in software architecture. 52% were architects, 20% executives, 14% managers, 8% consultants, 6% developers. Only concepts mentioned by at least two participants made the final findings.

The motivation matters: if ASRs are wrong, incomplete, inaccurate or lack detail, then an architecture based on them is also likely to contain errors. In practice stakeholders and requirements engineers frequently fail to express or effectively communicate ASRs to architects, preventing informed design decisions. This work sits within the twin peaks model, where problem and solution development interplay iteratively.

The framework has four sets of characteristics.

1. Definition

ASRs are those requirements that have a measurable impact on a software system's architecture. This delimits the portion of requirements that affects architecture in measurably identifiable ways. Empirically the distinction is real — participants did not perceive "temperature should be displayed in Celsius not Fahrenheit on this webpage" as architecturally significant, but did usually perceive "the system should provide five nines (99.999%) availability" as such.

"Significant" is measured by high cost of change — monetary or not. What counts as high is invariably project-specific.

2. Descriptions — how ASRs behave in the wild

The consequence the authors draw: a definitive judgement usually can only be made when the requirement really incurs a high cost, or when the architect requires it to make architectural decisions. During requirements gathering we can only say certain requirements are likely to be architecturally significant. "Finding a definitive list of ASRs is not feasible. ASRs need to be dealt with individually."

3. Indicators — pragmatic hints without full cost estimation

Although cost of change is the measure of significance, getting an accurate cost is challenging, and cost estimation usually happens after requirements are gathered. Indicators give pragmatic hints instead.

4. Heuristics — categories familiar to requirements engineers

Because indicators like difficult to achieve demand solution-space knowledge that requirements engineers typically lack, the framework offers heuristics: familiar categories where ASRs concentrate.

A closely related SEI source worth reading alongside it: Clements and Bass, Relating Business Goals to Architecturally Significant Requirements for Software Systems, CMU/SEI-2010-TN-018 — the origin of the eleven business-goal categories documented later in this module.

Quality attribute master list

Definitions worth memorising:

The term "non-functional requirements" is hard to trace, and it is probably not necessary to hunt for the first source. Many fundamental sources mention quality attributes as producing great influence on architecture. RUP was one of the first Agile/iterative methodologies widely adopted in enterprise software development. There is a controversial opinion expressed throughout the SEI series that only non-functional requirements define architecture. Even within SEI books there are opposing opinions on the value and meaning of "functional" and "non-functional" — Bass cautions against careless use of "functional", Clements against "non-functional".

Different sources categorise differently: Microsoft's guidance and the ISO/IEC 25010 standard produce categories, while the SEI approach does not group them. ISO/IEC 25010 gives the most verbose structure: it organises Software Product Quality into eight top-level characteristics, each broken into sub-characteristics — Functional Suitability (completeness, correctness, appropriateness), Performance Efficiency (time behaviour, resource utilisation, capacity), Compatibility (co-existence, interoperability), Usability, Reliability (maturity, availability, fault tolerance, recoverability), Security (confidentiality, integrity, non-repudiation, authenticity, accountability), Maintainability (modularity, reusability, analysability, modifiability, testability) and Portability (adaptability, installability, replaceability).

Do not rely too much on standard lists of quality attributes with the purpose of producing proper architecture. Those lists are a good starting point for having proper conversations with the client, and serve the purpose of not overlooking some important aspect of system design. Under certain conditions you can come up with a non-standard quality attribute that best describes some aspect of your requirements — something like Understandability may describe the steepness of the learning curve associated with a system's design. The SEI book even has a chapter with a methodology for such situations.

Quality attributes do not exist in a silo — they are connected and influence each other, and there are relations between quality attributes, functional requirements and constraints. Security may conflict with Usability; Performance may affect Maintainability. But there are also cases where achieving one makes another easier — Conceptual Integrity may help achieve better Maintainability. Consider these relations when designing, usually by accepting trade-offs.

The six-part general scenario

The SEI template that makes any quality attribute requirement testable:

Example — modifiability: Source: developer. Stimulus: wishes to change the UI. Environment: at design time. Artifact: code. Response: modification is made, no side effects. Response measure: in 3 hours.

QA requirements can be tracked as acceptance criteria. Options for elaborating them: user stories; in SAFe, NFRs modelled as backlog constraints; as quality requirements or system quality tests; as technical debt; split into the Definition of Done; acceptance criteria of a user story; constraints in a user story.

Per-attribute scenario tables

Availability — a measure of the impact of failures and faults. Mean time to failure, mean time to repair, downtime. The probability the system is operational when needed, excluding scheduled downtime:

$$\alpha = \frac{\text{MTTF}}{\text{MTTF} + \text{MTTR}}$$

Concrete (a crossing gate controller): main processor fails to receive an acknowledgement from the gate processor. Source: external to system. Stimulus: timing. Artifact: communication channel. Environment: normal operation. Response: log failure and notify operator via alarm. Measure: no downtime.

Interoperability — the degree to which two or more systems can usefully exchange meaningful information in a particular context. Exchanging data is syntactic interoperability; interpreting exchanged data is semantic interoperability. Purposes: to provide a service, and to integrate existing systems into a system of systems (SoS). The service may need to be discovered at runtime or earlier.

Concrete: our vehicle information system sends our current location to the traffic monitoring system, which combines it with other information, overlays it on a Google Map and broadcasts it. Source: vehicle information system. Stimulus: current location sent. Artifact: traffic monitoring system. Environment: systems known prior to runtime. Response: traffic monitor combines, overlays and broadcasts. Response measure: our information included correctly 99.9% of the time.

Performance — event arrival patterns are periodic (fixed frequency), stochastic (probability distribution) or sporadic (random). Event servicing concerns latency (time between arrival of the stimulus and the system's response), jitter (variation in latency), throughput (number of transactions per second), and events and data not processed.

Concrete: main processor commands the gate to lower when a train approaches. Source: external — arriving train. Stimulus: sporadic. Artifact: system. Environment: normal mode. Response: remain in normal mode. Measure: send signal to lower gate within 1 millisecond.

Security — six properties:

Concrete: hackers are prevented from disabling the system. Source: unauthorized user. Stimulus: tries to disable system. Artifact: system service. Environment: online. Response: blocks access. Measure: service is available within 1 minute.

Modifiability — three questions: What can change? When is it changed? Who changes it?

Concrete (a restaurant locator app): the user may change the behaviour of the system. Source: end user. Stimulus: wishes to change the locale of search. Artifact: list of available country locales. Environment: runtime. Response: the user finds an option to download a new locale database; the system downloads and installs it successfully. Measure: download and installation occur automatically.

Module interdependencies that drive modifiability: data types; interface signatures, semantics, control sequence; runtime location, existence, quality of service, resource utilization.

Testability — the ease with which software can be made to demonstrate faults through testing. Assuming the software has one fault, the probability of fault discovery on the next test execution. You need to control components' internal state and inputs, and observe components' output to detect failures. Testing activities can consume up to 40% of a project.

Concrete (a photo editor): new versions can be completely tested relatively quickly. Source: system tester. Stimulus: integration completed. Artifact: whole system. Environment: development time. Response: all functionality can be controlled and observed. Measure: entire regression suite completed in under 24 hours.

Testability tactics split into control and observe system state — specialized interfaces, record/playback, localize state storage, abstract data sources, sandbox, executable assertions — and limit complexity — limit structural complexity, limit non-determinism.

Usability — ease of learning system features (learnability), ease of remembering (memorability), using a system efficiently, minimizing the impact of errors (understandability), and increasing confidence and satisfaction.

Concrete (restaurant locator): the user may undo actions easily. Source: end user. Stimulus: minimize impact of errors. Artifact: system. Environment: runtime. Response: wishes to undo a filter. Measure: previous state restored within one second.

Usability tactics split into support user initiative — cancel, undo, pause/resume, aggregate — and support system initiative — maintain task model, maintain user model, maintain system model. Both aim at the user being given appropriate feedback and assistance.

Other architecturally significant usability scenarios worth recognising, each with real architectural implications: aggregating data; cancelling commands; using applications concurrently; maintaining device independence; recovering from failure; reusing information; supporting international use; navigating within a single view; working at the user's pace; predicting task duration; comprehensive search support.

Design decisions and tactics

A system design is a collection of design decisions. Some respond to quality attributes, some to achieving functionality. A tactic is a design decision to achieve a quality attribute response, and tactics are a building block of architecture patterns — a more primitive, granular, proven design technique that sits between stimulus and response, controlling the response.

Tactics are atoms, patterns are molecules. The focus of a tactic is on a single quality attribute response; within a tactic there is no consideration of trade-offs. Trade-offs must be explicitly considered and controlled by the designer. In this respect tactics differ from architectural patterns, where trade-offs are built into the pattern. From a practical standpoint SEI tactics give only a very high-level overview of possible approaches — like quality attribute checklists, they help start a proper conversation with clients and help avoid overlooking something.

Seven categories of design decision:

1. Allocation of responsibilities — system functions to modules
2. Coordination model — module interaction
3. Data model — operations, properties, organization
4. Resource management — use of shared resources
5. Architecture element mapping — logical to physical entities: threads, processes, processors
6. Binding time decisions — variation of the lifecycle point of module "connection"
7. Technology choices

Design checklists give design considerations for each QA, organised by design decision category. For allocation of system responsibilities under performance: which responsibilities will involve heavy loading or time-critical response? What are the processing requirements, and will there be bottlenecks? How will threads of control be handled across process and processor boundaries? What are the responsibilities for managing shared resources?

The utility tree

Captures all QA requirements (ASRs) in one place. "Utility" expresses the overall "goodness" of the system.

Construction rules:

• The most important QA goals are the high-level nodes — typically performance, modifiability, security and availability
• Scenarios are the leaves
• Output: a characterization and prioritization of specific quality attribute requirements
• Two ratings per scenario: High/Medium/Low importance for the success of the system, and High/Medium/Low difficulty to achieve (the architect's assessment)

Key: H = high (must-have), M = medium (important), L = low (nice-to-have).

The canonical SEI example (the Nightingale system):

Sometimes a simple table rather than a tree is better for a utility tree — it is more readable and maintainable. A tree is good during whiteboarding. Feel free to use either.

Careful with attribution: a DDoS attack is not an ASR — preventing a DDoS attack is. Availability, Performance and Usability are all affected by a DDoS attack, but prevention belongs under Security.

Quality Attribute Workshops

The eight QAW steps:

1. QAW overview and introductions — obtain the list of attendees, take notes as appropriate
2. Business/mission presentation — capture driving quality attributes, issues, notes
3. Architectural plan presentation — capture driving quality attributes, issues, notes
4. Identification of architectural drivers — share the information from steps 2 and 3, then after a few minutes ask for clarifications and corrections to the list of architectural drivers. That list helps facilitators ensure coverage during scenario brainstorming
5. Scenario brainstorming — elicit raw scenarios from the stakeholder community in round-robin fashion, using a raw scenario table
6. Scenario consolidation — merge similar and duplicate scenarios using stakeholders' input
7. Scenario prioritization — each stakeholder gets votes equal to 30% of the total number of scenarios generated
8. Scenario refinement — fully develop the scenario to include details such as how long, how much, how often, when, environment, who

The typical ASR process: Collect → Categorize → Review/Refine → Prioritize. For each "capture" step there can be one or more "refine" steps.

Deriving ASRs from business goals

Eleven categories of business goal. They are not completely orthogonal — some goals fit more than one category, and that is all right.

Non-architectural solutions are legitimate outcomes: "to optimize the budget, decrease the salary of the employees"; "to allow a user-friendly way to edit rich text, buy a licence for Microsoft Word."

This really describes the essence of the difference between a software architect and a solution architect. A solution architect addresses business goals, which likely — but not necessarily — includes building software elements.

Maintainability / Modifiability in depth

Maintainability is one of the key quality attributes present in almost any categorisation from any source. SEI gives a slightly different name to the attribute with the same meaning — Modifiability. ISO/IEC 25010 differentiates Maintainability and Modifiability but essentially the meaning is the same: the ease of applying a change to a product or system.

In practice we frequently deal not with building solutions from the ground up but with adding features to existing solutions, re-platforming, updating technology stacks and similar. Code is read far more frequently than it is written.

It is usually said that only one requirement is always stable — the inevitability of change.

Changes arise constantly: requirements change frequently even before the system reaches its intended users, and as the system is used the need for additional features arises and bugs are revealed. Most of the cost of a typical software system occurs after it has been initially released.

Three important ramifications of the SEI cost-of-modifiability formula:

1. The cost of making a system more modifiable may vary substantially — from development cost where a developer applies changes through code, to visual configuration where end-users apply changes and see results immediately (CMS systems)
2. The cost of making a system adaptable to changes should imply the potential costs of applying those changes
3. As systems evolve, other configuration mechanisms get added — the later you add those mechanisms, the higher the cost

Binding is the concept SEI uses heavily to elaborate on architecture flexibility. Binding time decisions heavily influence maintainability: the more choices to bind a value to a specific parameter exist, the more flexible the system. The value can be anything depending on scale — from reading a single value from configuration, to replacing a component during runtime via DI or service location. Late binding is usually considered more flexible than early binding, but that comes at the cost of implementing specific infrastructure for late binding.

Tools do not usually provide much help in increasing architecture maintainability. Static code analyzers such as NDepend help get an initial feeling for a system's complexity, but when it comes to real re-factoring or re-architecting such tools rarely provide thorough guidance. What does help: unified coding standards verified by linters; conducting architecture and code reviews; keeping an up-to-date backlog of technical, design and architecture debt.

Do not underestimate the value of conformity of architecture and code — practice tells that the maintainability of a solution where the same approaches are used everywhere is dramatically higher.

Note cohesion and coupling — very ubiquitous principles applicable at almost all levels.

Performance in depth

Performance is the quality attribute articulated explicitly in almost all requirements documents. If it is not articulated, the rule of thumb is to ask about it. If it is considered that there are no specific performance requirements, there still are some — always double-check with the client what their performance expectations are, because sometimes you will be surprised. The best method to check whether performance complies with expectations and the SLA is performance testing.

Key characteristics: bandwidth, latency, response time, throughput — usually articulated in the measure part of a scenario. Less common: the jitter of the response (allowable variation in latency), and the number of events not processed because the system was too busy to respond.

SEI gives two general tactic groups to achieve performance requirements: control resource demand and manage resources. The most ubiquitous tactics: Prioritize Events, Reduce Overhead, Increase Resource Efficiency, Increase Resources, Introduce Concurrency, Maintain Multiple Copies of Computations, Maintain Multiple Copies of Data, Schedule Resources. Some are highly specific — Manage Sampling Rate applies only to streaming data processing — while others such as Increase Resource Efficiency are generic.

Types of performance testing

Others exist — configuration testing, isolation testing. It is not required to remember every type, but it is important to understand the approaches and tools that measure performance in different circumstances.

A very good piece of advice: analyse performance on production data and in production environments. A very common mistake is evaluating system performance against data that is not similar to production data. Keep the performance testing environment very close to production, otherwise you run a high risk that results will not reflect the real state of things.

Analyse the error log and any other tracing captured during a test. When there is an error, take a memory dump and analyse it. Performance optimization is hard — always measure and use profilers such as Dynatrace or dotTrace.

Fowler's story (from the first edition of Refactoring): he and a colleague were invited to analyse performance problems in an enterprise-grade application. While they travelled, the development team held several brainstorming meetings on which aspects they might improve, and came up with a list of good ideas. When Fowler arrived he did not want to look at the list — the developers were offended — and instead ran a profiler against the code. The results showed the major bottleneck was the method for working with string objects. Once resolved, they achieved such a significant boost that no other improvements were necessary. Working with string objects had not been identified as a problem in the developers' original list. Lesson: when it comes to performance, always measure, don't guess.

Be careful analysing stakeholders' requirements: performance is usually a requirement even when not mentioned explicitly — however, high performance is usually very expensive to achieve.

Scalability

There are many definitions, and they all revolve around the system's ability to handle increased load while maintaining the expected SLA. Scalability might be considered a kind of Modifiability/Maintainability, but it became so popular with the advent of cloud computing and microservices that it is usually treated as a standalone quality attribute.

Two basic kinds: vertical (scaling up) and horizontal (scaling out). Horizontal scalability always implies the system is distributed; vertical scalability applies to both standalone monolithic and distributed architectures.

The scale cube

Selected scalability rules (Abbott, Scalability Rules)

Mostly applicable to web information systems.

Other high-level influences on scalability: whether the system leverages on-premise or cloud infrastructure (hybrid approaches and private clouds exist, and hybrid is becoming more common for medium and large systems); stateful or stateless design, where stateless components generally scale better but at the cost of storing state externally and making extra calls to get and save state data. Scalability testing is challenging and tightly related to performance testing.

Listen to your clients but always try to differentiate the real needs of the business from the wishes of very specific stakeholders. Ultimate scalability is very expensive to achieve — be mindful whether it is what is really needed.

Reliability, faults, errors, failures

Reliability is tightly coupled with fault tolerance, and under some conditions can be treated as a synonym — though not in all cases. Some sources don't define reliability as standalone and unite it with availability: "in fact, availability builds upon the concept of reliability by adding the notion of recovery — that is, when a system breaks, it repairs itself."

The precise chain (Hanmer, Patterns for Fault Tolerant Software):

Common errors:

• Timing or race conditions — communicating processes get out of synchronization and a race for resources occurs
• Infinite loops — continuous execution of a tight loop without pausing and without acknowledging others' requests for shared resources
• Protocol errors — errors in the messaging stream from non-conformance with the protocol: unexpected messages, messages sent at inappropriate times, or out of sequence
• Data inconsistency — data differs between two locations, e.g. memory and disk, or between different network elements
• Failure to handle overload conditions — the system is unable to handle the workload
• Wild transfer or wild write — data written to an incorrect memory location, or a transfer to an incorrect location, if there is a fault

These reliability metrics compose along a single failure cycle rather than standing alone: MTBF (mean time between failures) spans the whole interval from one failure to the next, and decomposes into MTTF (the correct-behaviour time before a failure), MTTD (mean time to diagnose — the time spent detecting and localising the cause once a failure has occurred) and MTTR (mean time to repair, from diagnosis to restored correct behaviour). The availability formula above uses MTTR to cover diagnose-plus-repair unless MTTD is tracked and budgeted separately, which matters when triaging where an incident's time actually went.

There is a special mindset for developing fault-tolerant systems, and key principles — some applicable to all systems, others only under certain conditions. Running experiments directly in production implies substantial maturity in monitoring, recovery and resiliency; it is not recommended unless you satisfy all the prerequisites and understand the potential impact of the worst case.

Netflix is the industry's good example of a highly reliable fault-tolerant system, running tests directly in production to evaluate and measure the impact of removing components or injecting errors into the execution flow. Some elements remove components running in production; others search for unused resources and non-compliance and apply fixes; yet others search for security vulnerabilities. These approaches proved efficient for achieving high reliability, but again are not recommended without mature operations and monitoring practices and a robust environment and architecture supporting resiliency and self-healing.

That mindset starts from a single core question — what can go wrong in any given situation? — and works outward from it: eliminate potential single points of failure, make deliberate trade-offs rather than accidental ones, keep the design simple (KISS), and follow established design and coding practices. Where a computation is critical enough to justify the cost, N-version programming — running independently developed implementations of the same function in parallel and comparing their outputs — is the tactic of last resort for masking a design fault that redundancy alone cannot catch, since identical replicas would simply repeat the same bug.

Availability mathematics

Almost any software SLA tries to provide availability claims. Scheduled downtimes may not be considered when calculating availability, because the system is deemed "not needed" then — but of course this depends on the specific requirements, often encoded in the SLA.

The conclusion from real IT industry examples is that achieving more than 99% availability may be much more challenging than it seems at first glance. When someone requests more than three nines availability, be extra careful — if that is a real requirement, it will be very expensive to achieve.

Composition rules (the practical formulas from the Module 9 homework guide, which are the same maths you need here):

Series — components that are single points of failure — multiply:

$$A_{total} = A_1 \times A_2 \times A_3 \times \dots \times A_n$$

Redundancy of m identical copies of a component — one minus the probability that all copies are unavailable:

$$A_{component} = 1 - (1 - A_1)^m$$

A component's total fair availability includes both the infrastructure it runs on (hardware + OS + provided software) and the software itself, because each can fail:

$$A_{component} = A_i \times A_s$$

Both formulas assume the basic scenario: copies are fully identical with the same availability and work constantly (no partial operation or unequal availability), and one software component runs per one infrastructure component — virtualisation with VMs and containers makes the calculation more complex.

Cloud specifics. Often you use existing IaaS/PaaS/SaaS services, so you do not calculate availability from scratch but rely on the numbers published by the provider — use the service SLA to estimate total availability. What adds real complexity is that each cloud can use its own definitions (for example, what "unavailability" means) and can have many exclusions, so always read what is written at the end of the agreement.

The tricky example: Amazon's SLA for EC2 uses a region-based approach and provides a fixed 99.99% only if you run EC2 in 2 or more Availability Zones — and this number does not increase if you use 3 or more zones. As a result, 99.99% is the maximum for one region. If you want better, you need to plan across different regions.

The formulas apply to both software and hardware components. SEI availability tactics include the obvious and popular ones: Monitor, Heartbeat, Redundancy, Exception Handling, Retry.

Security in depth

Threat modelling is the technique and toolset that helps understand potential security challenges and concerns — for example investigating how secure a web service and its environment are. You usually need a security expert involved in security design and testing if there are very specific and detailed security requirements; a security competence centre can perform threat modelling and other consulting as a service.

Concretely, threat modelling runs as five steps: identify the elements (decomposing the system into its components and modules), identify the threats against each, document them, rate them, and determine the countermeasures and mitigations worth applying. The deliverable is typically a data-flow diagram annotated with trust boundaries, paired with a rated list of threats and mitigations — that pairing, not the STRIDE letters alone, is what a completed threat model looks like.

STRIDE is a threat classification model developed by Microsoft for reasoning about security threats. It was initially created as part of the threat modelling process and is used in conjunction with a model of the target system constructed in parallel, including a full breakdown of processes, data stores, data flows and trust boundaries.

SEI organises its own security tactics catalogue around four response categories that parallel how Performance and Availability tactics are grouped: Detect Attacks (detect intrusion, detect service denial, verify message integrity, detect message delay), Resist Attacks (identify/authenticate/authorize actors, limit access and exposure, encrypt data, separate entities, change default settings), React to Attacks (revoke access, lock out the computer, inform actors, maintain an audit trail) and Recover from Attacks (restore state, maintain availability). Framing security tactics as a detect/resist/react/recover cycle is what lets an architect check coverage the same way they would for a fault-tolerance or performance design.

OWASP — an industry initiative gathering knowledge about the most common security threats and methods for preventing them. The OWASP Top Ten is a powerful awareness document representing a broad consensus about the most critical web application security flaws, produced by security experts worldwide and published annually. OWASP also provides a free tool for web application security testing that can be injected into a CI/CD pipeline to enable continuous security verification (DevSecOps).

Always understand your security testing methodology and strategy — other types such as penetration testing can be leveraged. Everything is driven by the requirements and the type of solution you are building.

Other quality attributes

You will find information on many other quality attributes, and that should not be a surprise: there is no single one-size-fits-all list. Refer to ISO/IEC 25010 section 4.2 for a substantial list.

Remember: knowing quality attribute lists, tactics and patterns is a very good starting point for having proper conversations with clients. For creating good architectures you have to analyse the broader picture — other ASRs, business needs, industry reference architectures and similar.

Common pitfalls in ASR gathering

Making quality attributes measurable

A quality attribute you cannot measure is not a requirement, it is a wish. The table below is a compiled metric vocabulary per attribute — it is what "define a set of measurable metrics" actually means. The guidance: select the 3–5 most important quality attributes, give the motivation for selecting each, list them for both baseline and target architecture, and note the component where each metric is measured.

Quality attributes under a service-oriented architecture

Quality Attributes and Service-Oriented Architectures (SEI, CMU/SEI-2005-TN-014) exists because software architecture is the bridge between mission/business goals and a software-intensive system, and quality attribute requirements drive architecture design — so it matters how a chosen style supports them.

The report's status column rates the maturity of SOA in each area: green = known solutions on relatively mature standards and technology; yellow = some solutions exist but need further research to prove usefulness; red = standards and technology immature, significant further effort required.

The report's own caveat, which is the real lesson: as with any architecture, trade-offs between quality attribute requirements must be made, and the resulting decisions may impact the organisation's ability to meet its business goals. In each system the quality attributes must be characterised specifically — using scenarios — and then weighed against this information.

Worked example — ASRs for Lumen Diagnostics

The Module 3 exercise done against the case study. The Priority column carries two values: business value and architectural impact, in that order.

Functional ASRs

Most functional requirements are not architecturally significant. These are, and note how many of them are about integration — that is the usual pattern.

Quality attribute ASRs

Constraints

The utility tree that follows

Notice the problem this tree exposes: five scenarios came out (H, H). That is not a result, it is a symptom — it means the prioritisation has not really been done. Before choosing tactics, go back to the stakeholders and force the ranking. Here, taking Compliance, Scalability and Performance as the top three is a defensible reading, because data residency is a legal precondition, capacity is a growth precondition, and response time is what every user experiences on every interaction.

Tactics for the top three

How each is verified at a release checkpoint

• Compliance — automated test asserting that a request carrying a region-A patient identifier cannot read or write region-B storage; plus an infrastructure audit of store locations
• Scalability — load test to 4,200 concurrent sessions plus 12%, observing that autoscaling engages and p95 holds; confirm scale-in also works, since only scaling out is usually tested
• Performance — synthetic transactions per screen class reporting p95 continuously, with the per-component budget instrumented so a regression identifies which component consumed its share

How ASR extraction goes wrong

These are the failure modes worth rehearsing, because they recur on nearly every engagement.

Copying instead of reformulating. A requirement lifted verbatim from the customer's document inherits its vagueness. If you cannot state the response measure, you have not finished writing the ASR.

Missing the architecturally significant functional requirements. Teams scan for quality attributes and skip the functional list entirely — but integration requirements hide there, and they are the ones that reshape the architecture. "One partner is automated, six hundred are manual" is a functional requirement with more architectural consequence than most of the quality attributes.

Discarding a generic phrase instead of clarifying it. "Highly configurable to support growth" is not usable as written — which makes it a requirement that needs a conversation, not one to drop. Dropping it is how you discover at UAT that configuration was expected to be self-service.

Confusing a testable requirement with a constraint. "One coherent look and feel across every portal" is a requirement you verify; it is not a constraint that removes design freedom. Filing it as a constraint hides it from the test plan.

Confusing interoperability with portability. Interoperability is about the quality of an integration — protocol, format, latency, error semantics. Portability is about running in a different environment. An integration is usually both a functional requirement (we must integrate) and a quality attribute requirement (within 15 minutes, over TLS, with defined failure behaviour).

Offering process measures as architectural tactics. "Ensure the team includes experienced operations engineers" may be sound advice and is not an architectural tactic. A tactic is a design decision about the system. Similarly, "it will be hosted by the supplier" is a constraint that creates quality attribute problems; it does not solve any.

Not separating assumptions from stated requirements. Where you assume a figure because none was given, mark it visibly. Otherwise your assumption is indistinguishable from the customer's commitment, and nobody ever revisits it.

Accepting an availability figure without pricing it. Nines are bought with redundancy, and each nine costs multiples of the last. If you cannot show the cost of an outage hour, you cannot tell whether the number you have been handed is too strict or too lax — which is why Module 6 derives the figure from the business arithmetic instead.

Exercises — Module 3

Task. Extract ASRs from Lumen Diagnostics and turn the quality-attribute ones into a utility tree plus tactics. Output: ASR list (functional / QA / constraints), utility tree, top-3 tactics with how you apply them to Lumen, and how you will check them at a release checkpoint.

ASRs

• Read the Lumen Diagnostics case
• Read the ASR characterisation article in the materials first
• List only ASRs, in three sections: Functional, Quality Attributes, Constraints
• Reformulate every ambiguous requirement; do not copy the customer's wording
• Prioritise each ASR on business value and architectural impact
• Write a short rationale for why each one is architecturally significant

Utility tree

• Build a utility tree of QA requirements (SEI drawing, or a table)

  • QA only (constraints optional); no functional ASRs
  • H / M / L from both the business and architectural views
• Mark assumptions visually, distinct from stated requirements

  • Example: you set availability to 99.9% because the brief gave no figure

Tactics

• Learn tactics from Software Architecture in Practice (and any other source you cite)
• From H,H / H,M / M,H scenarios, pick the top-3 quality attributes
• Choose tactics and show how they apply to Lumen, not a random SEI list
• Describe how you will check those attributes at a release checkpoint