Skip to content
Unit 5 of 18
RFPLumen DiagnosticsOpen anytime - every module Task uses this brief

Module 3 — Quality Attributes and ASRs

Contents · 36

What you'll be able to do

  • Separate architecturally significant requirements from ordinary ones, using cost of change and the indicators from Chen, Ali Babar and Nuseibeh.
  • Rewrite a vague customer requirement as a six-part quality-attribute scenario with a testable response measure.
  • Derive an availability target from the cost of an outage, state its measurement window, and split it across the components on a request path.
  • Derive peak concurrency from business volumes with Little's law, instead of guessing a concurrency ratio.
  • Build a utility tree, recognise (H, H) inflation, and force a ranking down to the top three quality attributes.
  • Choose tactics for those three and define a release-checkpoint test that could actually fail.
  • Write observability and cost-efficiency as testable scenarios — a detection time, a cost per referral — even though ISO/IEC 25010 lists neither.

Vocabulary

  • Quality attribute — "a measurable or testable property of a system that is used to indicate how well the system satisfies the needs of its stakeholders" (Bass, Clements & Kazman).
  • Non-functional requirement (NFR) — defines the criteria used to evaluate the whole system rather than a specific behaviour; also called quality attributes and described in detail in architectural specifications.
  • ASR (Architecturally Significant Requirement) — includes the most important requirements for architecture, whether functional or non-functional. An ASR directly impacts architecture design, whereas an NFR may or may not. That is why an ASR has more relevance than an NFR when referring to architecture requirements.

Only a subset of functional requirements, QA requirements and constraints are architecturally significant — the ASR set overlaps all three, drawing a subset from each. In practice QA requirements and constraints are almost always ASRs; functional requirements more rarely. A typical functional ASR concerns integration: "integrate with a very old version of SAP."

Rendering diagram…

Solid arrows mark the sets that are almost always architecturally significant; the dotted arrow marks the one that only sometimes is.

"Significant" is ultimately measured by high cost of change — monetary or not (time, resources, reputation, opportunity cost). What counts as high is invariably project-specific: a $10,000 cost can be high for a small-budget project and low for a large one. Some cost measures can be entirely qualitative yet still identifiable in their ability to distinguish ASRs.

A trade-off point is one at which no solution satisfies all involved requirements equally well, so the architect must select a design option that compromises some requirements to meet others.

The ASR characterisation framework

From Characterizing Architecturally Significant Requirements (Chen, Ali Babar, Nuseibeh — IEEE Software, March/April 2013). An empirical grounded-theory study of 90 practitioners across four countries (58% US, 28% India, 13% Netherlands, 1% UK), who had collectively worked for 500+ distinct organisations, with more than 1,448 years of accumulated software development experience and 761 years in software architecture. 52% were architects, 20% executives, 14% managers, 8% consultants, 6% developers. Only concepts mentioned by at least two participants made the final findings.

The motivation matters: if ASRs are wrong, incomplete, inaccurate or lack detail, then an architecture based on them is also likely to contain errors. In practice stakeholders and requirements engineers frequently fail to express or effectively communicate ASRs to architects, preventing informed design decisions. This work sits within the twin peaks model, where problem and solution development interplay iteratively.

The framework has four sets of characteristics.

1. Definition

ASRs are those requirements that have a measurable impact on a software system's architecture. This delimits the portion of requirements that affects architecture in measurably identifiable ways. Empirically the distinction is real — participants did not perceive "temperature should be displayed in Celsius not Fahrenheit on this webpage" as architecturally significant, but did usually perceive "the system should provide five nines (99.999%) availability" as such.

"Significant" is measured by high cost of change — monetary or not. What counts as high is invariably project-specific.

2. Descriptions — how ASRs behave in the wild

DescriptionWhat it means in practice
Hard to define and articulate"Users usually find it difficult to articulate [ASRs], as many of them are about abstract and general concepts." And ASRs are expected to be ready early in the process, which adds to the difficulty
Tend to be described vaguelyArchitects report receiving ASRs too vague to make informed decisions. Vague ASRs lead to bad decisions because architects make wrong assumptions about the missing details. The worked example: users requested the ability to receive notification about cash flows. Architects assumed email would be acceptable. During detailed design users explained they wanted real-time notification, the ability to subscribe to different account topics, and a UI showing all of this — which required publish-subscribe, a different architecture style entirely
Tend to be neglected initially"Typically [ASRs] are overlooked in the early phase of a project" — from lack of initial awareness of their significant effects. "Users do not typically have a good understanding of [ASRs]; people who conduct requirement analysis often do not document them properly. … Users will not ask for them unless you are dealing with a highly tech-savvy group." Happens more with less-experienced teams. Many times requirements aren't recognised as architecturally significant until they've incurred a high cost — and at that stage rectifying mistakes can be costly
Tend to be hidden within other requirementsFollowing the way people spontaneously express requirements, ASRs usually aren't emphasised; they're embedded in other requirements' descriptions. Short phrases like "highly available system of 99.999 percent uptime" or "fault tolerant" are often mentioned only briefly while describing something else — but these phrases can significantly affect architecture. When architects receive ASRs hidden this way, they usually aren't sufficiently elaborated and lack the key details needed
SubjectiveASRs tend to be requested based on opinion rather than fact-based objective decisions. "[ASRs requested by customers] usually contain subjectivity — for example, 'The system should be available for 24/7' — whether it [needs to] be or not." Small differences in ASRs can lead to big differences in resulting architectures
VariableASRs change over time, usually unavoidably, owing to business and technology change. "Technology … is now changing so quickly that companies are viewing products as having a very limited (and short) lifetime before needing to be redesigned." ASRs also vary over space — consider similar systems engineered as a software product line
Situational"A requirement is architecturally significant in one case, while being 'just a requirement' in the other case." Situational with respect to an existing architecture, a project's context or scope. A requirement might be significant where a "bad" architecture is in place but not where the architecture is "good". "A simple requirement in a smaller project might not be architecturally significant. Take the same requirement and enhance scope, add multiple interactions — then it could become significant"

The consequence the authors draw: a definitive judgement usually can only be made when the requirement really incurs a high cost, or when the architect requires it to make architectural decisions. During requirements gathering we can only say certain requirements are likely to be architecturally significant. "Finding a definitive list of ASRs is not feasible. ASRs need to be dealt with individually."

3. Indicators — pragmatic hints without full cost estimation

Although cost of change is the measure of significance, getting an accurate cost is challenging, and cost estimation usually happens after requirements are gathered. Indicators give pragmatic hints instead.

IndicatorWhy it signals significance
Wide impactWhen a requirement has wide impact — in terms of components, other requirements, code modules, stakeholders — it's usually architecturally significant. "[An ASR] has widespread impact across multiple components of the system"; "The more broadly a requirement and its resolution can be applied, the more significant it is"
Targeting trade-offsA trade-off point is one where no solution satisfies all involved requirements equally well, so the architect must compromise some to meet others. Such requirements directly affect the outcome of architecture decisions. "The trade-offs are the weak points — the raw nerves — of the architecture. If a new requirement happens to (unintentionally) target these trade-offs … then the probability of it becoming architecturally significant is higher." When a requirement targets a trade-off point, the details, accuracy and precision of its description become critical
Strictness (constraining, limiting, non-negotiable)Requirements satisfiable by multiple design options — or negotiable into such a form — allow flexibility in design. A strict requirement determines architectural decisions because it can't be satisfied by alternatives. "Our significant requirements were those that would be the limiting, or defining, characteristics of the product"
Assumption breakingWhen designing, the architect makes fundamental assumptions, explicitly or implicitly — for example that turning down the server at midnight for maintenance is acceptable. Later this might no longer hold, and the requirement that breaks the assumption forces a different design
Difficult to achieveJudging whether a requirement is difficult to achieve requires substantial knowledge of the solution space — which requirements engineers usually aren't expected to have. This limitation is precisely why the fourth set exists

4. Heuristics — categories familiar to requirements engineers

Because indicators like difficult to achieve demand solution-space knowledge that requirements engineers typically lack, the framework offers heuristics: familiar categories where ASRs concentrate.

Heuristic categoryLook here
Quality attributesThe largest source. The paper's own list: Configurability, Flexibility, Interoperability, Performance, Recoverability, Scalability, Stability, Security, Portability, Reusability, Testability, Auditability, Sustainability, Supportability, Usability
Core featuresThe central functionality of the system
ConstraintsRequirements that remove design freedom
Application environmentThe context the system must operate within

A closely related SEI source worth reading alongside it: Clements and Bass, Relating Business Goals to Architecturally Significant Requirements for Software Systems, CMU/SEI-2010-TN-018 — the origin of the eleven business-goal categories in Deriving ASRs from business goals below.


Quality attribute master list

OperationalDevelopmental
AvailabilityModifiability
InteroperabilityVariability
ReliabilitySupportability
UsabilityTestability
PerformanceMaintainability
DeployabilityPortability
ScalabilityLocalizability
MonitorabilityDevelopment distributability
MobilityBuildability
Compatibility
Security
Safety
Observability
Cost-efficiency

Definitions worth memorising:

The performance, interoperability, usability and modifiability definitions below follow Bass, Clements & Kazman, Software Architecture in Practice.

AttributeDefinition
PerformanceThe system's ability to meet its timing requirements — how fast and with what throughput it responds to events (ISO/IEC 25010: time behaviour, resource utilisation, capacity)
Interoperability"The degree to which two or more systems can usefully exchange meaningful information via interfaces in a particular context"
UsabilityHow easy it is for users to accomplish a desired task, and the kind of user support the system provides. Users experience it directly, which is why it is so visible
ReliabilityThe ability to continue to operate under predefined conditions
AvailabilityPart of reliability; the ratio of available system time to total working time
SecurityThe ability to reduce the likelihood of malicious or accidental actions, and the possibility of theft or loss of information
MaintainabilityThe ability of the system to support changes
ModifiabilityThe cost and risk of making a change — how easily a given change can be made without introducing defects. In ISO/IEC 25010 it is a sub-characteristic of Maintainability
TestabilityHow well the system allows tests to be performed according to predefined criteria
ScalabilityThe ability to handle increased load by adding resources, while keeping performance within the agreed bounds
ReusabilityThe chance of using a component or system in other components/systems with small or no change
SupportabilityThe ability of the system to provide useful information for identifying and solving problems
ObservabilityHow well the system's internal state can be inferred from what it emits — metrics, logs, traces — so operators can detect, localise and explain a failure, including one nobody predicted, without shipping new code. The term comes from control theory (Kálmán, 1960)
Cost-efficiencyDelivering the required qualities at the lowest sustainable run cost, measured per business unit — cost per referral, per booking — rather than as a total that rises with success

The term "non-functional requirements" is hard to trace, and it is probably not necessary to hunt for the first source. Many fundamental sources mention quality attributes as producing great influence on architecture. RUP was one of the first iterative methodologies widely adopted in enterprise software development. There is a controversial opinion expressed throughout the SEI series that only non-functional requirements define architecture. Even within SEI books there are opposing opinions on the value and meaning of "functional" and "non-functional" — Bass cautions against careless use of "functional", Clements against "non-functional".

Different sources categorise differently: Microsoft's guidance and the ISO/IEC 25010 standard produce categories, while the SEI approach does not group them. ISO/IEC 25010 gives the most verbose structure: its 2011 edition (ISO/IEC 25010:2011) organises Software Product Quality into eight top-level characteristics, each broken into sub-characteristics — Functional Suitability (completeness, correctness, appropriateness), Performance Efficiency (time behaviour, resource utilisation, capacity), Compatibility (co-existence, interoperability), Usability, Reliability (maturity, availability, fault tolerance, recoverability), Security (confidentiality, integrity, non-repudiation, authenticity, accountability), Maintainability (modularity, reusability, analysability, modifiability, testability) and Portability (adaptability, installability, replaceability).

The current edition, ISO/IEC 25010:2023, has nine characteristics: Safety is added, Usability is renamed Interaction capability, and Portability becomes Flexibility, which now includes scalability. Smaller changes include maturity becoming faultlessness and Security gaining resistance. The quality-in-use model moved out to ISO/IEC 25019:2023. Check which edition a customer or regulator cites before mapping requirements to it.

Do not rely too much on standard lists of quality attributes with the purpose of producing proper architecture. Those lists are a good starting point for having proper conversations with the client, and serve the purpose of not overlooking some important aspect of system design. Under certain conditions you can come up with a non-standard quality attribute that best describes some aspect of your requirements — something like Understandability may describe the steepness of the learning curve associated with a system's design. The SEI book even has a chapter with a methodology for such situations.

In regulated domains the most common non-standard attribute is privacy and compliance. ISO/IEC 25010 files confidentiality and accountability under Security, but when a regulator rather than a stakeholder sets the response measure, a separate node in the utility tree keeps those requirements visible. The Lumen worked answer uses one.

Two more non-standard attributes appear in almost every cloud engagement: observability and cost-efficiency. Neither is a top-level characteristic in either edition of ISO/IEC 25010. Their nearest homes are analysability under Maintainability — how readily the cause of a failure can be diagnosed — and resource utilisation under Performance efficiency, and neither gives you a response measure such as "paged within 5 minutes" or "$0.06 per referral". Practitioners treat both as quality attributes in their own right:

  • The AWS Well-Architected Framework and Microsoft's Azure Well-Architected Framework both give cost its own pillar, Cost Optimization, and put observability under Operational Excellence
  • Werner Vogels' The Frugal Architect (2023) opens with the law "make cost a non-functional requirement"
  • Majors, Fong-Jones & Miranda (Observability Engineering, O'Reilly, 2022) separate observability from monitoring: monitoring checks for the failures you predicted, while observability lets you ask a new question of the running system. The Monitorability in the master list is the weaker of the two

Both matter most when the supplier also runs the system, as Lumen's C01 requires. The worked answer adds them as QA17 and QA18; part 2 covers them in depth.

Quality attributes do not exist in a silo — they are connected and influence each other, and there are relations between quality attributes, functional requirements and constraints. Security may conflict with Usability; Performance may affect Maintainability. But there are also cases where achieving one makes another easier — Conceptual Integrity may help achieve better Maintainability. Consider these relations when designing, usually by accepting trade-offs.


The six-part general scenario

The SEI template that makes any quality attribute requirement testable:

Rendering diagram…
PartMeaning
Source of stimulusSome entity — a human, a computer system, or any other actuator — that generated the stimulus
StimulusA condition that needs to be considered when it arrives at the system
EnvironmentThe stimulus occurs within certain conditions. The system may be in an overload condition, or running, or some other condition may be true
ArtifactSome artifact is stimulated. This may be the whole system or some pieces of it
ResponseThe activity undertaken after the arrival of the stimulus
Response measureWhen the response occurs it should be measurable in some fashion, so the requirement can be tested

Example — modifiability: Source: developer. Stimulus: wishes to change the UI. Environment: at design time. Artifact: code. Response: modification is made, no side effects. Response measure: in 3 hours.

QA requirements can be tracked as acceptance criteria. Options for elaborating them: user stories; in SAFe, NFRs modelled as backlog constraints; as quality requirements or system quality tests; as technical debt; split into the Definition of Done; acceptance criteria of a user story; constraints in a user story.

Per-attribute scenario tables

Availability — a measure of the impact of failures and faults. Mean time to failure, mean time to repair, downtime. The probability the system is operational when needed, excluding scheduled downtime:

Availability = MTTF / (MTTF + MTTR)

PartOptions
SourceInternal, external
StimulusFault: omission, crash, timing, response
ArtifactProcessors, channels, storage, processes
EnvironmentNormal, degraded
ResponseLogging, notification, switching to backup, restart, shutdown
MeasureAvailability, repair time, required uptime

Concrete (a crossing gate controller): main processor fails to receive an acknowledgement from the gate processor. Source: external to system. Stimulus: timing. Artifact: communication channel. Environment: normal operation. Response: log failure and notify operator via alarm. Measure: no downtime.

Interoperability — the degree to which two or more systems can usefully exchange meaningful information in a particular context. Exchanging data is syntactic interoperability; interpreting exchanged data is semantic interoperability. Purposes: to provide a service, and to integrate existing systems into a system of systems (SoS). The service may need to be discovered at runtime or earlier.

PartOptions
SourceA system
StimulusA request to exchange information among systems
ArtifactThe systems that wish to interoperate
EnvironmentSystems wishing to interoperate are discovered at run time, or known prior to run time
ResponseThe request is appropriately rejected and appropriate entities (people or systems) notified; or appropriately accepted and information successfully exchanged and understood; or logged by one or more of the involved systems
MeasurePercentage of information exchanges correctly processed; percentage correctly rejected

Concrete: our vehicle information system sends our current location to the traffic monitoring system, which combines it with other information, overlays it on a Google Map and broadcasts it. Source: vehicle information system. Stimulus: current location sent. Artifact: traffic monitoring system. Environment: systems known prior to runtime. Response: traffic monitor combines, overlays and broadcasts. Response measure: our information included correctly 99.9% of the time.

Performance — event arrival patterns are periodic (fixed frequency), stochastic (probability distribution) or sporadic (irregular, with no predictable pattern). Event servicing concerns latency (time between arrival of the stimulus and the system's response), jitter (variation in latency), throughput (number of transactions per second), and events and data not processed.

PartOptions
SourceExternal, internal
StimulusEvent arrival pattern
ArtifactSystem services
EnvironmentNormal, overload
ResponseProcess events; change level of service (operation mode)
MeasureLatency, deadline, throughput, jitter, miss rate, data loss

Concrete: main processor commands the gate to lower when a train approaches. Source: external — arriving train. Stimulus: sporadic. Artifact: system. Environment: normal mode. Response: remain in normal mode. Measure: send signal to lower gate within 1 millisecond.

Security — six properties:

PropertyMeaning
Non-repudiationCannot deny the existence of an executed transaction
ConfidentialityPrivacy — no unauthorized access. A hacker cannot access your income tax returns
IntegrityInformation and services delivered as intended and expected. Your grade has not been changed since your instructor assigned it
AuthenticationParties are who they say they are
AvailabilityNo denial of service — a DoS attack won't prevent you from ordering a book. A clear intersection between Security and Availability
AuthorizationGrant users privileges to perform tasks
PartOptions
SourceUser/system, known/unknown
StimulusAttack to display info, change info, access services and info, deny services
ArtifactServices, data
EnvironmentOnline/offline, connected or disconnected
ResponseAuthentication, authorization, encryption, logging, demand monitoring
MeasureTime, probability of detection, recovery

Concrete: hackers are prevented from disabling the system. Source: unauthorized user. Stimulus: tries to disable system. Artifact: system service. Environment: online. Response: blocks access. Measure: service is available within 1 minute.

Modifiability — three questions: What can change? When is it changed? Who changes it?

PartOptions
SourceDeveloper, system administrator, user
StimulusAdd/delete/modify function or quality
ArtifactUI, platform, environment, external system
EnvironmentDesign, compile, build, run time
ResponseMake change, test it, deploy it
MeasureEffort, time, cost, risk

Concrete (a restaurant locator app): the user may change the behaviour of the system. Source: end user. Stimulus: wishes to change the locale of search. Artifact: list of available country locales. Environment: runtime. Response: the user finds an option to download a new locale database; the system downloads and installs it successfully. Measure: the new locale is usable within 5 minutes of the request, with no developer involvement.

Module interdependencies that drive modifiability: data types; interface signatures, semantics, control sequence; runtime location, existence, quality of service, resource utilization.

Testability — the ease with which software can be made to demonstrate faults through testing. Assuming the software has one fault, the probability of fault discovery on the next test execution. You need to control components' internal state and inputs, and observe components' output to detect failures. Industry estimates put testing at 30–50% or more of the cost of developing a well-engineered system (Bass, Clements & Kazman, 3rd ed.).

PartOptions
SourceDeveloper, tester, user
StimulusProject milestone completed
ArtifactDesign, code component, system
EnvironmentDesign, development, compile, deployment, or run time
ResponseCan be controlled to perform the desired test and results observed
MeasureCoverage; probability of finding additional faults given a fault; time to test

Concrete (a photo editor): new versions can be completely tested relatively quickly. Source: system tester. Stimulus: integration completed. Artifact: whole system. Environment: development time. Response: all functionality can be controlled and observed. Measure: entire regression suite completed in under 24 hours.

Testability tactics split into control and observe system state — specialized interfaces, record/playback, localize state storage, abstract data sources, sandbox, executable assertions — and limit complexity — limit structural complexity, limit non-determinism.

Usability — ease of learning system features (learnability), ease of remembering (memorability), using a system efficiently, minimizing the impact of errors, adapting the system to user needs, and increasing confidence and satisfaction.

PartOptions
SourceEnd user
StimulusWish to learn/use/minimize errors/adapt/feel comfortable
ArtifactSystem
EnvironmentConfiguration or runtime
ResponseProvide ability or anticipate (support good UI design principles)
MeasureTask time, number of errors, user satisfaction, efficiency, time to learn

Concrete (restaurant locator): the user may undo actions easily. Source: end user. Stimulus: wishes to undo a filter (minimize the impact of an error). Artifact: system. Environment: runtime. Response: the filter is undone and the previous state restored. Measure: within one second.

Usability tactics split into support user initiative — cancel, undo, pause/resume, aggregate — and support system initiative — maintain task model, maintain user model, maintain system model. Both aim at the user being given appropriate feedback and assistance.

Other architecturally significant usability scenarios worth recognising, each with real architectural implications: aggregating data; cancelling commands; using applications concurrently; maintaining device independence; recovering from failure; reusing information; supporting international use; navigating within a single view; working at the user's pace; predicting task duration; comprehensive search support.


Design decisions and tactics

A system design is a collection of design decisions. Some respond to quality attributes, some to achieving functionality. A tactic is a design decision to achieve a quality attribute response, and tactics are a building block of architecture patterns — a more primitive, granular, proven design technique that sits between stimulus and response, controlling the response.

Tactics are atoms, patterns are molecules. The focus of a tactic is on a single quality attribute response; within a tactic there is no consideration of trade-offs. Trade-offs must be explicitly considered and controlled by the designer. In this respect tactics differ from architectural patterns, where trade-offs are built into the pattern. From a practical standpoint SEI tactics give only a very high-level overview of possible approaches — like quality attribute checklists, they help start a proper conversation with clients and help avoid overlooking something.

Seven categories of design decision:

  1. Allocation of responsibilities — system functions to modules
  2. Coordination model — module interaction
  3. Data model — operations, properties, organization
  4. Resource management — use of shared resources
  5. Architecture element mapping — logical to physical entities: threads, processes, processors
  6. Binding time decisions — variation of the lifecycle point of module "connection"
  7. Technology choices

Design checklists give design considerations for each QA, organised by design decision category. For allocation of system responsibilities under performance: which responsibilities will involve heavy loading or time-critical response? What are the processing requirements, and will there be bottlenecks? How will threads of control be handled across process and processor boundaries? What are the responsibilities for managing shared resources?

Testability and usability tactics appear with their scenarios above. The catalogues for availability, modifiability, performance, scalability and security are in Module 3, part 2: the SEI tactic catalogues where SEI has one; scalability uses Abbott and Fisher's rules.


The utility tree

Captures all QA requirements (ASRs) in one place. "Utility" expresses the overall "goodness" of the system.

Rendering diagram…

Construction rules:

  • The most important QA goals are the high-level nodes — typically performance, modifiability, security and availability
  • Scenarios are the leaves
  • Output: a characterization and prioritization of specific quality attribute requirements
  • Two ratings per scenario: High/Medium/Low importance for the success of the system, and High/Medium/Low difficulty to achieve (the architect's assessment)
  • When the supplier will also operate the system, add Observability (refinements: detection, diagnosis) and Cost-efficiency (refinements: unit cost, cost visibility). Without them, operability and run cost — both of which the supplier pays for — never become testable

Key: H = high (must-have), M = medium (important), L = low (nice-to-have).

The canonical SEI example, the utility tree from a Nightingale ATAM exercise (adapted from Bass, Clements & Kazman, Software Architecture in Practice, 3rd ed.; four of its rows):

Quality AttributeAttribute RefinementASRPriority
PerformanceTransaction response timeA user updates a patient's account in response to a change-of-address notification while the system is under peak load, and the transaction completes in less than 0.75 second(H, M)
PerformanceTransaction response timeThe same, while the system is under double the peak load, and the transaction completes in less than 4 seconds(L, M)
ConfigurabilityUser-defined changesA hospital increases the fee for a particular service. The configuration team makes the change in 1 working day; no source code needs to change(H, L)
MaintainabilityRoutine changesA maintainer encounters search- and response-time deficiencies, fixes the bug, and distributes the fix with no more than 3 person-days of effort(H, M)

The two performance rows differ only in environment, and the priority drops with it; the configurability row is the model for Lumen's QA10.

Sometimes a simple table rather than a tree is better for a utility tree — it is more readable and maintainable. A tree is good during whiteboarding. Feel free to use either.

Careful with attribution: a DDoS attack is not an ASR — preventing a DDoS attack is. Availability, Performance and Usability are all affected by a DDoS attack, but prevention belongs under Security.


Quality Attribute Workshops

VariantNotes
QAW (SEI)Usually hard to organise because it requires the presence of all key stakeholders at the same time, actively involved. As requirements complexity or system size increases, it makes sense to conduct the full QAW
Abbreviated workshopEasier to organise because it needs less time and fewer people. Several documented variants exist, and most consultancies keep their own. Suits a moderate system with a reasonably clear domain
Architect-led elicitationThe lightest form: the architect drafts the scenarios and validates them individually with stakeholders. Depends heavily on the architect's experience, and suits small-scale, simple requirements — not a complex or unfamiliar business domain

The eight QAW steps:

  1. QAW overview and introductions — obtain the list of attendees, take notes as appropriate
  2. Business/mission presentation — capture driving quality attributes, issues, notes
  3. Architectural plan presentation — capture driving quality attributes, issues, notes
  4. Identification of architectural drivers — share the information from steps 2 and 3, then after a few minutes ask for clarifications and corrections to the list of architectural drivers. That list helps facilitators ensure coverage during scenario brainstorming
  5. Scenario brainstorming — elicit raw scenarios from the stakeholder community in round-robin fashion, using a raw scenario table
  6. Scenario consolidation — merge similar and duplicate scenarios using stakeholders' input
  7. Scenario prioritization — each stakeholder gets votes equal to 30% of the total number of scenarios generated
  8. Scenario refinement — fully develop the scenario to include details such as how long, how much, how often, when, environment, who

The typical ASR process: Collect → Categorize → Review/Refine → Prioritize. For each "capture" step there can be one or more "refine" steps.


Deriving ASRs from business goals

Eleven categories of business goal. They are not completely orthogonal — some goals fit more than one category, and that is all right.

CategoryContent
Contributing to the growth and continuity of the organisationIf the system were not successful, the organisation would cease to exist
Meeting financial objectivesThe system may be for sale, either standalone or by providing a service, in which case it generates revenue
Meeting personal objectivesFrom "I want to enhance my reputation by the success of this system" to "I want to learn new technologies"
Meeting responsibility to employeesFor developers: ensuring certain types of employees have a role, or providing opportunities to learn new skills. For operators: safety, workload, or skill considerations
Meeting responsibility to societySome organisations see themselves as being in business to serve society. Topics: resource usage, green computing, ethics, safety, open source issues, security, privacy
Meeting responsibility to the stateRegulatory conformance or supporting government initiatives
Meeting responsibility to shareholdersLiability protection and certain types of regulatory conformance such as Sarbanes-Oxley
Managing market positionIncrease or hold market share, various types of intellectual property protection, time to market
Improving business processesImproved processes may enable new markets, new products, or better customer support
Managing the quality and reputation of productsBranding, recalls, types of potential users, quality of existing products, testing support and strategies
Managing change in environmental factorsEncourages stakeholders to consider what might change in the business goals for a system

Non-architectural solutions are legitimate outcomes: "to optimize the budget, decrease the salary of the employees"; "to allow a user-friendly way to edit rich text, buy a licence for Microsoft Word."

This really describes the essence of the difference between a software architect and a solution architect. A solution architect addresses business goals, which likely — but not necessarily — includes building software elements.


Deriving quality attribute requirements from business arithmetic

This technique is the one most often skipped. The usual failure is asking the customer "how many nines do you need?", receiving a number chosen by intuition, and designing to it. The alternative is to compute it.

Worked here on Lumen's portals and booking path — the route a referral takes from the referral desk or the patient portal to a confirmed booking. Every input comes from the case's Volumes and usage table; the one modelling assumption is stated where it is used.

Step 1 — establish the unit economics

FactValueSource
Referrals per operating day~14,000Case
Share arriving in the busiest hour12%Case
Peak arrival rate14,000 × 12% = 1,680 referrals an hourDerived
Average operating hour (07:00–19:00)14,000 ÷ 12 ≈ 1,167 referralsDerived
Cost of a referral handled manually~$9Case
Target cost of an automated booking~$2Case
Penalty per referral during an outage$9 − $2 = $7Derived

The penalty rests on one assumption: while the booking path is down in operating hours, every arriving referral falls back to manual handling by phone and email. Two consequences fall straight out:

  • An outage hour at peak costs ≈ $11,760 (1,680 × $7); an average hour costs ≈ $8,170 (1,167 × $7)
  • The $7 that prices an outage is the same $7 the platform saves on every automated booking. The business case and the availability target are written in one currency, which is what makes the target defensible to a finance director

The peak figure is deliberately the worst case: every outage hour is priced at the busiest hour's arrival rate. It also leaves out overtime and missed slots, which Lumen's outage tolerance includes. The two simplifications pull in opposite directions, so state which one you are making whenever you quote the figure.

Step 2 — turn a cost appetite into an availability figure

Lumen's finance director will accept up to ~$100,000 a year in outage-related cost — a number a finance director can actually opine on, unlike "nines".

Tolerable outage = $100,000 ÷ $11,760 per hour ≈ 8.5 hours a year, with every hour priced at peak. Priced at the average hour it would be $100,000 ÷ $8,170 ≈ 12.2 hours; design to the worst case.

Turning 8.5 hours into a percentage needs a measurement window, and the window changes the answer:

WindowHours in a yearAvailability that allows 8.5 h down
24×7 — the patient portal is open around the clock8,7601 − 8.5 ÷ 8,760 ≈ 99.90%
Operating hours only — 07:00–19:00, ~300 days3,6001 − 8.5 ÷ 3,600 ≈ 99.76%

The same tolerance reads as 99.90% or as 99.76%. Neither is wrong; they count different hours. The 24×7 figure counts a 03:00 outage that costs the desk nothing; the operating-hours figure ignores it, although patients still use the portal at night. An SLO must state its window — otherwise a supplier's "99.8%" is looser than one reading and stricter than the other, and nobody can tell which was promised. QA01 in the worked answer uses the 24×7 form, 99.9%, which allows 8.76 hours; the quarter-hour difference from 8.5 is inside the precision of the inputs, and pricing every hour at peak already errs on the strict side.

Cap a single incident at 30 minutes (an assumption to agree with Lumen) and 8.5 hours becomes a budget of at most 17 full-length incidents a year (510 minutes ÷ 30) — a statement operations can be measured against, and one that tells you how much detection-and-recovery automation is worth building.

The direction of reasoning is what matters. Nobody asked for 99.9%; it was derived from volume, cost differential and arrival rate. This also answers "how much may I spend improving availability?" — any investment that removes outage hours and pays back against $11,760 an hour is justified, and one that does not, is not.

Step 3 — decompose the budget across components

A system-level number is unusable until each component knows its share. Four components sit in series on the booking path: every booking passes through all four, and any one of them down stops it.

Each component's share = ⁴√0.999 ≈ 0.99975, so each component needs ≈ 99.975% — about 2.2 hours of downtime a year, a quarter of the system budget. Rounding to the more familiar 99.97% looks harmless but is not: four components at 99.97% compose to 0.9997 × 0.9997 × 0.9997 × 0.9997 ≈ 99.88%, about 10.5 hours a year — roughly $123,000 at peak, over the appetite. Round availability budgets toward the stricter figure, never the looser one.

BudgetOverallPortal front endBooking/workflow serviceSearchDatabase
Availability, 24×799.9% (QA01)99.975%99.975%99.975%99.975%
Availability search, p952.0 s (QA02)0.3 s0.4 s0.9 s0.4 s

The latency split is an assumption to test, not a measurement. Adding p95s is conservative — the p95 of a sum is normally below the sum of the p95s — which is the safe direction for a budget.

And then the tactic that makes those component figures achievable rather than aspirational:

To avoid placing an unreasonable availability demand on any single component, deploy at least two instances of each behind a health-checked balancer. Four components each needing 99.975% as a single instance would be an expensive and fragile promise; the same figure from a redundant pair is routine. Two instances at 99.5% each give 1 − 0.005² ≈ 99.998%, provided they fail independently — the formula and its caveats are in part 2.

Note the corollary, which is where teams get caught: series composition multiplies. Adding a fifth component to this path drops the achievable total unless every component gets stricter. The cheapest availability decision available to an architect is usually removing a hop, not hardening one. On Lumen, the hops worth removing are the ones Lumen does not run: keep the automated partner group's scheduling API and the referral management system off the synchronous booking path — partner availability is served from a cache, and referral changes arrive by extract — so their outages make data staler instead of stopping bookings.

Capacity from volumes — Little's law

The same habit — derive, don't ask — settles capacity. Little's law (John Little, 1961) says that in any stable system the average number of items inside equals the arrival rate multiplied by the average time each one spends there:

L = λ × W

For a portal, L is concurrent sessions, λ is sessions started per hour, and W is session length in hours. The law holds whatever the arrival pattern, which is why it works from business volumes alone.

Patients. 14,000 referrals × 60% portal use × 3 sessions = 25,200 sessions a day. Assume patient sessions peak with bookings: 12% in the busiest hour gives λ = 3,024 sessions an hour. With W = 4 minutes, L = 3,024 × 4 ÷ 60 ≈ 200 concurrent patient sessions.

Staff and partners. These users stay signed in for hours, so the useful estimate is the share of each group signed in during the busiest hour. The group sizes are the case's; the shares are assumptions:

GroupNamed usersShare signed in at peakConcurrent
Referral desk35080%280
Scheduling managers and administrators4050%20
Centre staff90060%540
Partner practice users1,50040%600
Total2,790≈ 1,440

Total ≈ 1,650 concurrent sessions today. For year 3, grow the referral-driven groups by 1.12³ ≈ 1.405 (patients ≈ 283, desk ≈ 393, managers ≈ 28, centre staff ≈ 759) and double the partner users (1,200): ≈ 2,650. Applying referral growth to staff numbers is itself an assumption.

Two lessons:

  • Derive public-user concurrency from sessions, never from accounts. Lumen will hold roughly 2 million patient portal accounts a year. A generic "8% of named users are concurrent" rule would give 160,000 concurrent patients — 800 times the Little's-law figure
  • The derived numbers are modest. About 1,650 sessions produce 1,680 referrals and, at an assumed four availability searches per referral, 6,720 searches in the busiest hour — under two searches a second. Lumen's scalability risk lies in the Monday-morning peak, the year-3 growth and the move into more regions, not in raw throughput. A derived figure stops a team buying scale it does not need

Common pitfalls in ASR gathering

PitfallDescription
The "shopping cart" mentalityStakeholders' false impression that specifying requirements is like filling up a shopping cart
The "this is too technical for me" attitudeStakeholders treating ASR gathering as less important than, for example, use-case modelling
The "all requirements are equal" fallacyGiving all requirements the same priority — mostly as high
The "requirements that can't be measured" syndrome"The system must be always available and very performant"
The "not enough time" complaintStakeholders say there is no time for ASR elicitation — and the project later spends far more time reworking the architecture
The "all stakeholders are alike" misconceptionAddress the right questions to the right people — always classify stakeholders as learned in Business Architecture

Making quality attributes measurable

A quality attribute you cannot measure is not a requirement, it is a wish. The table below is a compiled metric vocabulary per attribute — it is what "define a set of measurable metrics" actually means. The guidance: select the 3–5 most important quality attributes (the Task below asks for the top three), give the motivation for selecting each, list them for both baseline and target architecture, and note the component where each metric is measured.

Quality attributeCandidate measurable metrics
Conceptual IntegrityList of design patterns and styles to be used; afferent coupling (Ca); efferent coupling (Ce)
MaintainabilityCyclomatic complexity; type size; percentage of comments; efferent coupling at type level (Ce)
ReusabilityList of exact components/libraries that must be reusable; reusable code base percentage
AvailabilityAvailability excluding planned downtime (%); planned downtime (minutes per day/week/month); time required to update software/hardware on the running system (minutes)
InteroperabilityList of exact supported integration protocols/standards; backward compatibility for integration API (%); integration API breaking changes (%)
ManageabilitySystem logs collected (yes/no); logging level changeable at runtime (yes/no); troubleshooting tools exist, actual, documented and known to administrators (yes/no); monitored by third-party tools (yes/no); exact list of information collected/traced/monitored for diagnostics and troubleshooting
ObservabilityTime to detect and time to localise a failure (minutes); share of business transactions traceable end to end (%); share of components emitting latency, traffic, errors and saturation (%); share of pages that needed action (%)
PerformanceEstimated end-users by location (total); concurrent users per location (average/peak); data storage size and estimated growth per year; number of records/documents in storage; page load time at p95/p99 (ms); function call time at p95/p99 (ms) — percentiles, not means, which hide the slow tail
ReliabilityFailure rate (failures per unit time); MTTF; MTTR; MTBF; time to switch to disaster recovery environment (seconds); number of Critical and High severity customer-reported bugs
ScalabilityArchitecture allows horizontal scaling (yes/no); time to scale up/down (seconds/minutes); scaling limits sufficient for the domain (servers, network bandwidth, disk); exact components that must scale out; exact scale-out conditions
SecurityPII security scenarios; ability to detect DDoS (yes/no); ability to react to DDoS (yes/no); access restricted by authentication/authorization (yes/no); prevents SQL injection (yes/no); prevents XSRF/CSRF (yes/no); secured connection (yes/no); passwords stored only as salted hashes from a slow key-derivation function such as Argon2id, bcrypt, scrypt or PBKDF2 — never encrypted (yes/no); audits and logs all user interaction for critical operations (yes/no); sensitive data protected — encrypted, not logged, secure channels only
TestabilityUnit test coverage (%); integration test coverage (%); exact list of required test environments (functional, performance, security); exact list of test approaches (manual/automated, unit, end-to-end, regression, integration)
AuditabilityList of operations that must leave an audit trail in 100% of cases; exact parameters about users and their activities recorded for audit
UsabilityReference to the specific UI/UX guideline to follow; list of devices, resolutions, OS versions, browsers/versions, locales/cultures to support; accessibility conformance level (for example WCAG 2.2 AA, EN 301 549 in Europe, or Section 508 for US federal systems); accelerators such as hotkeys and suggestion lists; number of clicks to reach particular functionality; mean time for an average user to get used to the system (minutes)
Cost-efficiencyRun cost per business unit — per referral, per booking — against a target; share of spend allocated to a component by tag (%); idle spend — resources running with no traffic (%); forecast error (%)

How ASR extraction goes wrong

These are the failure modes worth rehearsing, because they recur on nearly every engagement.

Copying instead of reformulating. A requirement lifted verbatim from the customer's document inherits its vagueness. If you cannot state the response measure, you have not finished writing the ASR.

Missing the architecturally significant functional requirements. Teams scan for quality attributes and skip the functional list entirely — but integration requirements hide there, and they are the ones that reshape the architecture. "One partner group is automated, about six hundred partners are manual" is a functional requirement with more architectural consequence than most of the quality attributes.

Discarding a generic phrase instead of clarifying it. "Highly configurable to support growth" is not usable as written — which makes it a requirement that needs a conversation, not one to drop. Dropping it is how you discover at UAT that configuration was expected to be self-service.

Confusing a testable requirement with a constraint. "One coherent look and feel across every portal" is a requirement you verify; it is not a constraint that removes design freedom. Filing it as a constraint hides it from the test plan.

Confusing interoperability with portability. Interoperability is about the quality of an integration — protocol, format, latency, error semantics. Portability is about running in a different environment. An integration is usually both a functional requirement (we must integrate) and a quality attribute requirement (within 5 minutes of each extract, over TLS, with defined failure behaviour).

Offering process measures as architectural tactics. "Ensure the team includes experienced operations engineers" may be sound advice and is not an architectural tactic. A tactic is a design decision about the system. Similarly, "it will be hosted by the supplier" is a constraint that creates quality attribute problems; it does not solve any.

Not separating assumptions from stated requirements. Where you assume a figure because none was given, mark it visibly. Otherwise your assumption is indistinguishable from the customer's commitment, and nobody ever revisits it.

Accepting an availability figure without pricing it. Nines are bought with redundancy, and each nine costs multiples of the last. If you cannot show the cost of an outage hour, you cannot tell whether the number you have been handed is too strict or too lax — which is why the business-arithmetic method derives the figure instead.


Check yourself

  1. A customer writes "the system must be fast and always available". What is missing before this is an ASR, and how do you supply it?
  2. Why does "delivered as a service, hosted by the Supplier" belong in the constraints list rather than in the utility tree?
  3. Lumen's 8.5-hour outage budget can be written as 99.90% or as 99.76%. Which belongs in the SLO?
  4. A team proposes 99.97% for each of the four components on Lumen's booking path. Does that meet QA01?
  5. Why is removing a hop often a cheaper availability decision than hardening one?
  6. Why derive patient concurrency with Little's law instead of applying a concurrency percentage to the number of accounts?
  7. Your utility tree has six (H, H) scenarios. What do you do before choosing tactics?
  8. What separates a tactic from a pattern, and why does the difference matter when you choose tactics?
  9. ISO/IEC 25010 lists neither observability nor cost-efficiency. Is that a reason to leave QA17 and QA18 out of Lumen's utility tree?
  10. Lumen's run cost is about $0.055 per referral, against a target of about $2 for an automated booking. Why does QA18 still rate H for business value?

Suggested answers

  1. A response measure and an environment — and usually a named source and stimulus. Rewrite it as six-part scenarios, for example p95 ≤ 2 s in the busiest hour and 99.9% measured 24×7, and mark the numbers as assumptions until the customer confirms them.
  2. It removes design freedom, but nobody can stimulate it and measure a response, so it cannot be prioritised as a scenario. Its consequences — every integration becomes an external call — appear as QA scenarios for latency, security and synchronisation, and those go in the tree.
  3. Either, provided the window is stated; the two figures count different hours. The 24×7 form also covers night-time patient use, which is why QA01 uses it. A figure without a window cannot be held to anything.
  4. No. 0.9997⁴ ≈ 99.88%, about 10.5 hours a year — roughly $123,000 at peak, over the $100,000 appetite. Each component needs ≈ 99.975%, which a redundant pair makes routine; round toward the stricter figure.
  5. Availabilities in series multiply, so every component on the synchronous path lowers the total however good it is. Taking the partner API and referral system off the path — cache and extract — turns their outages into stale data rather than stopped bookings, and costs the remaining components nothing.
  6. Public-facing accounts are numerous and mostly idle, so any percentage is a guess applied to the wrong number. Little's law needs arrival rate and session length, which the case gives: about 200 concurrent patients, against 160,000 from "8% of 2 million accounts".
  7. Treat it as a symptom: the ranking has not been done. Check that no constraint or functional requirement crept in, then take it back to the stakeholders and force an order. Expect some ratings to drop once early decisions are taken.
  8. A tactic is one design decision aimed at one quality-attribute response, with no trade-off analysis built in; a pattern bundles tactics and carries its trade-offs with it. When you pick tactics, the trade-offs between them are yours to find and state.
  9. No. A standard list is a prompt for the conversation, not a filter. Both scenarios have testable response measures, and both carry significance indicators. QA17 has wide impact: every component and every asynchronous hop must carry trace context, which is expensive to retrofit. And without it, QA01 cannot be operated: a 5-minute detection already uses a sixth of Step 2's 30-minute incident cap.
  10. The business case does not depend on it: $0.055 is under 3% of the $2. The bid does. The case scores five-year total cost at 25% of the evaluation and asks the Supplier to meet the requirements cost-efficiently; Module 9's model puts five-year run cost alone at about $1.37 million. Unit cost also shows whether growth is efficient: the platform lines are fixed, so it should fall as volume grows, and a rise is a warning.

Exercises — Module 3

Task. Extract ASRs from Lumen Diagnostics and turn the quality-attribute ones into a utility tree plus tactics.

Inputs: the case, including its Volumes and usage table, and the drivers, goals and stakeholder register from your Module 2 Task.

Output: a 4–6 page document containing an ASR table (functional, quality attribute and constraints, each with priority and a one-line rationale); at least three QA ASRs written as full six-part scenarios; an availability target derived from Lumen's outage tolerance for a request path other than the booking path worked above; a utility tree (drawing or table); tactics for the top three quality attributes, applied to Lumen; and a release-checkpoint test for each.

Time: about 4 hours. For tactic depth — the catalogues, availability formulas, SLOs and recovery objectives — use Module 3, part 2.

ASRs

  • Read Chen, Ali Babar & Nuseibeh, Characterizing Architecturally Significant Requirements (IEEE Software 30(2), 2013) first
  • List only ASRs, in three sections: Functional, Quality Attributes, Constraints
  • Reformulate every ambiguous requirement; do not copy the customer's wording
  • Prioritise each ASR on business value and architectural impact
  • Write a short rationale for why each one is architecturally significant

Scenarios and numbers

  • Write at least three QA ASRs as six-part scenarios, each with an environment and a response measure
  • Pick a second request path — the partner portal's slot upload, or stage-change notifications — price an outage hour on it, derive its availability target with the window stated, and split the target across its components
  • Take volumes from the case; where you need a share or a rate the case does not give, label it as an assumption
  • The Supplier will run the platform (C01), so consider an operational scenario — a detection time, or a run cost per referral — and say why you kept or dropped it

Utility tree

  • Build a utility tree of QA requirements (SEI drawing, or a table)
    • QA only (constraints optional); no functional ASRs
    • H / M / L from both the business and architectural views
  • Mark assumptions visually, distinct from stated requirements
    • Example: you set the response-time target to 2 s at p95 because the brief gave no figure

Tactics

  • Take tactics from the catalogues in Module 3, part 2 and Software Architecture in Practice; cite what you use
  • From H,H / H,M / M,H scenarios, pick the top-3 quality attributes
  • Choose tactics and show how they apply to Lumen, not a random SEI list
  • Describe how you will check those attributes at a release checkpoint

Self-assessment

CriterionWhat good looks like
Only ASRsEvery item has a one-line reason it is architecturally significant — cost of change, wide impact, strictness, a trade-off it targets. Plain features are left out, and constraints are not dressed up as quality attributes
TestableEvery QA ASR has an environment and a response measure; at least three are full six-part scenarios that a tester could turn into a test without asking you
Traceable numbersEvery figure either comes from the case, with the row it came from, or is labelled as an assumption. Capacity comes from volumes, not from a guessed concurrency ratio
Derived availabilityThe target is computed from the outage tolerance, its measurement window is stated, and the component split rounds toward the stricter figure
Real prioritisationFew (H, H) scenarios, no constraints in the tree, and a sentence on why each of the top three outranks the rest
Tactics that biteEach tactic is named from a catalogue, tied to a Lumen component, and has a release check that could fail

Worked answer — ASRs for Lumen Diagnostics

Attempt the Task above before reading this. It is one defensible answer, not the only one: compare your choices with it rather than copying it.

The Task done against the case. The Priority column carries two values: business value and architectural impact, in that order. SEI's utility tree rates importance and difficulty to achieve; rating architectural impact instead is a common variant, used here and in the Task — whichever you use, say so.

Every figure below that the case does not give — availability targets, response times, recovery objectives, time limits, the device matrix, detection times, the unit-cost ceiling — and QA05's reading of the health-data regulation requirement as data residency, is the architect's proposed assumption. The case supplies the volumes and costs in its Volumes and usage table, the 12% growth, the two new regions, the 15-minute referral extract and TLS 1.3; QA01 and QA06 are derived from those, with their assumptions shown in the method sections above; QA18's baseline comes from Module 9's cost model, which is built on the same volumes. A real answer marks each assumption as one to confirm with the customer.

Functional ASRs

Most functional requirements are not architecturally significant. These are, and note how many of them are about integration — that is the usual pattern.

#Functional ASRBusArchWhy it is architecturally significant
F01Intelligent search across partner availability and manually uploaded slotsMHCannot be served from the transactional store at acceptable latency — implies a dedicated search component and an indexing path, which changes the data flow
F02Automated integration with one partner group's scheduling APIMMIntroduces an outbound dependency on a third party, with its own failure and latency behaviour. Whether it sits on the synchronous booking path is an availability decision — here it does not
F03Manual integration path for the remaining ~600 partnersMHTwo very different ingress paths for the same domain concept. Points to one partner-availability API consumed by both the manual upload UI and the automated adapter, rather than two parallel implementations
F04The appointment workflow must be changeable and extensible after go-liveHHThe stage list is stated as a requirement and as something that will change. Hard-coding the seven stages fails the second half — pushes toward an externalised workflow definition
F05Consolidate multiple studies for one patient into a single visitMMIntroduces a grouping aggregate above the appointment, affecting the domain model and the booking transaction boundary
F06Single sign-on: staff through corporate SSO, patients through the patient identity serviceMMTwo identity populations with different assurance levels and lifecycles. Determines the identity architecture and the trust boundary between patient-facing and staff-facing surfaces. Partner users fit neither — a clarification question
F07Merge referral-system updates with in-platform amendments by a defined policy, alerting affected partiesMHTwo writers to one referral: the 15-minute extract, and amendments made in the platform by patients and the Scheduling Manager. Needs a per-field merge policy, idempotent processing of each extract and an alert path — it shapes the synchronisation component and the data model
F08Prioritise offers on clinical match plus Scheduling-Manager rules that change over timeHHRules that change without a release point to a rules component with versioned rule sets, separate from the booking workflow. Pairs with QA10

Quality attribute ASRs

#QA ASRBusArchNote
QA01Patient and Administration portals and the booking path available 99.9%, measured 24×7 over a rolling year (at most 8.76 h down); Partner portal 99%MHDerived, then negotiated. $100,000 ÷ $11,760 per peak outage hour ≈ 8.5 h a year — see the cost-of-outage method above. The SLO states its window
QA02Routine screens respond within 2 s at the 95th percentile in the busiest hourHHThe customer said "feel immediate". This makes it testable: a percentile, not an average, under a stated load. Full scenario below
QA03Daily operational reports within 60 s; analytical reports within 10 minutesMMSplits reporting by class, because one number for both is unachievable or wasteful
QA04Core tasks work on the agreed device and browser matrix; on a phone, task success and time are on par with desktopHHForces the native-versus-responsive-web decision, which is architectural, not cosmetic. Matrix and parity measure below
QA05Patient data — including backups and audit records — is stored and processed only in the jurisdiction where it was collectedHHPrivacy and compliance. Given planned expansion outside the current jurisdiction, this constrains the deployment topology and the data model
QA06Support ≈ 1,650 concurrent sessions in the busiest hour at launch, rising to ≈ 2,650 by year 3HHDerived from case volumes with Little's law — see the capacity derivation above. The shares of staff signed in, and growth applied to staff numbers, are assumptions
QA07Serve new regions; outside the primary region p95 must not degrade by more than 50%MHTurns "we will enter two more regions" into a measurable latency obligation
QA08Promotion of a release causes no more than 10 minutes of reduced serviceMMMakes "robust promotion procedures" measurable, and effectively selects a deployment strategy
QA09All authenticated traffic over TLS 1.3 or betterMMAlready specific in the source — carry it through unchanged
QA10A clinical-protocol or policy change is applied by the configuration team within one working day, without a code changeHMThe only way to make "configurable enough to absorb change" mean anything. Full scenario below; part 2's cost-of-modifiability inequality tests whether the mechanism pays
QA11Referral-system changes are reflected in the platform within 5 minutes of the extract that carries themHMMakes "frequent synchronisation" a number. "Within 15 minutes of the change" cannot be met by polling: the referral system publishes an extract every 15 minutes, so a change made just after one extract waits up to 15 minutes for the next, plus processing — about 20 minutes worst case. Measure from the extract, or ask Lumen for an event feed or a shorter interval; raise it as a clarification rather than burying it
QA12A partner user cannot read or change another partner's bookings, slots or patients; every attempt is refused and loggedHMSecurity. Some 600 practices share one portal, so isolation must hold in the API and data layer, not only in the UI. Full scenario below
QA13Every read or change of patient health information writes an audit record; one patient's access history can be produced within 1 hourHMPrivacy and compliance. Health-data regulation usually demands accountability for access, not only protection. Every access path must emit the record, so it belongs in the platform, not in each feature. Audit records hold identifiers, so QA05 applies to them too
QA14If a hosting region is lost, service resumes in a recovery region within 4 hours (RTO), losing at most 15 minutes of committed bookings (RPO)MHTurns "highly available and resilient" into recovery objectives. A 4-hour loss at peak costs ≈ 4 × $11,760 ≈ $47,000 — almost half the year's appetite in one event, which is why RTO is priced, not picked. In the new jurisdiction the recovery region must also satisfy QA05: a trade-off point if the provider has only one region there
QA15When a centre closes unexpectedly, its bookings for the next 48 hours are re-offered at other centres or partners within 2 hours of the Scheduling Manager marking it closed, with patients notified by email and SMSHMThe case names "a centre outage" as a design driver. An average centre performs fewer than ~80 studies a day (4.2M ÷ 180 ÷ 300 ≈ 78; partners perform some of the 4.2M), so ~160 bookings must be re-searched and re-offered in one burst — a bulk operation the one-at-a-time booking workflow has to support. The remaining time within Module 2's O5.2 (4 operating hours) covers detecting and marking the closure
QA16A new region, including one in a new jurisdiction, is in service within 8 weeks of the go decision, using configuration and infrastructure templates and no application code changeMHTurns "two further regions within three years" into a measurable scenario. Region as configuration rather than a code branch is what QA05's partitioning has to make possible
QA17A booking-path failure or slowdown that threatens QA01 or QA02 pages on-call within 5 minutes, naming the component consuming the budget; any referral can be traced end to end, from the extract that carried it to the patient's email and SMSMHObservability. The Supplier runs the service (C01), so detection time is the Supplier's cost: 5 minutes comes out of the 30-minute incident cap assumed in Step 2, leaving 25 to recover. Tracing a referral means carrying one correlation token across every asynchronous hop — extract, sync, booking, stage change, notification queue — so the impact is architectural, not a tooling purchase. Full scenario below
QA18Run cost per referral — cloud bill plus messaging invoices — stays at or below $0.06 at every volume up to year 3 (≈ 490,000 referrals a month); a rise of more than 10% above trend alerts the cost owner within one dayHMCost-efficiency. Baseline from Module 9's cost model: ≈ $19,360 a month ÷ ≈ 350,000 referrals (14,000 × ~25 operating days) ≈ $0.055. The $0.06 ceiling is an assumption. Business value is H because the case scores five-year total cost at 25% and asks for cost-efficiency; architectural impact is M because the big levers — SMS policy, the number of regional stacks — are set by other decisions, which QA18 then holds to account. Full scenario below

Six-part scenarios — QA02, QA10 and QA12

The measures are assumptions to confirm, except where they restate the case.

PartQA02 — PerformanceQA10 — ModifiabilityQA12 — Security
SourceA referral-desk agent, partner user or patientThe Scheduling Manager, through the configuration teamAn authenticated partner user — or whoever holds a stolen partner account
StimulusOpens a routine screen: worklist, booking detail, availability search or offer confirmationRequests a clinical-protocol or policy change, such as a new preparation rule for a modality or a changed prioritisation weightingRequests a booking, slot or patient record belonging to another partner, for example by editing an ID in a URL or API call
ArtifactThe booking path: portal front end, booking/workflow service, search, databaseRule sets and the workflow definition, held as versioned configurationPartner portal API and the booking store
EnvironmentBusiest hour: 1,680 referrals and 6,720 availability searches an hour, ≈ 1,650 concurrent sessions, no component degradedProduction, during operating hours, with bookings in flightProduction, normal operation
ResponseThe screen is returned complete and correctThe change is made in the Administration Portal, validated against test referrals, versioned and applied to new bookings; in-flight bookings keep the version they started withThe request is refused as "not found", so the response does not confirm that the record exists; the attempt is logged with user, partner, target and time; repeated attempts alert the security team
Response measurep95 ≤ 2 s per screen class, measured at the portal front end over each hourLive within one working day of the request; no code change or release; the previous version restorable within 15 minutes100% of cross-partner requests refused, verified at each release by automated tests over every partner-facing endpoint; an alert after 5 refusals in 10 minutes

Six-part scenarios — QA17 and QA18

These two describe how the Supplier runs the platform rather than what users see, so the stakeholders are the Supplier's on-call engineers and Lumen's finance director.

PartQA17 — ObservabilityQA18 — Cost-efficiency
SourceA fault or regression in a booking-path component or dependency: a bad release, a saturated database, a slow search nodeGrowth in referral volume; a design, configuration or price change; a runaway process or a forgotten environment
StimulusBooking-path requests start failing or slowing fast enough to put QA01 or QA02 at riskMonthly volume rises toward year 3, or a change raises spend — an extra SMS stage, a chattier service, a test stack left running
ArtifactThe booking path and the referral-to-notification flow — referral sync, booking/workflow service, search, database, notification dispatch — and the telemetry pipeline that observes themThe production and non-production estate, its bill allocated to components by tag, and the messaging provider's invoice; the monthly referral count
EnvironmentProduction, 24×7, including during a release and at night, when real traffic is too thin for an error ratio to mean muchNormal operation, any month from go-live to year-3 volume (≈ 490,000 referrals a month)
ResponseOn-call is paged with the SLO at risk, the component consuming the budget and example traces. From a referral, support retrieves its whole path: extract run, sync, booking, each stage change, each email and SMS dispatchCost per referral is computed daily from the allocated bill and the referral count; a deviation alerts the cost owner, naming the component whose spend moved
Response measurePaged within 5 minutes once a quarter of booking-path requests fail, or a quarter of one screen class's requests take over 2 s; slower burns page or raise a ticket on longer windows. Any referral from the last 30 days traceable end to end within 5 minutes of the question. No patient-identifying data in telemetry≤ $0.06 per referral in every month up to year-3 volume (baseline ≈ $0.055). The rolling 7-day figure more than 10% above the previous 28 days alerts within one day. At least 95% of spend allocated to a component

Both measures come from arithmetic, not taste. The "quarter of requests" is where a standard burn-rate alert on QA01's budget fires inside 5 minutes; smaller failure rates page more slowly, by design. The $0.06 ceiling gives about 10% headroom: growth alone pulls unit cost down to ≈ $0.050, because the platform lines are fixed, while the two new regional stacks push it back to ≈ $0.057. Part 2 derives both: burn-rate alerting and cost per referral.

QA04 — device matrix and task-completion parity

"Full functionality on phones and tablets" becomes testable only once the devices and the tasks are named. This matrix is the proposal to agree at discovery, alongside C02's framework-version list:

Device classViewport widthBrowsers (latest two major versions)Portals and tasks
Phone360–430 pxSafari on iOS, Chrome on AndroidPatient portal: every task. Partner portal: view bookings, upload slots
Tablet768–1,024 pxSafari on iPadOS, Chrome on AndroidEvery portal, every task
Desktop1,280 px and widerChrome, Edge, Safari, FirefoxEvery portal, every task

Parity measure: in usability tests on the matrix, patients completing view offer → confirm or decline → save the confirmation on a phone succeed at least 95% of the time, with median task time within 20% of desktop; every portal meets WCAG 2.2 AA. Administration configuration on a phone is deliberately out of scope — say so, rather than let "full functionality" imply it.

Constraints

#ConstraintBusArchNote
C01Delivered as a service, hosted and operated by the SupplierHHPlaces the system outside the customer's network — every integration becomes an external call, with the security and latency consequences that follow
C02Web UI on currently supported framework versions; exact list agreed at discoveryMMRecords the vague "current web technologies" line as the constraint it actually is, and defers the specifics honestly

Constraints stay out of the utility tree. Nobody can stimulate "hosted by the Supplier" and measure a response, so C01 cannot be prioritised as a scenario; its consequences appear in the tree as QA02, QA09 and QA11.

The utility tree that follows

QARefinementScenarioPriority
PerformanceResponse timeQA02 — routine screens within 2 s at p95 in the busiest hourH, H
PerformanceResponse timeQA03 — daily reports within 60 s; analytical reports within 10 minM, M
PerformanceGeographic latencyQA07 — outside the primary region, p95 degrades by no more than 50%M, H
Privacy and complianceData residencyQA05 — patient data, backups and audit records stay in the jurisdiction of collectionH, H
Privacy and complianceAccess accountabilityQA13 — every access to patient health information audited; one patient's history produced within 1 hH, M
SecurityConfidentialityQA12 — 100% of cross-partner requests refused and loggedH, M
SecurityTransportQA09 — all authenticated traffic over TLS 1.3 or betterM, M
AvailabilityDowntimeQA01 — Patient and Administration portals and booking path 99.9%, measured 24×7M, H
AvailabilityDowntimeQA01 — Partner portal 99%M, M
AvailabilityDisaster recoveryQA14 — region loss: RTO 4 h, RPO 15 minM, H
AvailabilityDisruptionQA15 — centre closure: next 48 h of bookings re-offered within 2 hH, M
ScalabilityCapacityQA06 — ≈ 1,650 concurrent sessions at launch, ≈ 2,650 by year 3H, H
UsabilityDevice reachQA04 — agreed device matrix; phone task success and time on par with desktopH, H
ModifiabilityConfiguration without codeQA10 — protocol or policy change live within one working day, no code changeH, M
ModifiabilityNew regionQA16 — a new region in service within 8 weeks, no application code changeM, H
DeployabilityRelease impactQA08 — no more than 10 minutes of reduced service per promotionM, M
InteroperabilitySynchronisationQA11 — referral changes reflected within 5 minutes of the extractH, M
ObservabilityDetection and diagnosisQA17 — booking-path SLO burn paged within 5 min; any referral traced from extract to notificationM, H
Cost-efficiencyUnit costQA18 — run cost ≤ $0.06 per referral up to year-3 volume; deviation alerted within a dayH, M

Notice the problem this tree exposes: four scenarios, under four different attributes, came out (H, H), and the Task asks for three. That is not a result, it is a symptom — the prioritisation has not really been done. Scoring constraint C01 as a "Maintainability" scenario at (H, H), a common slip, would push the count to five. Before choosing tactics, go back to the stakeholders and force the ranking. Here, taking Privacy and compliance, Scalability and Performance as the top three is a defensible reading: data residency is a legal precondition, capacity is a growth precondition — partners double and two regions arrive — and response time is what every user experiences on every interaction. QA04 is the one to argue down: once responsive web on the agreed matrix is chosen, its architectural impact falls to M. Ratings move as decisions are made.

Tactics for the top three

QA and scenarioTacticHow it applies here
Privacy and compliance — QA05, patient data stays in its jurisdictionData localisationEnumerate every jurisdiction the network operates in, including the two planned regions. Select a provider with a regional presence in each — and, for QA14, a recovery region inside each
Partition by jurisdictionOne patient-data store per region, each inside its jurisdiction; route at the identity boundary so a request is bound to its region before it touches patient data
Keep the global layer non-identifyingThe cross-region index holds only opaque keys and non-identifying attributes, so search and reporting work globally without moving regulated data
Encrypt with in-jurisdiction keysData at rest is encrypted with keys held in each jurisdiction's key service, so a copy that leaks across a border is unreadable
Scalability — QA06, ≈ 1,650 concurrent sessions at launch, ≈ 2,650 by year 3Scale out (X axis)Stateless application tier behind a load balancer; autoscale on request-queue depth rather than CPU, because requests spend most of their time waiting on the database and search rather than on CPU. Size for the Monday-morning hour that carries 12% of the day, not for the daily average
Partition data (Z axis)Region is already the partition key imposed by the compliance tactic — reuse it rather than inventing a second scheme. A new region is a new partition (QA16)
Scale out reads (X axis on the data tier)Managed database with read replicas serving the reporting and search-indexing paths; the write primary still scales up (vertically)
Performance — QA02, routine screens within 2 s at p95Response-time budgetDecompose the 2 s across the four components on the booking path and hold each to its share — the split in Step 3 above
Cache with event-driven invalidationCache partner availability with a short TTL, invalidated by partner update events rather than waiting for expiry. This also keeps the partner API off the booking path
Separate the read modelReporting reads a projection, not the transactional store, so a heavy report cannot degrade booking latency

How each is verified at a release checkpoint

  • Privacy and compliance — automated test asserting that a request carrying a region-A patient identifier cannot read or write region-B storage; plus an infrastructure audit of where stores, backups and keys live
  • Scalability — load test to the year-3 target of ≈ 2,650 concurrent sessions, with the busiest-hour mix grown by the same factor of 1.405 (≈ 2,360 referrals and ≈ 9,440 searches an hour), observing that autoscaling engages and p95 holds; confirm scale-in also works, since only scaling out is usually tested
  • Performance — synthetic transactions per screen class reporting p95 continuously, with the per-component budget instrumented so a regression identifies which component consumed its share

Tactics for QA17 and QA18

Neither is in the top three, but each gets a tactic now. Both are cheap to design in and expensive to retrofit, because the retrofit touches every component.

QA and scenarioTacticHow it applies here
Observability — QA17, SLO burn paged within 5 minutes; any referral traceablePropagate one context end to end, and alert on SLO burn rateThe referral sync gives each referral an opaque correlation token when it first arrives. The token rides with W3C Trace Context on every HTTP call and in message attributes across the notification queue, so one query returns the referral's whole path. Alerts fire on the burn rate of the booking-path SLIs, not on CPU, and a synthetic booking every minute keeps the SLI meaningful at night. Detail in part 2
Cost-efficiency — QA18, ≤ $0.06 per referralMeasure unit cost, then attack the largest lineTag every resource with component and region, divide the allocated daily bill by referrals, and alert on the rolling 7-day figure. SMS is ≈ $0.036 of the ≈ $0.055, so the lever is a requirement conversation with Lumen about which stages send SMS — not tuning functions that cost a fiftieth of a cent per referral. Detail in part 2

How each is verified at a release checkpoint

  • Observability — in staging, inject search failures sufficient to fail a quarter of booking-path requests and confirm the page arrives within 5 minutes and names search; retrieve five randomly chosen referrals end to end, from extract to SMS; scan a sample of the release's telemetry for patient identifiers and expect none
  • Cost-efficiency — price each change in the cost model before it ships (a new SMS stage, a new hop, a new always-on node), and fail the pipeline on any untagged resource. After release, the daily unit-cost alert is the running check