What you'll be able to do
- Plan and run an architecture review as a one-to-three-week assignment: agree the question, request the inputs, interview, walk scenarios through the design, and deliver a findings report
- Classify review findings as risks, non-risks, sensitivity points and trade-off points, group the risks into themes, and turn them into prioritised recommendations
- Choose between a full ATAM and a lighter method — Lightweight Architecture Evaluation, ARID, CBAM, risk-storming or a decision-centric review — for the review in front of you
- Write architecture principles with a rationale and implications, and enforce them with fitness functions, guardrails and compliance reviews
- Set up solution-level governance for a multi-party engagement: decision rights, review cadence, an ADR process and an escalation path
- Relate that solution-level governance to TOGAF's enterprise governance: the Architecture Board, principles, compliance and the Architecture Contract
Coverage note. This module covers evaluating a design and governing architecture. Its primary sources are more precise than any summary: the SEI's ATAM report (Kazman, Klein & Clements, CMU/SEI-2000-TR-004, free from the SEI), the architecture-evaluation chapter of Software Architecture in Practice (Bass, Clements & Kazman, 4th ed., 2021), and the architecture governance chapters of the TOGAF Standard. Read them alongside this module. Design itself — the design process and Attribute-Driven Design 3.0 (Cervantes & Kazman, 2016) — is taught at the end of Module 4. Here, the design is the thing under review.
The module has two halves. The first is evaluation: ATAM, how a review assignment actually runs, and the lighter methods you will use more often than a full ATAM. The second is governance: TOGAF's enterprise view, then the solution-level governance a solution architect sets up and runs on a delivery programme.
Architecture review and ATAM
Architecture review — a process where architectural decisions are evaluated as to how they enable or restrict the system in meeting its Architecturally Significant Requirements. Module 1 lists its goals; this module is about doing it.
ATAM — the Architecture Tradeoff Analysis Method — is worth seeing two ways:
- A context view: what goes in, and what comes out (the table below)
- A conceptual flow: how business drivers and the architecture meet in the analysis (the diagram further down)
| Inputs | Outputs |
|---|---|
| Business drivers and goals, presented by the sponsor | A concise presentation of the architecture and of the business goals |
| The architecture as documented, presented by the architect | Prioritised quality-attribute requirements, as scenarios in a utility tree |
| Quality-attribute requirements, as far as they are known | The architectural approaches, each mapped to the scenarios it serves |
| Stakeholders' scenarios, brainstormed during the evaluation | Risks and non-risks, sensitivity points and trade-off points, and risk themes |
ATAM evaluations run through four named phases: Phase 0 (Partnership and Preparation), Phase 1 (Initial Evaluation), Phase 2 (Complete Evaluation), and Phase 3 (Follow-up). Software Architecture in Practice gives the typical durations: Phase 0 proceeds informally over a few weeks; Phase 1 takes one to two days, followed by a hiatus of two to three weeks; Phase 2 takes about two days with the wider stakeholder group; Phase 3, the final report and follow-up, about a week. Phase 1 works through six steps — present ATAM, present business drivers, present the architecture, identify architectural approaches, generate the quality attribute utility tree, and analyse architectural approaches — and Phase 2 recaps Phase 1 for the larger group, then runs steps 7–9: brainstorm and prioritise scenarios, analyse the architectural approaches again, and present the results.
The evaluation team fills defined roles — in the SEI method a team leader, evaluation leader, scenario scribe, proceedings scribe, timekeeper, process observer, process enforcer and questioners, several of which a small team combines — alongside the assembled stakeholders. The analysis sorts findings into four categories:
- A risk is a potentially problematic architectural decision
- A non-risk is a good decision, recorded together with the assumptions it relies on, which are often left implicit
- A sensitivity point is a property of one or more components that is critical for achieving a particular quality-attribute response
- A trade-off point is a property that affects more than one quality attribute and is a sensitivity point for more than one
The dotted line is the point of the exercise. Risks are grouped into risk themes — systemic weaknesses several risks share — and each theme is traced back to the business driver it threatens. Fifteen risks mean little to a sponsor; three themes, each naming the business goal in danger, are something a sponsor can act on.
Write every finding as decision, scenario, reasoning — plus a recommendation for a risk — so that someone who was not in the room can check it. For example: the search index is refreshed from the booking database every 60 seconds (decision); when a desk user and a patient go for the same slot (scenario), the portal can offer a slot the desk booked up to a minute earlier (reasoning); so the booking transaction must re-check the slot against the database before confirming (recommendation). "Search might be stale" is not a finding; it is a worry.
Running a review as a solution architect
Most reviews a solution architect runs are not formal ATAMs. They come from four directions:
- Self-review of your own design before a bid or a design milestone — this module's Task
- Peer review inside your own organisation, before the customer sees the design
- Independent review for a customer: before go-live, of a troubled programme, or of another supplier's proposal
- Technical due diligence for an investor or acquirer: the same method, but the question is what it will cost to keep, scale or fix the system, so the findings carry cost estimates
Each borrows ATAM's vocabulary and its core move — walking scenarios through decisions — and fits them to the time and access available.
Scope and goals
Agree the question the review answers before you request a single document. "Review our architecture" cannot finish; "Can this design carry the year-3 volumes at 99.9%, and what must change before go-live?" can. Pin down:
- the sponsor, and who receives the report
- the decision the review informs: go-live, contract award, investment, re-platforming
- the yardstick: the ASRs, volumes and constraints the design is judged against. If no utility tree exists, building one is the first piece of work
- what is out of scope — code quality, the delivery process, commercial terms — unless the question needs it
Inputs to request
Send the request on day one. What arrives late, or not at all, is itself evidence.
- Business drivers, goals, and the ASRs or NFRs with their priorities
- Architecture documentation: the SAD and its views, the ADR log, interface specifications
- Deployment topology, infrastructure code and the list of environments
- Evidence of quality: load-test results, DR test reports, security assessments and the threat model
- Operational history: incident reports and postmortems, SLO reports, monitoring dashboards
- Cost: the run-cost model and recent bills
- Read access to the code repositories and the backlog, including recorded technical debt
- Team structure and the delivery plan
Interviews and walkthroughs
Interview people separately and early: the sponsor (drivers, and what "good" means to them), the architect or tech lead, two or three developers, operations, security and the product owner. People describe the system they live with, not the one in the documents, and the gap between the two is often the first finding. Ask each for the last serious incident, and for the decision they would reverse if they could.
Then run walkthroughs with the architect at a whiteboard:
- Scenario walkthrough — trace a top scenario's request through every component, with numbers: arrivals per second, calls per request, latency per hop
- Failure walkthrough — the same path when a dependency is slow, down, or returning rubbish
- Change walkthrough — a likely change, such as a new region or a new workflow stage: what is touched, who must agree, how long it takes
Where a finding rests on a claim nobody can evidence ("the index rebuild takes minutes"), measure it, or have a developer build a small proof of concept. That is what the developers on a review team are for.
Timeline and team
A review typically takes one to three weeks:
| Week | Work |
|---|---|
| 1 | Kick-off with the sponsor; inputs requested and read; interviews |
| 2 | Scenario, failure and change walkthroughs; targeted inspection of code, infrastructure and test results; a measurement or proof of concept wherever a finding needs evidence |
| 3 | Findings classified; risk themes and recommendations drafted; the draft walked through with the architect; final presentation to the sponsor |
A one-week review compresses this into interviews on day one, walkthroughs on days two and three, and the report at the end. A review that runs past three weeks usually never had its scope agreed. The team is the one Module 1 describes: at least one solution architect leading, optionally a delivery manager and a subject-matter expert to clarify requirements, plus developers for prototyping, code review or proofs of concept.
Walk the architect and team through the draft before the sponsor sees it. Nothing in the final report should surprise them. Where they disagree, record the disagreement next to the finding rather than deleting the finding: the independence and fairness that governance demands (below) apply to reviews too.
The findings report
| Section | Content |
|---|---|
| Executive summary | One page: the question, the verdict, the risk themes in business terms, and the decisions needed from the sponsor |
| Scope and method | What was in and out of scope, documents read, people interviewed, scenarios analysed |
| Drivers and scenarios | The business drivers and the utility tree the design was judged against, with priorities |
| Architectural approaches | The decisions identified, each mapped to the scenarios it serves |
| Findings | Each one classified as a risk, non-risk, sensitivity point or trade-off point, naming the decision, the scenario and the reasoning; non-risks carry their assumptions |
| Risk themes | Two to five themes, each grouping several risks and naming the business driver it threatens |
| Recommendations | Prioritised: the action, the finding it answers, effort, urgency (before go-live, next quarter, later) and a proposed owner |
| Appendices | Scenario walkthroughs, evidence gathered, interview list |
Keep the recommendations to a number the sponsor can act on — five to ten. Forty recommendations of equal weight is a way of making none.
Executive summaries
Review reports and proposals share one skill above all: writing the right executive summary is critical — and difficult. For a review, the sponsor should be able to read one page and know the verdict ("fit to go live, subject to three conditions"), the risk themes in business terms, and what you need them to decide.
For a proposal, as the capstone guidance puts it, after reading your executive summary the customer should see that:
- You understand the business, the current state and the goal(s)
- The proposed solution will cover the current and future needs
- The solution will be delivered on time and on budget
A review checklist
The scenarios drive the analysis; the checklist catches the areas they missed. Run through it once the walkthroughs are done.
| Area | Ask | Evidence that settles it |
|---|---|---|
| Traceability to ASRs | Does every ASR map to at least one decision, and every significant decision to an ASR? | Component records' "requirements covered" fields (Module 6); the ADR log |
| Quality-attribute coverage | Is each top scenario met by named tactics, at the stated future volumes — for Lumen, ≈ 2,650 concurrent sessions in year 3? | Load-test results, a capacity model, a prototype measurement |
| Failure modes and recovery | For each external dependency and stateful component: what happens when it is slow, down or wrong? Are RTO and RPO stated and tested? | A failure-mode table, DR test reports, runbooks |
| Security and data protection | Trust boundaries, identity flows, encryption in transit and at rest, key management, audit of patient-data access, data residency | Threat model, data-flow diagram with data classifications, penetration-test report |
| Operability and observability | Is there an SLI for every SLO, and can on-call tell which component consumed the latency budget? Do alerts fire on symptoms users feel? | Dashboards, alert rules, incident postmortems |
| Deployability | Can a release go live within the agreed reduced-service window and be rolled back? Are environments rebuilt from code? | Pipeline definition, deployment history, a rehearsed rollback |
| Cost | Is there a run-cost model driven by volumes, and is its dominant line known? For Lumen, SMS (≈ $12,600 a month at illustrative prices) dwarfs email (≈ $42) | Cost model, cloud bills, unit cost per booking |
| Evolution | Can the design absorb the stated growth — referrals ×1.405 over three years, partners ×2, a new jurisdiction — without redesign? Which decisions are expensive to reverse? | ADRs and their consequences, dependency analysis, the version-upgrade policy |
Lighter-weight evaluation methods
A full ATAM costs several days of a large group's time and weeks of elapsed time. A lighter method usually answers the question; choose by what is under review and the decision it informs.
| Method | Source | What it is | Choose it over a full ATAM when |
|---|---|---|---|
| Lightweight Architecture Evaluation | Bass, Clements & Kazman, Software Architecture in Practice, evaluation chapter | ATAM's steps compressed into about a day, run by people inside the organisation who already know the method and the system | The review is internal and recurring — say at each design milestone — and the stakeholders already agree on the business drivers |
| ARID (Active Reviews for Intermediate Designs) | Clements, CMU/SEI-2000-TN-009 (2000), combining ATAM's scenarios with Parnas & Weiss's active design reviews (1985) | The design's future users — developers who will build against it — brainstorm scenarios, then try to carry them out by writing code or pseudocode against its interfaces | The design is partial: an interface or service others will consume, such as Lumen's partner-availability API, and you need to know it is usable before it is finished |
| CBAM (Cost Benefit Analysis Method) | Kazman, Asundi & Klein, Quantifying the Costs and Benefits of Architectural Decisions (ICSE 2001) | Stakeholders assign a utility to each response level of the scenarios; the architect costs each candidate strategy; strategies are ranked by benefit per unit of cost | Several options meet the requirement and money decides between them — active-active versus warm-standby recovery, for example. Usually run after an ATAM |
| Risk-storming | Simon Brown, riskstorming.com and Software Architecture for Developers | The team draws C4 diagrams at two or three levels; each person silently marks risks on them, rated by probability and impact; the group converges, then agrees mitigations. An hour or two | Early in delivery, with the whole team, to surface risks fast and share ownership of them. Weak on quality-attribute trade-offs, so pair it with scenarios for the top ASRs |
| DCAR (Decision-Centric Architecture Review) | van Heesch, Eloranta, Avgeriou, Koskimies & Harrison, IEEE Software 31(1), 2014 | Reviews decisions one at a time — the forces for and against each — rather than scenarios. About half a day | The team keeps an ADR log, and you want to check at a release checkpoint whether recorded decisions still hold as volumes, requirements and constraints move |
| Well-Architected review | The cloud providers' Well-Architected frameworks (see Module 9) | A structured question set per pillar — security, reliability, cost and so on — walked with the team, producing a list of high- and medium-risk items | The system runs on one major cloud and you want a fast, repeatable health check against that provider's practice; it complements, not replaces, a scenario-based review of the solution's own ASRs |
A self-review borrows ATAM's structure at the scale of one person. Its weakness is the author's blind spot: you analyse the scenarios you designed for. Reduce it by asking a peer to supply two scenarios you did not choose, or by risk-storming your container diagram with someone who did not draw it.
Architecture governance
The conceptual setup, worth reading slowly:
Suppose you are interested that an entity works in a way you want it to work. If you are managing the entity then you have the necessary authority to ensure it. However, if you don't manage the entity directly, then what do you do? To do anything you need to have some authority or influence over the entity. You can discuss with the management team of the entity and come to an agreement on what needs to be done. Then you also come to an agreement on how to ensure that what you have agreed is followed. In other words, you lay down a governance framework.
Useful analogies to reach for: how corporate governance works, and how a government makes all parties — physical and legal entities — follow rules.
The six characteristics
The TOGAF Standard's Architecture Governance chapter (after Naidoo, Corporate Governance, 2002) lists six characteristics that show both why governance is valuable and why it is necessary, inside an organisation and in its dealings with every party involved. In paraphrase:
| Characteristic | Meaning |
|---|---|
| Discipline | Every party commits to follow the procedures, processes and lines of authority the organisation has set up |
| Transparency | What was done, and the reasoning behind each decision, can be inspected by the authorised parties on both the organisation's and the suppliers' side |
| Independence | Processes and decision-making are designed so that conflicts of interest are avoided, or at least kept to a minimum |
| Accountability | Named groups — a governance board, for example — hold the authority for the decisions and actions they take, and answer for them |
| Responsibility | Every contracted party behaves responsibly towards the organisation and its stakeholders |
| Fairness | No decision, process or implementation may give one party an unfair advantage over the others |
The three strategy elements
There are three important elements of an architecture governance strategy that relate particularly to the acceptance and success of architecture within the enterprise:
- Architecture Board
- Architecture Principles
- Architecture Compliance
Of the three, the Architecture Board usually has to come first: without a body that holds the authority, principles and compliance reviews have no owner.
It is important to consider all of these to ensure a successful approach to architecture governance, and to the effective management of the Architecture Contract — in TOGAF, the joint agreement between the parties developing an architecture and its sponsors on the deliverables, quality and fitness-for-purpose expected.
Governance in practice
Architecture governance includes control, compliance, management, accountability — the core definition from TOGAF. At enterprise level its concrete examples are:
- Consistency between sub-architectures
- Identifying re-usable components
- Architecture compliance over the enterprise
The scenario to recognise: there are different units in an organization with different projects in these units. They can build overlapping frameworks and systems in their own way. The enterprise architect's job is to detect such a situation and mitigate or fix it — by identifying reusable components, creating consistency between sub-architectures, and so on. That enterprise-level governance belongs mainly to enterprise architects.
A solution architect meets it from two sides. As a subject: your design must pass the customer's Architecture Board and comply with its principles, so read them before you design, not the week before the board meets. And as an owner: on a delivery programme, the same three elements — a body with authority, principles, compliance checks — exist at solution scale, and the solution architect usually runs them.
Solution-level governance
At solution level, governance answers three questions for one programme: who decides what, how decisions are recorded and checked, and what happens when design and delivery drift apart. These are its instruments.
Design authority and architecture review board
A design authority is the standing forum that owns the technical design of one delivery programme. The lead solution architect usually chairs it; its members are the tech leads, security, operations and, on a customer engagement, the customer's architect. It meets often — weekly during build — decides within its remit, and escalates what lies outside it.
An architecture review board (ARB) usually sits higher and meets less often. Typically it is the customer's enterprise board, reviewing the solution at gates such as design sign-off and go-live against enterprise principles and standards. For a supplier, the customer's ARB is an external gate to prepare for; the design authority is the forum you run. On a small programme one joint body can play both roles. What matters is that every class of decision has exactly one forum that owns it.
ADR review
ADRs (Module 6) become a governance instrument once they have a review step:
- Proposed ADRs, with the options considered, are circulated before the design authority meets
- The forum accepts, rejects or returns each one; acceptance is recorded with the date and who decided
- An ADR that crosses into another party's decision rights — an interface it owns, a cost line, a service level — needs that party's approval, not just notice
- A reversed decision gets a new ADR that supersedes the old one; nothing is deleted
The test of a healthy log: a newcomer can find why any significant decision was taken, by whom, and what was rejected.
Architecture principles with rationale
A principle is a rule that guides many decisions. TOGAF's template gives each one four parts: name, statement, rationale and implications. The rationale is what makes a principle governable: when two principles conflict, the rationales are what you weigh. A principle without one is an unjustified constraint — the same test Module 4 applies to design principles.
One Lumen principle, as an illustration:
| Part | Content |
|---|---|
| Name | Patient data stays where it was collected |
| Statement | Identifiable patient data is stored and processed only in the jurisdiction where it was collected |
| Rationale | Health-data regulation applies per region, and one of the two planned regions is in a new jurisdiction (Module 3, QA05) |
| Implications | One patient-data store per region; cross-region search and reporting use non-identifying keys only; any exception needs a recorded dispensation approved by Lumen |
Keep the set short: five to ten for a programme. A principle that restates good practice ("code is reviewed") belongs in the engineering standards. The principles worth governing are the ones a reasonable design could break.
Fitness functions
Ford, Parsons & Kua (Building Evolutionary Architectures, 2017; 2nd ed. 2022, with Sadalage) define an architectural fitness function as any mechanism that gives an objective assessment of how well the system meets an architectural characteristic. Turning a principle or an ASR into one moves compliance from the next review to every change. They classify fitness functions along several dimensions, among them atomic (one characteristic) or holistic (several together), triggered (on a build or release) or continual (in production), and automated or manual.
| Characteristic | Fitness function | Kind |
|---|---|---|
| Data residency (QA05) | An infrastructure-code test fails the build if any patient-data store is declared outside its region | Atomic, triggered, automated |
| Response time (QA02) | Synthetic transactions per screen class measure p95 against 2 s at busiest-hour load; a regression blocks the release | Atomic, triggered and continual, automated |
| Structure (F03) | A dependency rule fails the build if a portal calls the partner adapter directly instead of the partner-availability API | Atomic, triggered, automated |
| Availability (QA01) | A monthly error-budget report against QA01's rolling-year budget: 0.1% of 8,760 h = 8.76 h a year, about 43 minutes a month on average (720 × 0.1% = 0.72 h) | Holistic, continual, automated |
Guardrails and paved roads
Two ways to make compliance the default instead of inspecting for it afterwards:
- A guardrail is a control in the platform that prevents or flags a breach as it happens: a cloud policy that refuses to create a storage bucket outside the approved regions, a pipeline that rejects an unsigned container image, a budget alert on a cost line. Preventive guardrails stop the action; detective ones raise it.
- A paved road — Netflix's term; Spotify's version is the golden path — is the supported, documented default way to build and run a component: a service template with logging, tracing, authentication and deployment already wired to the principles. Teams may leave it, but then they own what the road would have given them, and they record why in an ADR.
Guardrails stop the worst mistakes cheaply; paved roads make the compliant choice the easiest one. Together they let most principles be met without a meeting, which leaves the design authority free for decisions that need judgement.
Compliance reviews against principles
Automated checks cover what can be measured. The rest needs a compliance review at defined checkpoints — end of design, before each release to production, and periodically in service:
- Review the design and the as-built system against each principle, with a checklist derived from the principles' implications
- Record a result per principle. TOGAF grades conformance on a scale that runs from non-conformant through compliant to fully conformant
- Where a breach is accepted, grant a dispensation: a recorded, time-limited exception with an owner and an expiry date, so a "temporary" exception comes back for review instead of becoming permanent
Governance on a multi-party engagement
On Lumen, three parties make decisions that shape the system, and only one of them is the Supplier:
- Lumen, the customer: the sponsor; the Scheduling Manager, who owns rules and workflow; its IT department ("Lumen IT"), which owns the referral management system, the patient identity service and single sign-on; and its data-protection function. In GDPR terms Lumen is the controller of patient data and the Supplier a processor — an assumption: confirm the equivalent roles under each region's law
- The Supplier, which designs, builds, hosts and supports the platform
- The automated partner group, which owns the scheduling API the platform calls, with its versions, limits and change schedule
The ~600 manual partners use the system but hold no design decisions; Lumen represents them.
The difficulty is that the Supplier is held to requirements whose levers sit with the other two parties. QA11 ("within 5 minutes of the extract that carries them") is measured from the extract precisely because the interval is Lumen's lever; it still depends on Lumen publishing each extract on time and in the agreed format. The partner integration depends on an API whose owner probably has no contract with the Supplier at all — the commercial relationship is Lumen's (confirm it). Governance has to say who holds each lever.
Decision rights. Apply the Module 2 RACI rule — exactly one party decides each class of decision:
| Decision class | Decides | Must be consulted |
|---|---|---|
| Business rules, prioritisation, workflow stages, configuration | Lumen (Scheduling Manager) | Supplier, on whether the change is configuration or code |
| Patient-data residency, retention and access | Lumen (data-protection function) | Supplier |
| Technical design inside the contract: components, technology, patterns | Supplier design authority | Lumen's architect, through the ADR log |
| Interfaces Lumen owns: referral extract, identity service, SSO | Lumen IT | Supplier |
| The partner group's API: contract, versions, rate limits | The partner group | Supplier (technical), Lumen (commercial) |
| Service levels, scope, anything that changes price | Steering committee (Lumen and Supplier) | All three parties, as affected |
Cadence. Treat the frequencies as a starting assumption to agree with Lumen:
- Supplier design authority — weekly during build; reviews proposed ADRs; Lumen's architect has a standing invitation
- Joint architecture board — Lumen and Supplier architects, every two to four weeks in build and monthly in service; owns the principles and dispensations, and approves ADRs that cross into Lumen's decision rights. The partner group attends when its interface is on the agenda
- Partner interface review — before any release, on either side, that touches the API, backed by contract tests both sides run
- Steering committee — monthly; scope, cost, service levels and escalations
Escalation. Give each level a time limit before an issue moves up:
Agree in advance what happens when a limit passes — typically, the Supplier proceeds on the documented default and records the risk. Otherwise an unresolved decision stalls delivery while the clock on the 12-month first release keeps running.
Check yourself
- A review note says: "The search index is refreshed every 60 seconds." Is that a risk, a non-risk, a sensitivity point or a trade-off point?
- ATAM builds a utility tree with the decision-makers in Phase 1, then brainstorms scenarios with a wider group in Phase 2. Why both?
- A sponsor asks you for "an architecture review" of a system going live in four weeks. What do you agree before you request any documents?
- When would you choose ARID over a full ATAM, and who are the reviewers?
- Two recovery designs for Lumen both meet the availability scenario; one costs three times as much as the other. Which method helps, and what does it add?
- What turns an architecture principle into something you can govern rather than a slogan?
- A fitness function and a guardrail can both enforce "patient data stays in its region". What is the difference, and why use both?
- The Supplier's design for Lumen depends on the referral system's 15-minute extract. Why is that a governance question, not only a design one?
Suggested answers
- As written, none: it is a property. It becomes a sensitivity point if a quality-attribute response — say, how often a taken slot is offered — depends critically on it, and a trade-off point if shortening it also hurts another attribute, such as booking latency from indexing load. Whether it is a risk or a non-risk depends on whether 60 seconds meets the scenario's response measure.
- The utility tree records what the architect and decision-makers believe matters. Phase 2 tests that against the wider stakeholders: a scenario they rank high that is missing from the tree shows the architecture was built for different priorities — itself a finding.
- The question the review must answer (for example, "fit to go live, and under what conditions?"), the scope, the ASRs and volumes the design is judged against, who receives the report and the decision it informs. Without these the review cannot finish.
- When the design is intermediate — an interface or service that others will build against — and you need to know it is usable before it is complete. The reviewers are its future users, who try to carry out scenarios by writing code or pseudocode against it.
- CBAM. It adds the cost of each option and the utility stakeholders place on each response level, so options are ranked by benefit per unit of cost instead of by whether they merely pass.
- A rationale traced to a driver or ASR, so it can be weighed when it conflicts with another principle; stated implications, so teams know what it requires; a defined check — a fitness function or a compliance review; and a route for time-limited exceptions.
- A fitness function measures and reports the characteristic, for example a build test that fails when a store is declared outside its region. A guardrail acts in the platform itself, refusing or flagging the resource. The guardrail makes the breach hard to commit; the fitness function proves, release by release, that it has not been.
- The Supplier cannot change the extract's interval, format or failure behaviour — Lumen owns them — yet QA11 depends on the extract's publication, format and failure behaviour, and the end-to-end freshness users see depends on the interval too. Without an agreed interface, a forum that decides changes to it and an escalation route, the Supplier is held to a target it does not control.
Exercises — Module 8
Task. Review your own Lumen design as an evaluation team would, then set up the governance that would keep it on course through delivery. Output: (a) a mini-ATAM review of 2–4 pages: a utility-tree excerpt for your 3–4 top scenarios, 3–5 architectural approaches, a findings table with at least 2 risks, 1 non-risk, 1 sensitivity point and 1 trade-off point, and 1–2 risk themes; (b) a one-page governance model for the Lumen engagement covering decision rights, review cadence, the ADR process, 4–6 principles with rationale, and how compliance is checked. Time: about 3 hours — 2 for the review, 1 for the governance model. Inputs: the Lumen Diagnostics case, including its volumes table; your Module 3 utility tree; your Module 4 ADD iteration and style decisions; your Module 6 SAD and ADRs.
(a) Mini-ATAM review of your design
- Restate the two or three business drivers your top scenarios serve (from your Module 2 work)
- Utility-tree excerpt: pick 3–4 top scenarios from your Module 3 tree, including the one your Module 4 ADD iteration addressed; give each a measurable response
- Architectural approaches: list 3–5 decisions from your ADD iteration and SAD, one line each, with the ADR or SAD section that records it
- Analyse: walk each scenario through the approaches it touches, with numbers from the case's volumes table; label every other figure as an assumption
- Findings: at least 2 risks, 1 non-risk with its assumptions, 1 sensitivity point and 1 trade-off point, each naming the decision and the scenario
- Group the risks into 1–2 risk themes, each naming the business driver it threatens
- One prioritised recommendation per risk
- Counter your blind spot: before you start, write down the decision you are least sure of, and make sure the analysis reaches it
(b) Governance model for the Lumen engagement
- Parties: Lumen, the Supplier and the automated partner group, plus anyone else you decide holds decision rights
- Decision rights: start from the pattern in this module and add at least two decision classes it leaves out; exactly one party decides each
- Review cadence: the forums, how often they meet, who attends — during build and in service
- ADR process: who proposes, who reviews, which ADRs need Lumen's approval, how superseded ADRs are kept
- 4–6 principles, each with name, statement, rationale (traced to a driver or ASR) and implications
- Compliance: for each principle, how it is checked — a fitness function, a review checkpoint, or both — and how dispensations are granted and expire
- Escalation path with time limits
- Assign each risk theme from (a) to the forum that owns it
Self-assessment
| Criterion | What good looks like |
|---|---|
| Scenarios | Each traces to an ASR ID from Module 3 and has a measurable response; at least one is quantified from the case's volumes table |
| Approaches | Named decisions from your own design, each pointing to the ADR or SAD section that records it — not generic pattern names |
| Findings | Each names a decision, a scenario and the reasoning; risks state a consequence; the non-risk lists its assumptions; the trade-off point names both quality attributes it pulls between |
| Risk themes | Each groups at least two findings and names the business driver at stake; a sponsor could act on it without reading the findings table |
| Decision rights and cadence | Exactly one deciding party per decision class, including the classes Lumen or the partner group own; every forum has a frequency, members and a remit |
| Principles and compliance | Every principle has a rationale you could defend to Lumen and at least one concrete check; at least two checks are automated fitness functions |
Worked answer — a mini-ATAM fragment for Lumen
Attempt the Task before reading this. It is deliberately a fragment — two scenarios, two approaches, six findings, one risk theme — to show the reasoning, not a full report. The decisions are plausible ones, not the only right ones; every figure the case does not give is marked as an assumption.
Utility-tree excerpt
| ID | Quality attribute → refinement | Scenario | Priority |
|---|---|---|---|
| S1 | Interoperability → synchronisation (QA11 as first worded) | A referral cancelled in the referral management system during Monday's busiest hour is reflected in the platform — referral closed, held slot released, patient notified — within 15 minutes (the referral desk's wording in the evaluation workshop, before QA11 was restated) | H, M |
| S2 | Modifiability → reaction to change (QA10, F04) | The Scheduling Manager asks for a Pre-authorisation stage between Triaged and Offered for MRI referrals. The configuration team applies it within one working day with no code change; studies already past Triaged finish on the definition they started on | H, M |
Architectural approaches
| ID | Decision | Serves |
|---|---|---|
| A1 | Poll the referral system's 15-minute change extract. An ingestion job upserts each referral, keyed on referral ID and source timestamp, applies the merge policy, and emits referral amended and referral cancelled events | QA11, change management |
| A2 | A workflow engine runs the stage list as a versioned definition held in configuration. Each study is pinned to the definition version it started on; stage transitions emit events that drive notifications and update the search index | F04, QA10 |
Walking S1 through A1. Take the worst case: the cancellation lands just after an extract is cut, so it waits 15 minutes for the next one. Add the delay before the extract is published (assume up to 2 minutes), the ingestion job's poll interval (assume 1 minute), and processing. In the busiest hour a run carries 1,680 ÷ 4 = 420 new referrals, plus amendments and cancellations (assume 20% more, ≈ 504 records); at an assumed 50 ms a record that takes about 25 seconds. Releasing the slot and queuing the notification take seconds. Worst case: 15 + 2 + 1 + 0.5 ≈ 18.5 minutes. On average the wait for the next extract is half the interval: 7.5 + 2 + 0.5 + 0.5 ≈ 10.5 minutes. The design meets the target on average and misses it in the worst case — by construction, not under load.
Walking S2 through A2. A stage that only assigns, waits, notifies or times out is a new definition version, released as configuration. A Pre-authorisation stage that must call a payer or another external system is not: that call is code.
| # | Type | Finding |
|---|---|---|
| 1 | Risk | The scenario as the stakeholders phrased it cannot be met by polling — the same problem Module 3's worked answer found when it restated QA11. The target equals the extract interval, leaving nothing for publication, polling or processing; the worst case is ≈ 18.5 minutes. Recommendation: agree QA11 with Lumen in its restated form, "within 5 minutes of the extract that carries them" — at worst 15 + 2 + 5 = 22 minutes after the change — and make sure the evaluation workshop's stakeholders accept it; or ask Lumen IT for a shorter interval or a change feed |
| 2 | Sensitivity point | The extract interval. S1's response moves minute for minute with it, and the Supplier does not control it: Lumen IT does |
| 3 | Non-risk | Processing each run as one batch. ≈ 504 records in ≈ 25 s uses under 3% of the 900-second window; at year-3 volumes (× 1.405, ≈ 710 records) about 35 s. Assumptions: at most 50 ms a record, and the extract is a delta rather than a full snapshot of open referrals. If it is a snapshot, re-assess |
| 4 | Risk | A lost extract is silent. If the extract is a delta and one run fails or is skipped, its changes are not late but lost, and nothing detects it. Recommendation: track extract sequence numbers; alert on a gap, or when the last processed extract is more than 20 minutes old; agree with Lumen IT how a missed extract is re-requested |
| 5 | Risk | QA10's "no code change" holds only for stages built from existing step types; a stage that calls an external system needs development. Recommendation: agree a catalogue of configurable step types with Lumen and word QA10 against it, so a new step type is a change request |
| 6 | Trade-off point | Pinning each study to its starting definition version. It serves modifiability — a stage change is safe to release in a day, with no migration of studies in flight — but for as long as the longest-running study lives (Requested → Billed can take weeks: assumed), two or more stage lists are live at once. Reports, the Administration Portal and regression tests must handle each, a cost to testability and reporting |
Risk theme — timeliness and flexibility are promised against interfaces nobody has agreed. Findings 1, 2 and 4 hinge on the referral extract — its interval, its delta semantics, its failure handling — which Lumen IT owns and no agreement describes. Finding 5 hinges on a configuration catalogue that does not yet exist. Both threaten the Automation driver: a referral that has to be put right by hand costs about $9 of desk time against the $2 automated target. The remedy lies in governance more than in design — an interface agreement for the extract, and a configuration catalogue in the contract — which is why part (b) of the Task follows part (a).
A full answer would add the search index for F01, whose refresh interval is a second sensitivity point on S1 (a released slot is not bookable until the index sees it), and scenarios for QA01, QA02 and QA05.