What you'll be able to do
- Choose between virtual machines, containers and serverless functions for a workload, and explain the trade-off in isolation, start-up time and operating effort
- Apply the NIST cloud model, shared responsibility and the immutable-infrastructure rules to a deployment decision
- Size compute, memory, disk and network from a workload's numbers, and plan the performance tests that confirm the estimate
- Choose a web and mobile delivery approach, and a cache topology, eviction policy and persistence pattern, from the users and the access pattern
- Derive monthly call and message volumes from business counts, price them, and add the database, backup, egress, observability, non-production and per-region rows a real run-cost model needs
- Compose a deployment's availability from provider SLAs, project its run cost over five years, and name the line that dominates each
- Turn a run-cost model into unit cost, allocation and commitment decisions with the FinOps Foundation's framework
- Specify an observability platform that measures the SLOs, alerts on error-budget burn and keeps patient data out of telemetry
- Map the AWS, Azure and Google Cloud Well-Architected pillars to quality attributes, and run a Well-Architected review knowing what it misses
This unit is where the design meets a price list. Modules 3 to 6 gave Lumen its quality-attribute requirements, its styles and its views; here you choose what they run on — the compute model, the cloud services, the node sizes, the front-end and cache approach — and then do the two calculations an RFP evaluator reads first: will it stay up, composed from the SLAs of what you chose, and what will it cost to run, over the five years the case scores at 25%. The Task carries both. The last three sections turn the calculations into practice: FinOps keeps the bill near the forecast, an observability platform proves the availability you composed, and the providers' Well-Architected frameworks give you a checklist to review the whole design against. Data stores, messaging, NoSQL and search are in Module 9, part 2.
Virtualization, containers, orchestration
A reasonable order in which to learn this domain:
- Virtualization — concepts and types
- Containers — VMs vs containers, Docker
- Orchestration — scheduling, service discovery, rolling updates; in practice Kubernetes
- Non-functional capabilities of the resulting platform
Three implementations of virtualization differ by how they intercept the guest OS: full (software-emulation) virtualization, where the hypervisor traps and emulates every privileged instruction so the guest OS runs unmodified; paravirtualization, where the guest OS is itself modified to cooperate with the hypervisor via hypercalls instead of trapped instructions, trading portability for lower overhead; and hardware-assisted virtualization, where CPU extensions such as Intel VT-x and AMD-V let the hypervisor run guest instructions natively. This last variant is what made virtualization efficient enough to become the cloud's substrate.
A virtual machine is a software emulation of a computer that runs in an isolated partition of a real machine, complete with its own guest operating system, whereas a container is essentially a sandbox for a single process that shares the host kernel instead of emulating hardware — the reason containers start and stop in a fraction of a second while VMs must boot a full OS. Docker itself resists a one-line definition: it functions at once as an image format, a container runtime, an image-build system, a remote management API, a log-collection daemon, and an orchestrator, which is why practitioners often reduce it to "a wonderful mechanism for shipping tarballs."
Containers share one host OS and kernel; each VM duplicates a full guest OS — the source of both the VM's stronger isolation and its slower startup.
The diagram shows a hosted (type 2) hypervisor, which runs as software on a host operating system — the desktop case. Cloud providers run bare-metal (type 1) hypervisors directly on the hardware, with no host OS in between.
Orchestration is what turns a pile of containers into a platform. You declare the desired state — this image, three replicas, these resource limits, this health check — and the orchestrator's control plane keeps reality matching it: it schedules pods (one or more containers) onto nodes with spare capacity, restarts or replaces the ones that fail their health checks, gives each service a stable name (service discovery), and replaces old versions with new ones a few at a time (rolling updates). Kubernetes is the de facto standard, and managed offerings (Amazon EKS, Azure Kubernetes Service, Google Kubernetes Engine) run its control plane for you, but you still plan the version upgrades and own the node pools and policies. For a small team running a modest workload, a managed container service (AWS Fargate, Azure Container Apps, Google Cloud Run) or serverless functions can deliver the same capabilities with far less to operate; choose Kubernetes when you need its control, not by default.
Cloud core
The NIST definition
The NIST definition of cloud computing has been the most accepted and is used as the basic definition. A simplified version: remote computing resources provided as a service. By computing resources we usually mean network, computing units (virtual machines), and storage.
NIST SP 800-145 defines cloud computing through five essential characteristics, three service models (SaaS, PaaS, IaaS) and four deployment models (private, community, public, hybrid). The five essential characteristics:
| Property | Meaning |
|---|---|
| On-demand self-service | A consumer can unilaterally provision computing capabilities as needed, without requiring human interaction with each service provider |
| Broad network access | Capabilities are available over the network and accessed through standard mechanisms that promote use by heterogeneous thin or thick client platforms |
| Resource pooling | The provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, with different physical and virtual resources dynamically assigned and reassigned according to consumer demand |
| Rapid elasticity | Capabilities can be elastically provisioned and released, in some cases automatically |
| Measured service | Resource usage can be monitored, controlled and reported, providing transparency for both the provider and the consumer |
Automated elasticity matters because the actual demand is often not what we can predict — automated elasticity provided by cloud can be far more beneficial than managing scale manually.
Deployment models: public, private, hybrid — and also a community cloud.
OpenStack is an open-source cloud operating system and community founded by Rackspace and NASA in 2010. It is an abstraction over the hypervisor and provides a unified interface to access the various resources — compute, storage (object and block), and network.
Service models
Security and compliance is normally a shared responsibility between the cloud provider and its customers — for example, the AWS Shared Responsibility Model describes such segregation.
| Model | Definition |
|---|---|
| IaaS | Gartner: a standardized, highly automated offering where compute resources, complemented by storage and networking capabilities, are owned by a service provider and offered to the customer on demand. The resources are scalable and elastic in near real time and metered by use. Self-service interfaces are exposed directly to the customer, including a web-based UI and an API. The resources may be single-tenant or multitenant, and hosted by the service provider or on-premises in the customer's data center — the public/private cloud distinction |
| PaaS | NIST: the consumer deploys its own applications onto provider-managed languages, libraries, services and tools, controlling the application and its configuration but not the underlying servers, operating systems or storage. Gartner now evaluates IaaS and PaaS providers together, in its Strategic Cloud Platform Services Magic Quadrant |
| hpaPaaS | Gartner's former name for a special subset of PaaS: high-productivity application platform as a service — the category Gartner now calls enterprise low-code application platforms (LCAP). These provide services for declarative, model-driven application design and development, and simplified one-button deployments. They typically create metadata and interpret it at runtime; many allow optional procedural programming extensions. The underlying infrastructure is opaque to the user — they do not deal with servers or containers directly. The rapid application development features are often referred to as "low code" and "no code" support. hpaPaaS solutions provide: UI capabilities via responsive web and mobile apps; orchestration or choreography of pages, business processes, and decisions or business rules; a built-in database; and one-button deployment |
| SaaS | NIST: the consumer uses the provider's applications running on cloud infrastructure, managing at most user-specific configuration. For the SaaS builder, one of the main architectural problems to solve is tenant isolation — on different layers — for multi-tenant systems |
| Serverless | Not a NIST service model: the provider runs code or containers on demand per request or event, scales to zero and bills per invocation and duration. The usual reference point is the Serverless Compute Manifesto; weigh it against VMs and containers on cold-start latency, execution limits and cost at steady load |
Cloud service taxonomy
IaaS services
| Group | Typical services |
|---|---|
| Networking | Virtual networks; internet gateway; network ACLs; security groups; load balancing; DNS; CDN |
| Storage | File & block storage; object storage; performance and access tiers |
| Compute | Instance characteristics; instance types; compute features |
PaaS services
| Group | Typical services |
|---|---|
| Application | aPaaS; API management; queues |
| Database | Relational databases; NoSQL; cache |
| Analytics | Managed data warehouses, data-processing and streaming services — see Data and analytics in part 2 |
On API management vs an API gateway — the distinction matters:
An API gateway is a runtime component: it receives requests and routes, secures and throttles them on their way to backend services. API management is the wider discipline around it — the entire lifecycle of an API (design, publication, versioning, retirement), a developer portal and onboarding, and analytics that answer business questions such as who your heaviest consumers are and how traffic is trending — with tools for all stakeholders, not just the request path.
Architecting for the cloud
Disposable resources and immutable infrastructure
With fully automated deployment methods you can replace old components with new versions to ensure your systems maintain their initial "known-good" state. Managing a fleet of instances becomes much simpler with immutable infrastructure, since there's no need to track the changes that would occur.
With immutable infrastructure you know what's running and how it behaves. Deploying updates can become routine and continuous, with fewer failures occurring in production, and all changes are tracked by your source control and CI/CD processes.
The four rules:
| Rule | Rationale |
|---|---|
| Don't modify instances in place | This is the core pattern in immutable infrastructure. Modifying server instances is difficult to successfully automate and track, and allowing it will result in manual modifications. While in theory it is possible to have policies and tools in place to keep things consistent, in practice both fail with some regularity. There is no concise and trustworthy way to know if and where you have configuration drift |
| Replace instances to update them | If nothing is changed in situ, then to introduce change we must be able to deploy new instances without affecting the customer experience. While this can be done manually it rarely works well, as humans are bad at following rote procedures consistently. Plan to automate instance replacement |
| Plan for instance and zone failures at all times | Instance failure should be a normal part of doing business — whether unintended due to bugs or other issues, or intentional in order to perform component replacements. Zone failures on AWS, Azure and Google Cloud should also be relatively painless if you architect well — spread instances across availability zones (on Azure, zone-redundant services or zone-spanning VM Scale Sets; an Azure Availability Set only spreads VMs across racks inside one datacenter) |
| Don't let instances get stale | Not unlike a computer, the longer an instance has been in use, the greater the chance it will have drifted from the optimal configuration. Don't run server instances for a long time |
Pets vs cattle: are you operating in the type of environment that, if one server crashes, everything goes down — a "Pet" — or are you in a scenario in which the loss of a server means that nothing happens, because the "herd" still exists and performs just as before — "Cattle"?
Availability itself has a computable definition worth carrying into an ASR. For components chained with no redundancy, overall availability is the product of each component's availability — e.g. 99.5% × 99.9% × 99.9% ≈ 99.3%, so single points of failure compound quickly. For components deployed redundantly in parallel, availability instead becomes A₁ + (1 − A₁) × A₂, which is the same as 1 − (1 − A₁)(1 − A₂) — e.g. two components each at 99.5% combine to roughly 99.9975% — the quantitative case for why redundancy, not just component quality, is what buys the "nines" of uptime. This is also why zone failures need a redundancy plan rather than hoping a single datacenter is enough.
Two caveats. The parallel formula assumes the copies fail independently and that failover is instant; copies in one zone, or sharing a deployment pipeline, fail together, and the load balancer, DNS and any third-party dependency in the request path still sit in series with the redundant pair. And a provider's SLA is a contractual credit threshold, not a measurement of the availability you will engineer — use it as an input to the calculation, not as the result. The nines table in Module 3, part 2 translates each figure into downtime per year and month.
Automation
As failures are expected and accepted, we should design our architecture with this in mind. In order to recover quickly from a failure we have to react quickly. Another motivation to move fast in the cloud is to be able to respond quickly to any new requirement or change in the market. Fortunately several tools and strategies support these notions:
| Capability | Meaning |
|---|---|
| Auto recovery | When something goes down it should be noticed and recreated in an automated way |
| Auto scaling | Automatically scale up or down to the actual load and calling pattern |
| Observability | Our system must be fully interrogable at every time — observability is a very fundamental requirement |
| Easy experimentation | To support faster time-to-market and respond to market changes quickly, we should make experimentation easier — using frameworks that allow quick application development and deployment while meeting the requirements above |
When the destination is decided, the move itself is usually planned against the Rs of cloud migration — AWS currently lists seven: retain (keep it on-premises, revisit later), retire (decommission the application and its host, migrating nothing), rehost (lift-and-shift onto IaaS with minimal effort), relocate (move a virtualised estate as-is, e.g. VMware workloads, to the provider's equivalent hosted platform), replatform (some application changes, such as moving the database to a hosted DB service), repurchase (replace the application with a SaaS product), and refactor or re-architect (rework the application to use cloud-native services). Other sources group and name them differently, so cite the version you use. Which R applies to a given application is itself an architectural call, driven by financial, security, business, technical, and compliance assessments of that specific workload.
Cascading failures
In interservice communication, one slow application instance without proper isolation can easily slow down the entire system.
A memorable analogy for amplification: a mailing list where somebody asked something; after a point most people weren't interested in the discussion, and somebody sent an unsubscribe request to the list — everybody started to do the same, creating a huge message flood and making the situation more serious.
Many different techniques help here — see the bulkhead and circuit breaker patterns in Module 4.
Loose coupling and application-level resilience
Interservice communication should be isolated, and resilience is implemented at the application level with a specific toolkit: bulkheading, proper timeouts — timeout deadlines matter — circuit breaking, and async libraries. These are the Module 4 patterns arriving as cloud operational necessities rather than optional refinements.
Cloud Native — and what it is not
Cloud Native is a commonly misunderstood term. The point is that Cloud Native is not about cloud and leveraging the cloud.
The term refers to software built for change, scale, resilience and manageability. It is often equated with microservices and containers, but those aren't required. Whether running in public or private clouds, a cloud-native app takes advantage of the elasticity and automation offered by the host platform.
The cloud-native approach is far more about "how" than it is about "where."
The right question is: what capabilities and practices make it economical to work in small batches, test hypotheses, and learn? The answer is capabilities like cheap application builds, safe deployments, automated management of deployments (start/stop/scale/health checking), security, distributed configuration, and traffic routing. Containers play a foundational role in implementing these capabilities and form the basis of the platforms that provide them — but the capabilities are what are important, not the containers.
The 12-factor principles that matter most here
The 12-factor methodology provides factors applicable in any language to software delivered as a service. Three of them carry most of the weight for cloud design:
| Principle | Why it matters |
|---|---|
| Stateless and share-nothing processes | Data — whether for state or other reasons — limits the ability to distribute or easily scale a process. Never design a process that expects a set of data to be persisted locally. All data should be persisted outside the process using an attached service |
| Dependencies explicitly declared and isolated | In practice this means the application is environment agnostic and doesn't expect some tool or library to exist outside what is explicitly declared |
| Config distinct from code | Anything distinct to an execution environment — development versus production credentials, connection strings, unique hostnames — kept separate from the code that uses it. The twelve-factor app stores config in environment variables, not in config files checked in alongside the code; in current practice secrets such as credentials go one step further, into a secrets manager that injects them at runtime |
Why containers, specifically
Containers are increasingly popular because once an application is containerised it is easy to package, ship, and run anywhere. Unlike traditional virtualization, containers provide lightweight virtualization at the application level, and are therefore much better at squeezing every last ounce of available capacity.
The operationally decisive property: containers can start and stop at sub-second speeds, which makes it easy both to scale out and to recover from failures — simply by starting up new containers.
Hardware selection and sizing
Primary resources are the typical resources that matter for most types of solutions. Additional resources are important for special scenarios:
- GPU grids are often used for video encoding and for deep learning (neural networks). There are cases where neural network training performance on 1 GPU node (AWS P2) was the same as on a cluster with 144 CPUs — by replacing CPU nodes with GPU nodes the solution became 20× more cost effective
- Disk size is important for large DBs, NoSQL and big data solutions
- Physical size/dimensions and energy consumption are critical in IoT and wearable devices
Normally, software performance is bound to one or more primary hardware resources. Knowing which types of workloads and solutions are CPU-bound or memory-bound helps make initial estimations or pick starting points for performance testing.
CPU
| Concept | Detail |
|---|---|
| CPU bound | Improving the CPU — faster, more cores, better cache — will improve application performance; the application spends the majority of its time using the CPU doing calculations |
| I/O waits | The application is not using CPU and is waiting for I/O — reading from disk, making a request over the network to a DB or service |
| Cores vs threads | If your application doesn't have any I/O waits, which is rare, optimal performance is when the number of threads equals the number of cores |
| Context switching | Having more running threads than cores leads to interrupting one task and switching to another, which adds CPU overhead. With no I/O wait, context switching degrades performance. When there is I/O wait it is totally acceptable and common practice to have more threads than cores |
| Metrics | CPU usage and load average are the typical metrics. CPU utilization should normally be below 70–80%, and load average — tasks for CPU waiting in queue — below 0.7 per core |
Memory
Memory on the node is consumed not only by your application — there are other consumers such as the kernel, page cache, daemons and agents. It's not uncommon to leave extra free memory unallocated to leave more space for page cache, especially in disk-I/O-heavy applications like databases and NoSQL data stores.
- Adding more memory is not the right way to solve memory leaks — address it in your application instead
- Swap can extend memory with disk space but slows performance when used; therefore in cases where high performance is required, swap should be turned off
- The Linux kernel OOM Killer will kill your process if it considers it too aggressive in memory consumption. Ensure proper hardware and app allocated memory sizing, and keep memory utilization below 70–80%
- Garbage collection performance — allocating too much memory to a process using a GC-backed runtime (JVM, CLR, JavaScript, Go) can and normally will impact performance. Ideally keep nodes smaller and spread memory across the cluster. Depending on the runtime you may want to keep RAM allocated below 8–16 GB, otherwise investment in GC tuning may be required
Disk
| Metric | Definition | Most critical for |
|---|---|---|
| IOPS | Input/output operations per second — the number of IO operations a disk can perform per second, disregarding the size of each operation | Operational storages based on RDBMS or NoSQL |
| Throughput | IOPS × IO size — the volume of IO operations handled per unit of time (GB/s) | Big data and analytical solutions |
| Latency | How fast a single IO operation can be performed (ns, ms) | Low latency systems |
Different storage is optimized for different types of performance, so the choice depends on the quality attributes of the system.
- SSD vs HDD — SSD often provides better performance, especially latency on random reads/writes. The difference on sequential access patterns is less significant. The primary drawback of SSD is cost
- Local attached disks vs network storage (SAN, NAS) — network storage provides a lot of benefits especially from a maintenance and reliability perspective, and in most cases better TCO. At the same time, using network storage removes "data locality", which is critical for data-intensive applications, increasing disk operation latencies and network I/O
- Access patterns — random vs sequential, read-intensive vs write-intensive vs mixed. Disks have different effectiveness with different access patterns (see the HDD vs SSD example), so understand your solution's access patterns to make a proper disk selection
- Security — data security at rest is critical for most solutions. While software encryption is often sufficient, some solutions require advanced security either forced by regulations like FIPS or by protecting sensitive information. In those cases, disks with hardware-based security may need to be considered — with support of SED and ISE
Network
Network is a huge topic deserving its own course; these are the key points covering the needs of most solutions.
| Metric | Definition | Critical for |
|---|---|---|
| Bandwidth | The max rate of data transferred (MB/s) | — |
| Throughput | The actual data transfer rate | Large-volume data transfer solutions: big data, video streaming, some ML scenarios like neural networks on distributed GPU grids |
| Latency | The delay between sender and receiver first byte | Low latency solutions |
Reliability — jitter, error rate, network partition — has significant impact on certain types of solutions:
- Networks with high jitter are not suitable for low-latency solutions: despite "advertised" network performance that might provide sufficient latency in theory, high jitter will cause it to regularly breach SLAs
- Network partition rate is critical for data stores that are on the CP side of CAP but at the same time try to maintain high availability. Google Cloud Spanner is a CP data storage that offers a 99.999% availability SLA in its multi-region configurations (99.99% for regional ones), made possible by the high quality of Google's network, where network partitions are extremely rare
Software-defined networking (SDN) built on top of network hardware topology can have huge impact on performance and reliability. For example, Google's Andromeda SDN upgrade from version 2.0 to 2.1 reduced network latency for intra-zone VM communication by 40% while using the same hardware.
Security — network security is critical for most modern solutions, and this is one of the biggest challenges in the IoT field. While for most solutions generic cloud network hardware security plus correct network topology (DMZ, VPCs) plus software security (encryption) is enough, additional hardware options exist such as Hardware Security Modules (HSM).
Sizing for availability and geography
Multi-region requirements are driven not only by availability and DR SLA but by the geographical distribution of consumers. At the same time the number of availability zones is purely availability related.
Going below the minimal number of nodes will breach the availability SLA — therefore if that number of nodes provides more capacity than your application needs, consider downsizing the nodes rather than reducing their count.
Autoscaling delay
There is a delay. You normally don't want to increase the number of instances in a cluster the moment utilization reaches a threshold (e.g. CPU > 80%), as it could be an occasional spike. The common approach is to set autoscaling to kick in after the threshold was passed for a few sequential minutes. Bear in mind that once new nodes are added, there is extra start-up time for your app, ranging from milliseconds to minutes depending on its type.
Autoscaling delay ≈ metric collection and evaluation window (including the minutes the threshold must hold) + instance provisioning and boot time + application start-up time + health-check registration before the load balancer sends traffic to the new node.
In some cases load shedding, rate limiting or queue-based load levelling is required to ensure your app can survive big spikes until autoscaling kicks in. With that said, delay in autoscaling is not always suitable for low-latency applications, as rejecting, throttling or queueing requests breaks latency SLAs — those scenarios require preemptive scaling.
For distributed datastores that leverage sharding or partitioning, autoscaling is rarely used, because changing the topology of a partitioned cluster often requires re-partitioning before a node can start serving requests. On large volumes of data this may take hours, and the load spike can be over by then. At the same time, for non-partitioned data stores, creating additional read replicas is a much faster process and can be considered for autoscaling.
Sizing tactics
Estimation tactics are usually employed in the initial stages of the project, when it is not possible to test whether a hardware configuration will correspond to the solution's quality attributes. Often used as a starting point for the first performance testing.
| Tactic | Example |
|---|---|
| Similar systems | "We had a previous version of our service running on 15 c4.large nodes; as we didn't make any significant changes that might affect performance, we will start with the same cluster" |
| Previous experience | Examples of CPU/RAM/disk/network-bound systems that give you an idea of what type of resource would be most needed for your solution |
| Calculation based | Knowing the number of rows in your DB and their size, you can estimate the total data volume. Knowing the size of request/response and max connections to your service, you can estimate the RAM required |
| Vendor recommendations | Unless it's a fully custom-built app, most software products (DBs, NoSQL) have recommended hardware requirements. Remember it is not possible to provide recommendations for every possible use case — never trust them as-is |
| Zooming / proportional testing | If developing a bank system, measure performance on 1K transactions per 100 users, then propagate the results to the ASRs accordingly |
Fact-based tactics are based on monitoring of a real implementation as close as possible to production, often peak, loads.
| Tactic | Example |
|---|---|
| Current system | Whether it is a modernization effort or a cloud migration, whenever there is an existing system, collect as many metrics as possible. Not only may some hardware requirements stay the same, it will also help with estimations |
| Performance testing | Critical for hardware sizing — make sure you test for peak loads as well |
| Shadow traffic | If replacing an existing implementation, before releasing to production it is not uncommon to route production traffic to it so traffic goes to both old and new solutions, while the old one still serves production responses. Not a replacement for performance testing, as it might not show behaviour under peak load — but the benefit is that it is real traffic, as opposed to the often synthetic tests used in performance testing |
| Production monitoring | Once the system is in production, monitor it closely to see if you under- or over-provisioned hardware resources, and adjust accordingly |
| Cloud clone | Use cloud infrastructure to test a full clone of the system — rent cloud capacity for a short period on a separate account, monitor performance and VM loads, then estimate the resources actually used |
| "The best one minus one" | A top-level CIO tactic: when customers go for hardware they are OK to pay for good enough, but not top-notch systems. So select the line-up above the average — not Intel Core i9, but i7 |
Which tactics you apply depends on what information you have or can get. If you have an existing system and are replacing it, you can use the same RPS adjusted with usage growth projections and other factors. If this is a completely new system you'll need to get creative and estimate the number of concurrent users and their behaviour. If it is a service consumed by other existing services, you can look at their RPS.
Lumen is a new system, so there is no RPS to copy: apply the calculation-based tactic to the case's Appendix B. The busiest hour brings 14,000 × 12% = 1,680 referrals (≈0.47 a second) and, at an assumed four searches per referral, about 6,720 availability searches (≈1.9 a second). Little's law gives ≈1,650 concurrent sessions today and ≈2,650 in year 3 — the derivation is in Module 3. The live data is 6.5M records × ~4 KB with indexes ≈ 26 GB, plus ≈ 19 GB of stage history (2.1M changes a month × 18 months × ~0.5 KB, assumed) — roughly 45 GB. None of this needs more than the smallest production node classes. Lumen's sizing is decided by availability and residency — how many zones, how many regional copies — not by capacity: the "sizing for availability and geography" rule above, applied literally.
Most distributed data platforms publish minimum and recommended production cluster specifications — node class, memory, disk type and node count. Use them as the starting point for performance testing and adjust from measurement, never as the answer.
Worked performance-testing iteration
A simple microservice that just serves GET requests. Expected peak load 3,000 requests per second; SLA: response latency percentile 95 shouldn't exceed 20 milliseconds.
| Step | Approach | Outcome |
|---|---|---|
| 1. Start small | Start with a small hardware sizing you believe makes sense based on experience or an estimation-based technique. Start with small loads too and increase gradually to see where the system breaks or slows down | The target 3k RPS was not reached, CPU utilization was high, and the latency SLA was breached |
| 2. Interpolate | From the observations CPU was obviously the bottleneck, so increase both the number of CPUs per node and the number of nodes. Also, while memory wasn't bottlenecked it constantly held at 80% utilization, close to max, so add memory as well | This setup reached the required 3k RPS without issues. But CPU utilization at peak was 22% — it looks like we over-provisioned |
| 3. Optimize | Reduce the size of the cluster, since we over-provisioned | Under-provisioned this time, but not as badly as on the first run. Pretty close |
| 4. Final result | Increase cluster size one more time | Found the happy medium: CPU utilization at 53% under peak load, which is safe even if the workload is slightly higher than expected peaks |
Usually performance testing takes much more iterative steps than in this example, especially where your service is backed by a data storage where you have impact on hardware sizing as well — in that case you have much more variability of parameters to test.
Web and mobile
Choosing a web platform
Platform choice is best framed as a decision tree, and these are the questions worth asking:
| If the requirement is… | The answer tends to be |
|---|---|
| Just a simple website with static HTML pages, fast page configuration and customisation | Static site generators — Hugo, Jekyll, Astro, or a framework's static export (e.g. Next.js) |
| Content managed centrally but delivered to multiple client channels / different web apps | Headless CMS — Strapi, Contentful, Drupal headless |
| A conventional content-managed site | Traditional CMS — WordPress, Drupal, Joomla |
| Behaviour driven by configuration files and a custom configuration system | Dynamic websites with a custom configuration system |
Two selection criteria deserve explicit attention, because they are architectural rather than cosmetic:
- Why security matters in the choice. Different site builders and CMS platforms provide different levels of security out of the box. Judge a platform on its attack surface and patch discipline rather than its reputation: how large and well-vetted the plugin ecosystem is, how quickly core and plugin fixes ship, and whether you can keep up with them. WordPress, for example, is attacked far more often than most CMSs — largely because of its market share and its plugin ecosystem, where most of its reported vulnerabilities sit
- Why performance matters in the choice. Different builders are built on platforms and frameworks with different performance characteristics. Some provide optimisation techniques out of the box — GZIP compression, caching. And simple static site generators are much faster than the alternatives
That tree is for content sites. Lumen's three portals are something else: authenticated, transactional applications whose pages are mostly per-user data — a worklist, an offer, a booking. They are built with an application framework (React, Angular or Vue, often server-rendered through Next.js or similar) over the platform's APIs, and a CMS at most feeds their help text. The tree applies to Lumen's public website, not to the portals.
Micro frontends
The micro frontends approach is a great technique to divide the workload in a multi-team setup and isolate code.
The trigger condition is a frontend that has become too big to be supported as one unit, or teams that must ship independently. It is the Module 4 microservices argument — Conway's Law included — applied above the API boundary rather than below it. Cam Jackson's article Micro Frontends on martinfowler.com is the reference.
Choosing a mobile approach
| Approach | Definition | Fits when |
|---|---|---|
| Native | A mobile application created for one platform | Games, graphical apps, apps needing deep device capability |
| Cross-platform | The same codebase adapted to run on different platforms | Broad reach with shared logic |
| PWA | A progressive web application which has an icon on the mobile desktop and can provide other device-like capabilities | Only mobile web reach needed |
| Hybrid | An application that runs as a web application and can be wrapped as a native one | Reuse of web assets with store distribution |
For Lumen the deciding facts are that 45% of patient sessions are already on phones, a patient opens the portal about three times per referral, and nothing in the requirements needs device hardware. A responsive web portal, optionally installable as a PWA, meets QA04 without an app-store release cycle; a native app would need a requirement the case does not contain.
SEO as an architectural concern
SEO means the process of optimising a web application so search engines can display it at the top of search results. For private application parts — e.g. the admin section — SEO should not be available.
That second sentence is the architectural instruction: indexability is a per-area decision, not a site-wide one, and private areas must be excluded deliberately. But keeping a page out of search results is hygiene, not protection: noindex and robots.txt are requests that well-behaved crawlers honour, and robots.txt even advertises the paths it lists. Private areas — an admin section, or anything showing patient data — must sit behind authentication and authorisation; excluding them from indexing comes on top of that.
The web performance metrics worth putting into a performance ASR for a front end are Google's Core Web Vitals, measured at the 75th percentile of real user visits: Largest Contentful Paint (LCP) ≤ 2.5 s for loading, Interaction to Next Paint (INP) ≤ 200 ms for responsiveness, and Cumulative Layout Shift (CLS) ≤ 0.1 for visual stability. First Contentful Paint and Time to First Byte remain useful diagnostics behind them. Older metrics you will still meet in documents — First Meaningful Paint, First CPU Idle, Time to Interactive, First Input Delay — have been retired; INP replaced First Input Delay as a Core Web Vital in March 2024.
A second, complementary model worth knowing by name is Google's RAIL model, which frames performance around four user-facing phases rather than page-load timings alone: Response (a visible response to input within 100 ms, so handle events within about 50 ms), Animation (produce each frame within about 10 ms, leaving the browser the rest of the 16 ms frame budget), Idle (non-critical work chunked into blocks of 50 ms or less so it never blocks input), and Load (content delivered and interactive within 5 seconds on a first visit from a mid-range phone on a slow network, and within 2 seconds on repeat visits). Where the load metrics above describe a single startup event, RAIL describes the steady-state responsiveness a user experiences throughout a session.
Cache
High availability for a cache means the ability to continue operating despite the failure of members of the cluster. Applied to distributed caching, HA means uninterrupted, consistent data access. That uninterrupted access is a function of cache topology, not just clustering. An embedded (in-process) cache lives inside the application's own process (for Java, its JVM) — on-heap, bounded by that process's memory, or off-heap, bounded by the server's RAM — and gives the lowest latency but no sharing across instances; a distributed cache runs as its own cluster reachable by many application nodes, and within it data is either replicated (every node holds a full copy, so reads stay local but every write fans out everywhere) or partitioned (each entry lives on one primary node plus backups, trading a network hop for far greater aggregate capacity).
Distributed caches that provide this HA guarantee are commonly implemented as In-Memory Data Grids (IMDGs) — products such as Hazelcast, Apache Ignite and Oracle Coherence fall into this category, while the most widely deployed distributed caches today are Redis and its open-source fork Valkey, usually consumed as a managed cloud service. Beyond plain key-value storage, mature IMDGs add transactional ACID support, typically via a two-phase commit (2PC) protocol, and rely on concurrency mechanisms such as MVCC (multi-version concurrency control) to keep locking overhead low while still guaranteeing consistency across the cluster.
Eviction algorithms
The choice of eviction policy is an architectural decision driven by the access pattern, and getting it backwards is a classic mistake. Eviction is not the only way an entry leaves the cache: time-to-live (TTL) expires it a fixed duration after it was written regardless of use, while time-to-idle (TTI) expires it after a period with no access; the two settings are independent and are normally combined with an eviction algorithm rather than substituted for one.
| Algorithm | Behaviour | Cost and best fit |
|---|---|---|
| LRU — Least Recently Used | Discards the least recently used items first | Requires keeping track of what was used when. In hardware CPU caches that bookkeeping is costly, which is why processors use approximations. In software an exact LRU is cheap — a hash map plus a doubly linked list gives O(1) lookup and update — and some stores (Redis among them) approximate it by sampling to save memory. LRU is actually a family — members include 2Q (Johnson and Shasha) and LRU/K (O'Neil, O'Neil and Weikum) |
| LFU — Least Frequently Used | Counts how often an item is needed; those used least often are discarded | Suits stable popularity distributions |
| MRU — Most Recently Used | Discards, in contrast to LRU, the most recently used items first | Chou and DeWitt (VLDB, 1985) found MRU to be the best replacement policy when a file is scanned repeatedly in a looping sequential pattern, and later work (VLDB, 1996) reported the same advantage for other cyclic access patterns, such as repeated scans over datasets larger than the cache: by holding on to older entries, MRU keeps the ones the next pass will need. It fits workloads where an item's chance of reuse grows with its age |
| Random replacement | Discards a randomly chosen item when space is needed | Needs no record of access history at all, which is why some processors, ARM designs among them, have used it in hardware caches |
The MRU result is the one worth remembering, because it is counter-intuitive: under a looping sequential scan, LRU evicts exactly the pages you are about to need again, and MRU wins. Cache policy must follow the access pattern, not habit.
Eviction policy is only one lever; the other is how the cache stays synchronised with the system of record, which splits into synchronous and asynchronous persistence integration. Synchronously, read-through and write-through route every read or write through the cache so the client only ever talks to the cache; asynchronously, write-behind queues writes and flushes them later for lower write latency, while refresh-ahead proactively repopulates an entry before it would expire.
Read-through and write-through both put the cache between the client and storage — the application never talks to storage directly. The more common alternative, cache-aside, reverses this: the application reads the cache, falls back to storage on a miss, and populates or invalidates the cache itself.
Worked cloud cost model
This is the method the Task asks you to apply: derive volumes from business counts, then apply the price list. Never the reverse — a spreadsheet that starts from prices tells you nothing about what to change. Two shortcuts to avoid. Multiplying peak concurrency by the 730 hours of a month prices every hour as if it were the busiest one. Multiplying each record change by every workflow stage counts one stage change several times. Count each business event once, at its source.
Every price below is an illustrative 2026 list price for one region of a large public cloud — an assumption, not a quote. Substitute your provider's current rates.
Step 1 — the counts
| Count | Monthly value | Derivation |
|---|---|---|
| Studies | 350,000 | 4.2M a year ÷ 12 |
| Notifying stage changes | 2.1M | 350,000 studies × 6, each counted once |
| Messages | 2.1M emails + 2.1M SMS | The case asks for both on every notifying change |
| Patient portal sessions | 630,000 | 350,000 referrals × 60% using the portal × 3 sessions |
| Staff and partner user-days | 69,750 | 2,790 named users × 25 operating days (300 ÷ 12) — generous, as it assumes every user works every operating day |
| Availability searches | 1.4M | 14,000 referrals × 4 searches (assumed) × 25 days |
| Referral extract runs | 2,880 | One every 15 minutes: 96 a day × 30 |
| Live records | 6.5M | Appendix B; ≈ 18 months × 350,000 studies |
Step 2 — derive call volumes, showing the arithmetic
| Flow | Monthly calls | Derivation |
|---|---|---|
| Patient portal API requests | 12.6M | 630,000 sessions × 20 requests a session (assumed) |
| Staff and partner portal API requests | 13.95M | 69,750 user-days × 200 requests a day (assumed) |
| External requests — through the API gateway | 26.55M | The two lines above. The 1.4M searches are inside them, not added again |
| Stage-change handling | 2.1M | One invocation per notifying stage change: it updates the search index, writes the audit record and queues the two messages |
| Notification dispatch | 4.2M | One invocation per message sent |
| Referral sync | 0.42M | 2,880 extract runs + 420,000 changed referrals (350,000 new + an assumed 20% amended or cancelled) |
| Partner availability | 0.02M | 8,640 polls of the automated partner group's API (every 5 minutes, assumed) + 15,000 manual slot uploads (600 practices, about one each per operating day, assumed) |
| Function invocations | ≈ 33.3M | 26.55M external + 6.75M internal |
Function size, assumed: 512 MB of memory and a 200 ms average duration, so 0.1 GB-second per invocation — an API handler that makes one or two database calls. Calls to the database, cache and partner API happen inside these invocations and are not billed as invocations of their own.
Step 3 — apply the price list
Two kinds of line behave differently. Volume-driven lines scale with the counts and do not care how many deployments serve them. Platform lines are paid once per regional stack, whatever the traffic.
The platform is sized from the derivations, not from habit. Each regional stack holds about a quarter of the ≈45 GB of live data, so a 2 vCPU / 16 GB database instance keeps a region's working set in memory. The search index holds bookable slots eight weeks ahead at 1.5× demand — 14,000 × 1.5 × 46 operating days (eight weeks at ~300 operating days a year) ≈ 966,000 slot documents at ~2 KB, under 2 GB for the whole estate (assumed).
| Item | Basis (illustrative 2026 list price) | Units | Monthly |
|---|---|---|---|
| API gateway — external requests only | $3.50 per 1M | 26.55M | $92.93 |
| Functions — invocations | $0.20 per 1M | 33.3M | $6.66 |
| Functions — duration | $0.0000166667 per GB-second | 33.3M × 0.1 = 3.33M GB-s | $55.50 |
| $2.00 per 100,000 | 2.1M | $42.00 | |
| SMS | $6.00 per 1,000 | 2.1M | $12,600.00 |
| Data egress to the internet | $0.09 per GB | ≈ 1,030 GB: 26.55M responses × ~15 KB, plus 630,000 patient sessions × ~1 MB of front-end assets | $92.54 |
| Logs and audit trail | $0.50 per GB ingested; $0.03 per GB-month stored | ≈ 67 GB a month (~2 KB per invocation); 12 months kept ≈ 800 GB | $57.28 |
| Reporting store | Columnar files in object storage at $0.023 per GB-month; serverless SQL at $5 per TB scanned | 50 GB; ~2 TB scanned (assumed) | $11.15 |
| Volume-driven subtotal | $12,958.06 | ||
| Relational database — primary plus a synchronous standby in a second zone | $0.25 per instance-hour (2 vCPU, 16 GB) | 2 × 730 h | $365.00 |
| Read replica — operational reports and read-heavy queries | $0.25 per instance-hour | 730 h | $182.50 |
| Database storage | $0.115 per GB-month | 3 copies × 50 GB | $17.25 |
| Backups — 35 days of automated backups | $0.095 per GB-month | ≈ 100 GB | $9.50 |
| Search cluster — three nodes, one per zone, with storage | $0.22 per node-hour; $0.11 per GB-month | 3 × 730 h; 60 GB | $488.40 |
| Observability floor — metrics, traces, dashboards, alerts | Flat (assumed) | One stack | $150.00 |
| Platform, one regional stack | $1,212.65 | ||
| Per-region deployment — four regional stacks | 4 × $1,212.65 (assumed: one stack per region, the cautious reading of "comply … in each region of operation") | 4 | $4,850.60 |
| Production | Volume-driven + four stacks | $17,808.66 | |
| Non-production — development, test, staging | 30% of production, excluding email and SMS, which go to sandboxes in test (assumed) | 30% × $5,166.66 | $1,550.00 |
| Total | ≈ $19,360 a month |
Step 4 — project five years
The case scores total cost over five years at 25%, so the bid needs a five-year figure, not a monthly one. Grow the volume-driven lines by the case's 12% a year. Hold the platform flat: the small nodes chosen above absorb ×1.57 growth. Recompute non-production each year at 30% of production's non-messaging lines. Year 1 is priced at Appendix B volumes; because go-live is 12 months after signature, a stricter model would start year 1 at ×1.12.
| Year | Growth | Volume-driven | Platform, four stacks | Non-production | Monthly | Year total |
|---|---|---|---|---|---|---|
| 1 | ×1.00 | $12,958 | $4,851 | $1,550 | $19,359 | $232,304 |
| 2 | ×1.12 | $14,513 | $4,851 | $1,561 | $20,925 | $251,100 |
| 3 | ×1.25 | $16,255 | $4,851 | $1,574 | $22,679 | $272,152 |
| 4 | ×1.40 | $18,205 | $4,851 | $1,588 | $24,644 | $295,730 |
| 5 | ×1.57 | $20,390 | $4,851 | $1,604 | $26,845 | $322,137 |
| Five years | ≈ $1,373,400 |
Still outside the model. The two new regions and any disaster-recovery copy — both are yours to add in the Task. The provider's support plan, often priced as a percentage of the bill. Web application firewall, private networking and key management. The environments that run during the 12-month build. And people: in Module 7's TCO structure, maintenance is the dominant term, and none of it is here. This is run cost, not total cost of ownership.
What to take from this
Four lessons, all of them architectural rather than financial.
SMS is half the messages and two-thirds of the bill. 2.1M emails cost $42; the same number of SMS messages cost $12,600 — 65% of the monthly total, and ≈ $960,000 (70%) of the five-year figure. Unit economics beat volume intuition. That single line justifies a design conversation with Lumen, not only inside the design team. The RFP requires email and SMS on every stage change, so sending SMS only where a missed message costs an appointment (say Offered and Confirmed), or letting patients opt out, is a requirement change Lumen must agree to. Two SMS stages instead of six saves $8,400 a month — ≈ $640,000 over five years, more than every other line in the model combined.
Per-region deployment multiplies the fixed lines, not the variable ones. Four regional stacks cost $4,851 a month. One shared stack — database instances twice the size (4 vCPU, 32 GB at $0.50 an hour) and three times the database storage and backups (150 GB per copy) — would cost about $1,814. Reading "comply in each region" as "deploy in each region" therefore costs ≈ $3,040 a month — ≈ $47,000 a year once non-production follows it. Ask it as a clarification question: does any current region require in-region hosting, or only the new jurisdiction? The answer is worth more than any tuning.
The search cluster is sized for availability, not for the index or the traffic. An index under 2 GB and ≈1.9 searches a second at peak both fit the smallest node class. The cluster has three nodes because it spans three zones and must keep a quorum when one fails. That is the sizing rule from earlier in this unit: keep the node count the availability target needs, and shrink the nodes instead.
Every hop is a line here and a factor in the availability product. A separate data-access service between the functions and the database would add an invocation to almost every one of the 33.3M, and a component in series to every request path. It appears in no requirement. The availability budget in Module 3 shows why the cheapest availability decision is usually removing a hop; the cost model says the same thing in dollars.
FinOps
The cost model is a forecast. FinOps is the practice that keeps the real bill close to it after go-live and makes each team accountable for what its design spends. In the FinOps Foundation's framing (a Linux Foundation project; FinOps Framework, revised 2025) it is both a way of operating and a culture: engineers, finance and business owners share responsibility for technology spend, and decide from current data how to get the most value from it. For the architect the consequence is narrower: cost is a quality attribute, and like any other it needs a measure, a target and an owner. Module 3 records Lumen's as QA18 in the worked ASR list: run cost per referral at or below an assumed $0.06 at every volume up to year 3, a deviation alerted within a day, and at least 95% of spend allocated. Module 3, part 2 treats cost as a quality attribute and lists the design tactics; this section is the practice that meets the target month after month.
The framework
The Framework's six principles were reworded in its recent revisions to cover all technology spend, not only public cloud. Each has an architectural reading:
| Principle | What it asks of the design |
|---|---|
| Teams need to collaborate | Engineering, finance and the business set cost targets together; the architect supplies the model that links design choices to the bill |
| Business value drives technology decisions | Judge spend by cost per unit of value, not by the size of the invoice. A bill that grows with referrals is healthy; one that grows faster than referrals is not |
| Everyone takes ownership for their technology usage | Every resource has an owner who sees its cost — possible only if the design makes cost attributable |
| FinOps data should be accessible, timely and accurate | Cost and usage arrive daily and already allocated, not once a month with the invoice |
| FinOps should be enabled centrally | One team owns rates — negotiation and commitments; delivery teams own usage |
| Take advantage of the variable cost model of the cloud | Design to scale down and switch off as readily as to scale up |
Its three phases run as a loop, repeated for each capability as the practice matures:
- Inform — allocate, report, forecast and benchmark: who spends what, and what it buys
- Optimize — find improvements in usage (right-sizing, scheduling, removing waste) and in rates (commitments, negotiated prices)
- Operate — put the improvements into practice and hold them there with budgets, policies, KPIs and ways of working
The Framework groups its capabilities into four domains — understand usage and cost, quantify business value, optimise usage and cost, and manage the FinOps practice — and, since 2025, defines scopes: the segments of technology spend a practice covers, such as public cloud, SaaS and data centre. Scopes matter to Lumen. Its largest line, SMS, may be billed by a messaging provider rather than on the cloud invoice, and a practice that watches only the cloud bill would miss two-thirds of the spend.
Unit economics
The figure that survives every conversation with the business is cost per unit of value. For Lumen the unit is the referral, the event the case already prices: $9 handled manually, $2 automated. Today ≈ $19,360 a month ÷ ≈ 350,000 referrals (14,000 × ~25 operating days) ≈ $0.055 per referral, of which $0.036 is SMS; Module 3, part 2 splits it line by line. Run forward:
| Volume basis | Referrals a month | Run cost a month | Per referral |
|---|---|---|---|
| Appendix B — year 1 of the five-year table | 350,000 | $19,359 | $0.055 |
| Year 3 of the five-year table (×1.25) | ≈ 439,000 | $22,679 | $0.052 |
| QA18's year-3 volume (×1.405), four stacks | ≈ 491,750 | ≈ $24,650 | $0.050 |
| The same, with the two new regions as two more stacks | ≈ 491,750 | ≈ $27,800 | $0.057 |
Two "year 3"s appear because the five-year table starts at Appendix B volumes on go-live, while Module 3 counts three years of growth from today (×1.405), which is year 4 of the table. Say which one you mean.
Three things follow. Scale reaches only the part that is not messaging: SMS stays $0.036 a referral at any volume, so growth alone never takes unit cost much below it, and only the SMS decision does — two SMS stages instead of six makes today's figure ≈ $0.031. QA18's headroom is thin once the topology grows: the two new stacks take year-3 unit cost to $0.057, 5% under the $0.06 ceiling, and the model still leaves out the disaster-recovery copy QA14 needs — the Task's DR row is where QA18 is won or lost. And a per-referral target separates growth from waste: a monthly dollar ceiling would fail as referrals grow, and pass a design that is quietly getting less efficient. Set beside value, $0.055 is under 3% of the $2 an automated referral costs Lumen, so the run-cost argument in the bid is about the five-year score, not about whether automation pays.
Allocation and tagging
Nobody can own a cost they cannot see. Allocation assigns every line of the bill to an owner and a purpose.
- Use the account boundary first. One AWS account, Azure subscription or Google Cloud project per environment and region makes the two splits that matter most — production against non-production, and region by region — exact, with no tag to forget. Non-production is $1,550 of Lumen's $19,360 and the spend most likely to be left running; report it on its own line.
- Then a small mandatory tag set (labels on Google Cloud): environment, region or jurisdiction, component — the Module 5 container — owner and cost centre. Enforce it when resources are created, through policy as code in the deployment pipeline and the providers' policy services (AWS tag policies or Azure Policy, for example), rather than in a monthly clean-up. Track unallocated spend as a metric: QA18 requires at least 95% of spend allocated, and the remainder is where waste hides.
- Write down the rule for shared costs. The gateway and the observability platform serve every region. Split them by a driver the business recognises — referrals per region — and agree it before anyone reads a report.
- Allocate the per-message lines too. SMS and email are billed per message, not per resource. A separate sender configuration per region, or a region attribute on every send that the provider reports back, lets the largest line be split like the rest.
Rates: commitment discounts
Providers discount steady usage in exchange for commitment, in three main forms:
- Reserved capacity — a resource type in a region for one or three years: AWS Reserved Instances (including for Amazon RDS) and Azure Reservations
- Savings plans — an hourly spend commitment across a family of services, more flexible about instance type and region: AWS Savings Plans and the Azure savings plan for compute, and, since December 2025, AWS Database Savings Plans, which are not tied to a region — a one-year commitment covering RDS and Aurora on recent instance families, among other database services
- Committed use discounts — Google Cloud's form, either resource-based (vCPUs and memory in a region) or spend-based (an hourly amount on a service such as Cloud SQL)
The arithmetic is the same for all of them. A commitment at discount d costs (1 − d) of the on-demand price whether you use it or not, so it beats on-demand only if the resource runs for more than (1 − d) of the term.
Worked on the database line. Each regional stack runs three instances — primary, standby and read replica — at $547.50 a month on demand, so four stacks cost $2,190. The discounts below are assumptions for illustration; real rates vary by provider, engine, term and payment option.
| Option | Four stacks, a month | Saving a month | Beats on-demand if it runs for at least |
|---|---|---|---|
| On demand | $2,190 | — | — |
| One-year commitment, 30% off (assumed) | $1,533 | $657 | 70% of the term — 8.4 of 12 months |
| Three-year commitment, 50% off (assumed) | $1,095 | $1,095 | 50% of the term — 18 of 36 months |
The databases run 24×7, so on usage alone the three-year term wins. The catch is the clarification question from the cost model. Reserved capacity is bought per region: if Lumen answers that only the new jurisdiction needs its own stack, the four current stacks become one shared stack on larger instances, and at least three regions' reservations are stranded — $821 a month on the three-year term, until it ends. A region-flexible spend commitment, where offered, avoids stranding at a lower discount: it follows the spend to the consolidated stack, though a larger shared database can still outgrow or undershoot the hourly amount committed. Get the residency answer first. Then commit for one year on the instances certain to run all year, the primary and standby (≈ $438 a month saved at the assumed rate), and leave the read replica and non-production on demand until measured load justifies more.
Module 3, part 2 lists commitments among its cost tactics for the nodes that run all month; the break-even says when that pays, and the residency answer says whether. The search nodes carry a second open question — part 2 of this module asks whether F01 needs a separate cluster at all — so leave them on demand until that is settled too.
Commit where the money is steady and large. The functions line is $62 a month and not worth a commitment. The SMS line is not a cloud commitment at all: its rate is negotiated with the messaging provider, as volume tiers or committed spend.
Usage: right-sizing, scheduling, budgets and anomalies
- Right-sizing matches each resource to measured demand. It needs production metrics, so it is an Operate habit rather than a design-time guess, and it stops at the floor that availability sets: the search cluster's three nodes are a quorum, not over-provisioning — shrink the nodes, not the count.
- Scheduling switches off what nobody is using. Development and test environments used 07:00–19:00 on weekdays need 60 of the week's 168 hours; stopping them outside those hours removes 64% of their instance time. If two-thirds of Lumen's $1,550 non-production line is instance time (assumed), that is ≈ $660 a month.
- Budgets alert when actual or forecast spend crosses a threshold, per account, region or service (AWS Budgets, Azure Cost Management budgets, Google Cloud Billing budgets).
- Anomaly detection compares each day's spend with its own history and alerts on a change nobody planned, catching in a day what a monthly budget sees at month end. The large providers offer it natively. QA18 applies the same idea to unit cost — a rolling 7-day cost per referral more than 10% above the previous 28 days alerts the cost owner within a day — which is the better signal, because a bill that rises with referrals is not an anomaly.
Lumen's most expensive failure mode is on the SMS line. A notification worker that sends every message three times — a retry loop without the idempotent receiver from part 2 — adds $25,200 a month. The defences are architectural first: idempotent consumers, a dead-letter queue, and a cap on messages per referral per day in the notification service. The financial alert comes second, and it should watch message counts, which arrive in minutes, rather than dollars, which arrive with the invoice.
Showback and chargeback
Showback reports each team's or unit's allocated cost to it without moving money; chargeback bills that cost to their budget. Chargeback changes behaviour more, but only works once allocation is trusted, the shared-cost rule is agreed and the people charged can change what they are charged for. Start with showback. On Lumen the Supplier operates the platform (C01) and invoices Lumen; a monthly cost per referral for each region is a reporting deliverable the Supplier can offer, and whether Lumen then charges its regions internally is Lumen's decision.
Designing for cost visibility
Most of FinOps is operations. Five parts of it are architecture, and are cheap only if decided early:
- Account and project structure by environment and region, so the main splits are exact
- Business metrics beside cost — referrals received, bookings confirmed, messages sent — emitted by the platform, so cost per referral is computed daily rather than reconstructed from an invoice. This is QA18's measure
- Billing data as a feed — the providers' cost and usage exports, increasingly in the FinOps Foundation's FOCUS format (FinOps Open Cost and Usage Specification), loaded into the reporting store from part 2 and joined to the business metrics
- Cost in the delivery pipeline — the cost change of every infrastructure pull request estimated before merge (Infracost does this for Terraform), and a cost-per-referral fitness function that alerts when the rolling figure breaches QA18
- Per-region messaging configuration, so the largest line is attributable too
Observability platform
QA01 and QA02 are promises; observability is how you know you are keeping them, and how quickly you find out when you are not. Module 3 records both as QA17: a booking-path failure or slowdown that threatens QA01 or QA02 pages on-call within 5 minutes, and any referral can be traced end to end, from the extract that carried it to the patient's email and SMS. Module 3, part 2 covers the signals, OpenTelemetry, the burn-rate arithmetic and the rules for patient data. This section is the platform that implements them: where telemetry flows and is stored, how the alerting meets QA17 once the pipeline's own delay is counted, what stays in the region, and what it all costs.
Collection and storage
OpenTelemetry (OTel) standardises how telemetry is produced and moved — SDKs, semantic conventions, the OTLP protocol and the Collector — but not where it is kept. That split is the platform's first design decision, and the reason to adopt OTel: the services are instrumented once, and the stores stay a choice you can revisit — cheaper storage, another vendor, a different store per region — without touching the code.
| Signal | On Lumen | Typical destinations |
|---|---|---|
| Traces | A booking confirmation: gateway, booking function, database, outbox | Jaeger, Grafana Tempo; AWS X-Ray, Azure Monitor Application Insights, Google Cloud Trace |
| Metrics | Requests and errors per route and screen class, latency histograms, queue depth, messages sent | Prometheus-compatible stores; Amazon CloudWatch, Azure Monitor Metrics, Google Cloud Monitoring |
| Logs and events | A rejected row in a referral extract; "booking confirmed", with the referral's correlation token | OpenSearch, Grafana Loki; CloudWatch Logs, Azure Monitor Logs, Google Cloud Logging |
The Collector is usually deployed in two tiers. An agent beside each workload — for serverless functions, a layer or extension that runs in the function's own environment — receives telemetry locally and forwards it. A gateway tier of Collectors in each regional stack does the heavy work once: redaction, tail sampling, batching and export. Tail sampling needs every span of a trace at the same gateway instance, so the agents route by trace ID; OTel's load-balancing exporter exists for exactly this.
Telemetry is stored in the region that produced it; only SLO aggregates — counts and latency buckets, with no patient data — feed the cross-region view. QA13's audit records never share the operational pipeline.
End-to-end referral tracing rests on the opaque correlation token the referral sync assigns when a referral first arrives; Module 3, part 2 explains why one trace cannot span a referral's days. The platform's share is plumbing. Carry the token and W3C Trace Context in the attributes of every queue message as well as on every HTTP call; index the token in the log and event store, so one query returns a referral's history in seconds; and keep logs and events for every referral while traces are tail-sampled. That combination is what makes QA17's "any referral" affordable.
SLO dashboards and burn-rate alerts
An SLO dashboard shows, for each user journey — book an appointment, view an offer, upload slots — the SLI over its window, the error budget left and the current burn rate; per-server CPU belongs on a different screen. Where each of the three ASRs gets its measurement:
| ASR | SLI | Produced by | Window and alert |
|---|---|---|---|
| QA01 | Valid booking-path and portal requests answered without error or timeout ÷ valid requests | The portal front end, cross-checked at the gateway, plus a synthetic booking every minute, so the SLI never runs dry at night and an outage that stops users arriving at all still counts | 24×7 over a rolling year; burn-rate pages on a 30-day budget |
| QA02 | Routine-screen requests answered within 2 s ÷ all routine-screen requests, per screen class; met at 95% | Real-user monitoring in the portal front end, with gateway latency beside it to show where the time goes | Each hour, judged on the busiest; a short-window page when a quarter of one screen class runs over 2 s |
| QA17 | Time from injected fault to page; share of sampled referrals whose full history one token query returns | Fault-injection rehearsals in staging, and game days in production | Each release, and each quarter |
Expressing QA02 as the share of fast requests, rather than as a p95, states the same target as a count — and counts add up across regions and hours, which percentiles do not. So the platform stores latency as histograms, never as pre-computed percentiles.
The pipeline has a delay budget. Module 3, part 2 derives the alert rules: the 1-hour, 14.4 burn-rate rule pages a failure of a quarter of booking requests after ≈ 3.5 minutes of data, which is where QA17's 5 minutes comes from. The remaining ≈ 1.5 minutes belong to the platform — the metric export interval, Collector batching, the rule-evaluation interval and page delivery. Defaults spend it fast: a 60-second export interval and a 60-second evaluation interval can add two minutes on their own and push that page past 5 minutes. Set both to 15–30 seconds for the booking-path SLIs, keep the alerting path out of any batch pipeline, and measure the whole chain by injecting a fault and timing the page.
PHI-safe telemetry
Module 3, part 2 sets the rules: no identifiers, allow-listed attributes, residency, a separate audit trail. The platform decides where each rule is enforced:
- In the code, first. Attributes come from an approved list in a shared instrumentation library, and request and response bodies are never recorded in production
- In the regional gateway Collector, as the backstop. Attributes not on the list are dropped, and patterns such as identifier formats, email addresses and phone numbers are masked, before anything is exported. Patterns miss things, so this catches mistakes; it does not replace the first control
- At the platform's edge. OTel baggage propagates to every downstream service, third parties such as the SMS provider included. Never put identifiers in it, and strip it from calls that leave the platform
- In the choice of backend. Every store sits in the region of collection (QA05). A SaaS observability vendor that cannot keep each jurisdiction's telemetry in that jurisdiction is ruled out, however good the product
- In access and retention. Role-based access to the stores, audited where the data could hold patient information, and retention measured in weeks for operational telemetry
The controls underneath — classification, pseudonymisation, key management and audit — are in Module 10.
The cost of telemetry
Telemetry is billed by volume, and its volume is a design decision. In the worked model it is two lines — logs and audit trail at $57 a month, and the $150 observability floor in each of four stacks — together $657 a month, about one dollar in eight of production's non-messaging spend ($5,167). More than half of the $57 is ingestion, so splitting the audit trail from operational logs, as QA13 requires anyway, is a compliance decision more than a saving. Three drivers move the total:
- Log volume. The model assumes ~2 KB per invocation. Logging request and response bodies at ~20 KB would make it ≈ 670 GB a month — ≈ $335 to ingest, before storage — and put patient data in the logs
- Trace volume. Tracing every invocation at ~4 spans of ~0.5 KB would add ≈ 67 GB a month, ≈ $33 at the model's ingestion price; tail sampling every error and slow trace plus 10% of the rest, as Module 3, part 2 assumes, cuts that to a few dollars. At these prices sampling is about noise and retention more than money. Some vendors price per span instead: at an illustrative $1.50 per million spans, tracing everything — 133M spans — costs ≈ $200 a month, and the sampling rate becomes a budget decision. Read the pricing model before setting it
- Metric cardinality. Metric stores bill per time series: each distinct combination of a metric's name and label values. A latency histogram is already ten or more series, one per bucket. Label it by practice (600) and outcome (3) and it becomes 600 × 3 × 10 = 18,000 series; at an illustrative $0.10 per series a month, that is $1,800 a month — nearly three times the whole telemetry budget above, for one metric. Keep high-cardinality detail such as practice and referral in traces and logs, and keep metric labels to small, fixed sets
Well-Architected frameworks
Each large provider publishes a Well-Architected framework: pillars, design principles and review questions distilled from customer architectures. They are not standards and not neutral — the recommendations end in the provider's own services — but their questions make a free, well-tested checklist, and evaluators recognise the vocabulary.
| Framework | Pillars (checked November 2026) |
|---|---|
| AWS Well-Architected Framework | Six: operational excellence, security, reliability, performance efficiency, cost optimization, and sustainability, added in 2021 |
| Azure Well-Architected Framework | Five: reliability, security, cost optimization, operational excellence and performance efficiency |
| Google Cloud Well-Architected Framework, formerly the Google Cloud Architecture Framework | Six: operational excellence; security, privacy and compliance; reliability; cost optimization; performance optimization; and sustainability, made a full pillar in January 2026 |
All three add guidance for particular workloads. AWS publishes lenses in the Well-Architected Tool's catalogue, among them a Healthcare Industry Lens (2022) with healthcare-specific design principles and scenarios. Azure publishes guidance per workload type. Google Cloud adds cross-pillar perspectives, currently for AI and ML and for financial services, with no healthcare perspective at the time of writing.
Mapping the pillars to quality attributes
| Module 3 attribute | AWS and Azure pillar | Google Cloud pillar | Lumen ASRs |
|---|---|---|---|
| Availability, including disaster recovery | Reliability | Reliability | QA01, QA14, QA15 |
| Performance | Performance efficiency | Performance optimization | QA02, QA03, QA07 |
| Scalability | Performance efficiency | Performance optimization | QA06 |
| Security | Security | Security, privacy and compliance | QA09, QA12 |
| Privacy and compliance | Security — no pillar of its own | Security, privacy and compliance | QA05, QA13 |
| Deployability | Operational excellence | Operational excellence | QA08 |
| Observability | Operational excellence | Operational excellence | QA17 |
| Cost-efficiency | Cost optimization | Cost optimization | QA18 |
| Modifiability | Operational excellence, in part | Operational excellence, in part | QA10, QA16 |
| Usability | No pillar | No pillar | QA04 |
| Interoperability | No pillar | No pillar | QA11 |
| No Module 3 attribute | Sustainability (AWS only) | Sustainability | None |
Three things the mapping shows:
- The frameworks cover operating qualities well and product qualities poorly. Usability, interoperability and most of modifiability have no pillar, so a design can pass a Well-Architected review and still fail QA04, QA10 and QA11
- Privacy and compliance has no pillar of its own on AWS or Azure; it sits inside security. Where residency is a legal precondition, as QA05 is for Lumen, review it as a topic in its own right whichever framework you use
- Sustainability has no Lumen ASR. A review will ask about it; the straight answer is "no requirement", followed by what the design does anyway — functions that scale to zero, right-sized nodes, non-production switched off at night
A Well-Architected review
A Well-Architected review walks a workload through the framework's questions, pillar by pillar, records high- and medium-risk items and agrees an improvement plan. It is light next to ATAM: no utility tree and no scenario brainstorming, a few hours with the team, and free self-service tooling such as the AWS Well-Architected Tool and Microsoft's Azure Well-Architected Review. Module 8 places it among the lighter-weight evaluation methods.
Its limits are the mirror image of ATAM's strengths. The questions are generic rather than derived from Lumen's business drivers; it finds missing practices, not trade-off or sensitivity points; and it rewards using the provider's services. Run it after the ASR-driven review, not instead of it. On Lumen: once at the end of discovery on the proposed design, with the healthcare lens if the chosen cloud offers one, and again before each new region goes live (QA16) — a new regional stack is exactly where drift from the template appears.
Check yourself
- Why does a container start in well under a second while a virtual machine takes tens of seconds — and what does the VM buy you in return?
- A team patches its production servers in place every month to save time. Which immutable-infrastructure rules does this break, and what does the team lose after a year of it?
- Two instances of a service, each 99.5%, sit behind a load balancer at 99.99% and call a single database at 99.95%. Estimate the availability, and give two reasons the real figure will be lower.
- Lumen's busiest hour is Monday morning, every week. Why is reactive autoscaling a poor fit for it, and what would you do instead?
- Would you cache Lumen's availability-search results with a long TTL? Say what goes wrong, and what you would do instead.
- In the worked cost model, why is the API gateway priced on 26.55M calls rather than on all 33.3M invocations?
- The worked model totals ≈ $19,360 a month. Which two changes would move it most, and who must agree to each?
- Why can't you copy a provider's 99.99% SLA into your availability table as the component's availability?
- Finance wants to buy three-year reserved capacity for all four regional database stacks now, before discovery. What do you advise?
- The booking path uses the 1-hour, 14.4 burn-rate rule, but metrics are exported every 60 seconds and alert rules evaluated every 60 seconds. Does the platform meet QA17, and what would you change?
Suggested answers
- A container is an isolated process sharing the host's kernel, so starting it is starting a process; a VM must boot a whole guest operating system. The VM's separate kernel gives stronger isolation, which matters for untrusted or multi-tenant workloads.
- "Don't modify instances in place" and "Replace instances to update them" — and, by running the same servers all year, "don't let instances get stale". After a year nobody can say what is configured where (drift), rebuilding a server becomes risky, and failures no longer reproduce in test.
- The pair: 1 − 0.005² = 99.9975%. In series: 0.999975 × 0.9999 × 0.9995 ≈ 99.94%. It will be lower because the two instances fail together (same zone, same bad release), failover is not instant, and the figures are SLA thresholds rather than measured availability.
- The autoscaling delay — metric window, provisioning, start-up and health checks — lands in the first minutes of the peak, exactly when the referral desk is busiest. The peak is predictable, so scale on a schedule before 07:00 on Mondays and keep reactive scaling as the safety net.
- No. Slots disappear with every booking, so a long TTL offers slots that are already taken. Use a short TTL, or invalidate on booking, and always re-check the slot against the system of record when the booking is confirmed — the cache can speed up the search but must not decide the booking.
- Only requests from outside the platform — the portals — cross the gateway. Stage-change handling, notification dispatch and sync are triggered by queues and schedules inside the platform and never pass through it.
- Restricting SMS to the stages that matter (≈ −$8,400 a month), which is a requirement change Lumen must accept; and one shared stack instead of four (≈ −$3,040 a month before non-production), which depends on Lumen's answer about where the regulation requires hosting. Neither is the Supplier's decision alone.
- An SLA is a service-credit threshold, not a measurement. It carries conditions (for example two or more zones) and exclusions, it is measured monthly, and it covers only the provider's infrastructure — your software and configuration fail too, which is why a component's availability is infrastructure × software.
- Not yet. Reserved capacity is bought per region, and the residency clarification may consolidate the four current regions into one shared stack; then at least three regions' reservations are stranded — $821 a month at the assumed 50% discount, until the term ends; a region-flexible spend commitment, where offered, avoids that at a lower discount. Settle the stack count first, then commit for one year on the instances certain to run all year (primary and standby), keep the read replica and non-production on demand, and revisit after a year of measured load.
- Not reliably. When a quarter of booking requests fail, the rule needs ≈ 3.5 minutes of data (0.864 ÷ 0.25), which leaves ≈ 1.5 minutes for the pipeline; a 60-second export interval and a 60-second evaluation interval can add up to two minutes before the page is even sent. Shorten both to 15–30 seconds for the booking-path SLIs, keep alerting out of any batch pipeline, and prove the result by injecting a fault of that severity and timing the page.
Exercises — Module 9
Task. Design, size and cost the production deployment of Lumen Diagnostics on one public cloud — AWS, Azure or GCP. Output: a 5–7 page document — one deployment diagram with half a page on residency and disaster recovery, a per-component availability table, a monthly and five-year run-cost table covering production and non-production, and a page on FinOps and observability. Time: about 7 hours, most of it on the availability table.
Inputs
- The case's Volumes and usage (Appendix B of the RFP) — every count starts here
- QA01 (availability), QA02 (response time), QA05 (residency), QA14 (recovery objectives), QA16 (a new region in service quickly), QA17 (observability) and QA18 (cost-efficiency) from your Module 3 ASR list — compare them with the worked ASR list
- The cost-of-outage method in Module 3, which tells you what an outage hour costs Lumen and how the budget splits across the booking path
- The composition rules and recovery objectives in Module 3, part 2 — Availability mathematics, and its SLI and SLO definitions
- Your Module 5 container diagram: the deployment view places those containers
- The worked cloud cost model above — extend it, don't copy it
1. Deployment view
- Draw the production deployment with your chosen cloud's stencils (draw.io, Lucidchart, Visio, or cloudcraft.co for AWS)
- Show regions and availability zones, and which containers run in each. Say whether Lumen's four current regions share one stack or get one each, and why
- Show how the region outside the current jurisdiction keeps its patient data there (QA05): its own stack, what crosses the border if anything (de-identified aggregates for central reporting, for example), and where its backups live
- State a disaster-recovery strategy — backup and restore, pilot light, warm standby or active-active — that meets the RPO and RTO in QA14 (or your own), and show the outage cost that justifies them
- Mark where the DR copy runs, and check that it does not break residency — in the new jurisdiction the provider may offer only one region
2. Availability table
- One row per component on the booking path and the patient-portal path: DNS and edge, gateway, compute, database, search, identity provider, and any third party called synchronously
- For each: how it is deployed (single, zone-redundant, multi-region), the availability you use, and its source — the provider SLA document and the condition it depends on (for example "two or more zones"), or your own estimate, labelled as one
- Compose with the series, redundancy and infrastructure × software rules, showing the arithmetic, and state the measurement window (24×7 or operating hours)
- Compare the result with QA01. If it misses, change the design, not the numbers; if it passes only on SLA figures, say what you would measure in production to confirm it
3. Run-cost table
- Start from the worked model's counts. Replace its prices with your provider's current list prices for your chosen regions, citing the price pages
- Add the rows your deployment view adds: the DR copy, and the two new regions (assume they go live at the start of year 3 unless you say otherwise)
- Show production and non-production as separate lines, with your non-production assumption stated
- Give the monthly figure at today's volumes, each of years 1–5 at 12% growth, and the five-year total
- Close with one paragraph: the line that dominates, and the change that would move it most — and who must agree to it
4. FinOps
- State the run cost per referral from your own cost table at today's volumes and at QA18's year-3 volume (×1.405), with the two new regions and your DR copy included, and say whether QA18's $0.06 ceiling holds
- Say how cost is allocated per region and per environment: the account or project structure, the mandatory tags and the rule for shared costs
- Make one rate or usage recommendation — a commitment, with its term, scope, break-even and what it assumes about the residency answer, or a right-sizing or scheduling change — with its monthly saving
5. Observability
- For QA01, QA02 and QA17, give the SLI, where it is measured, the window and the alert — burn rates and windows for QA01 — and show that the page, including your pipeline's own delay, arrives within QA17's 5 minutes when a quarter of booking requests fail
- Show how one referral is followed end to end — trace context across the queue, the correlation token, the sampling rule — and what is kept out of telemetry and in the region (QA05)
Self-assessment
| Criterion | What good looks like |
|---|---|
| Traceability | Every count in the cost table and every target in the availability table traces to Appendix B, a Module 3 ASR or a labelled assumption; nothing is invented silently |
| Deployment view | Regions, zones and the placement of every Module 5 container are visible; residency for the new jurisdiction and the DR copy are drawn, not only described |
| Disaster recovery | RPO and RTO are numbers tied to the outage cost, and the strategy can meet them — asynchronous cross-region replication gives an RPO of seconds to minutes, never zero — and the recovery steps are written down well enough to rehearse |
| Availability | Each figure cites its SLA and condition; series and redundant elements are composed correctly; correlated failure and the measurement window are addressed; the comparison with QA01 is honest |
| Run cost | Volumes come from counts, never peak × 730 h; the gateway is priced on external calls; database, backups, egress, observability, non-production and per-region rows are present; five years are projected with growth |
| FinOps | Unit cost per referral is derived from the cost table, not asserted, and checked against QA18 at year-3 volume with the new regions and the DR copy included; allocation gives exact per-region and per-environment figures; the recommendation shows its saving, its break-even and the condition under which it would be wrong |
| Observability | QA01, QA02 and QA17 each have an SLI, a measurement point, a window and an alert; the detection time includes the pipeline's own delay; no patient identifier reaches telemetry, and telemetry stays in its region |
| Judgement | The dominant cost line and the biggest availability risk are named, each with the decision Lumen must make about it |