Recitables: the one-page sheet

Everything here should be recallable without notes, out loud, in under five seconds. It is the night-before layer and the whiteboard layer: numbers you will be asked to produce from memory, formulas you will be asked to apply live, and one-sentence positions you should be able to state and defend.

The sheet grows as topics are written, and it gets cut at the end. The final form is roughly forty entries, chosen by recall value rather than by completeness. Anything you can look up in the moment does not belong here.

Latency and capacity

Little's Law. $L = \lambda W$, that is, concurrency = throughput x latency. A service at 3,000 QPS and 50 ms holds 150 requests in flight, so it needs 150 threads and 150 database connections or the pool is the ceiling. Inverted: a pool of 50 at 50 ms caps you at 1,000 QPS regardless of how many app servers you add.

Queueing. Wait scales as $1/(1-\rho)$. At 50 percent utilisation latency is 2x unloaded, at 80 percent it is 5x, at 90 percent 10x, at 95 percent 20x. The knee is 70 to 80 percent; that is why capacity targets live there. It is arithmetic, not conservatism. Service-time variance multiplies the wait by $(1+C^2)/2$ at any utilisation, which is why one slow query type poisons a shared pool and why bulkheads exist.

Universal Scalability Law. $C(N) = N / (1 + \alpha(N-1) + \beta N(N-1))$. The $\beta$ coherence term is quadratic, so throughput does not plateau, it declines past an optimum.

Tail at scale. Fan out to $N$ servers each with probability $p$ of being slow and the aggregate is slow with probability $1 - (1-p)^N$. At $N = 100$, $p = 1%$, that is 63 percent. So the percentile you must control is set by your fan-out: at $N = 60$ and a 99 percent target you need each backend's p99.98. Hedge at the p95, and gate the hedge on its own rate, because under overload it amplifies.

Exactly-once is impossible (Two Generals). What you build is at-least-once plus deduplication, which is why idempotency keys are the actual guarantee at every boundary.

Availability

Serial dependencies multiply. Three 99.9 percent dependencies in series give 99.7 percent, which is 2h 10m a month rather than 43m. You cannot be more available than the product of your hard dependencies.

Burn-rate alerts (30-day window). Page at 14.4x over 1h with a 5m short window (2 percent of budget); page at 6x over 6h with a 30m short window (5 percent); ticket at 1x over 3d with a 6h short window (10 percent). Long window proves significance, short window proves it is still happening.

Set the SLA looser than the SLO. Internal 99.95, contractual 99.9. The gap is your reaction buffer.

Disaster recovery

RTO is time to restore; RPO is data you may lose. Independent. The ladder, cheapest first: backup and restore (hours), pilot light (tens of minutes), warm standby (minutes), active-active (near zero, 200 percent-plus cost).

Light in fibre is ~5 µs per km, so ~1 ms per 100 km round trip. London to Virginia is ~75 ms, which is why synchronous cross-continent replication is off the table and your cross-region RPO is your replication lag.

Your real RTO is the max of your critical dependencies' RTOs. The three everyone forgets: identity provider, DNS (and its TTL), secrets manager.

Storage and consistency

Quorum. $R + W > N$ for strong consistency. LOCAL_QUORUM is the practical multi-datacenter default in Cassandra.

Raft cluster sizing. 3 tolerates 1 failure, 5 tolerates 2. Never an even number: 4 tolerates the same 1 as 3, more slowly.

Write latency floor for consensus = fsync + intra-cluster round trip, roughly 1-2 ms on NVMe in one AZ, 5-15 ms across AZs.

Serializability and linearizability are orthogonal. Linearizability is recency on a single object; serializability is isolation across objects. Strict serializability is both.

Postgres REPEATABLE READ is snapshot isolation and permits write skew. Only SERIALIZABLE prevents it, and every serializable system requires application retry logic on 40001.

Cassandra rules of thumb. Partition under ~100 MB and ~100k rows. gc_grace_seconds default 10 days, and you must run full repair more often than that or deleted data resurrects.

DynamoDB. 400 KB per item, 1 MB per query page, 100 items per transaction (at ~2x write cost), ~3,000 read units or ~1,000 write units per partition before throttling. LSIs cap a partition key at 10 GB; GSIs do not.

OpenSearch. 10-50 GB per shard. JVM heap at 50 percent of RAM and under 32 GB so compressed object pointers survive; the rest is page cache for Lucene.

LLM serving

KV cache per token $= 2 \times L \times H_{kv} \times d_{head} \times b$ bytes. Llama 3 70B in fp16: $2 \times 80 \times 8 \times 128 \times 2 = 320$ KiB per token, so 2.5 GiB for an 8k context. KV cache, not weights, caps concurrency.

Prefill is compute bound (arithmetic intensity ~ sequence length). Decode is memory bandwidth bound (intensity ~ batch size). H100 ridge point is roughly 300 FLOPs per byte. This is why batching transforms decode throughput and does almost nothing for prefill.

Prompt caching economics. A cache read costs ~0.1x input price; a write costs 1.25x (short TTL) or 2x (long TTL). Break-even: two requests on the short TTL, three on the long. Caching is a prefix match, so one timestamp in the system prompt disables it entirely. In a multi-turn conversation the breakpoint belongs at the end of the latest turn, not the system prompt.

Reciprocal rank fusion. $1/(k + \text{rank})$, summed across lists, $k = 60$. Uses rank rather than score because BM25 is unbounded and corpus-dependent while cosine is bounded and compressed. Large $k$ flattens the curve, so a document must rank well in several lists rather than topping one.

GQA cuts KV cache by the grouping factor: 64 query heads with 8 KV heads is 8x smaller cache, so 8x more concurrent sequences. PagedAttention recovered the 60 to 80 percent that naive allocation wasted to fragmentation, and gives prefix sharing for free.

Agent evaluation. 95 percent per-step accuracy over 20 steps is 36 percent task success. Use task-level success against a checkable end state, and pass^k (all $k$ attempts succeed) rather than pass@1.

The lethal trifecta. Private data + untrusted content + external communication. Remove any one leg and prompt injection cannot complete.

Cost per request = (input tokens x input price) + (output tokens x output price), divided by cache hit rate. Levers ranked: prompt caching, model routing, context trimming, semantic caching, distillation, batch API.

Delivery and process

Deploy is not release. Deployment moves bits, release exposes users, feature flags decouple them.

Composite deployment answer. Rolling as the mechanism, canary as the policy, flags as the exposure control, rings for blast-radius sequencing.

Minimum detectable effect. To detect 0.1 percent to 0.2 percent error rate at 95 percent confidence and 80 percent power you need about 23,500 requests per arm. At 200 QPS a 1 percent canary is 2 QPS, so 3.3 hours. A 30-minute 1 percent canary cannot detect a doubling.

PR size. Review effectiveness collapses past roughly 400 lines.

Capacity commitment. headcount x available days, minus on-call, interviews, support and meetings; commit to 60-70 percent of what remains.

Three-bucket budget. Features / reliability and tech debt / keep-the-lights-on, with a published split (60/25/15 is a common healthy target).

Forecast with percentiles, not dates. "85 percent confidence we finish by the 22nd" from the cycle-time distribution.

Positions worth having in one sentence

• Exactly-once is at-least-once plus deduplication plus atomic visibility, and it stops at the Kafka boundary.
• Never call flatMap without a concurrency argument. The default 256 is a buffer constant, not a limit.
• An L4 load balancer balances connections, and HTTP/2 has one connection, so gRPC pins to a backend and freshly scaled pods get nothing.
• Cache invalidation bugs are consistency bugs, so choose the staleness you can tolerate before you choose a cache.
• Jittering TTLs does not fix a stampede on one hot key, because every client reads the same expiry.
• CFS throttling is about instantaneous parallelism, not average utilisation. A 1 CPU limit with 16 runnable threads freezes for 94 ms of every 100.
• PKCE is required for confidential clients too, because a secret authenticates the app and does not bind the code to the request.
• A canary asks whether it is safe; an A/B test asks whether it is better. Every A/B variant rides through a canary first.
• Retrieval failure and grounding failure are different bugs. The context injection test separates them in five minutes.
• A modest upstream latency increase can cross a pool's Little's Law ceiling and produce a nonlinear failure downstream. Instrument pool wait, not just dependency latency.
• Series count, not sample rate, is the metrics cost model. One unbounded label turns hundreds of series into millions, and WAL replay after the OOM leaves you blind.
• Post-filtering an ACL leaks existence, not just recall. Never do it.
• Money is integers in minor units, and balances are a projection of an append-only entry log, never the source of truth.
• Access patterns first, schema second. In DynamoDB the key schema is the query plan and you cannot change it later without rewriting the data.
• Interleaving picks the ranker; A/B decides whether to ship it. One to two orders of magnitude more sensitive, and it cannot see revenue.
• Never post a DataLoader as a module singleton. Its memoisation crosses users, which makes it a data leak rather than an optimisation.
• Extraction forces for a service: independent scaling, independent deploy cadence, fault isolation, polyglot need, regulatory separation. Codebase size is not one.
• Fine-tuning is for form; retrieval is for facts. Retrieval also wins whenever you need citations or per-user access control.
• A reranker cannot fix recall. It reorders what retrieval returned; recall@k is the ceiling.
• Prompt-layer defences against injection are mitigation, not a boundary. Architecture is the defence.
• KEV first, then EPSS times exposure, then CVSS. And patching velocity is bounded by inventory accuracy.
• A watermark is a heuristic, not a guarantee, which is why allowed lateness and side outputs exist. One idle partition freezes the whole job.
• Most agent failures are tool-design failures, not model failures.
• The alternative was X, I am not choosing it because Y, if Z changed I would revisit. Say this at least twice per design round.