Module 4 Β· Foundation

Reliability Theory β€” when things fail together

Complex systems fail.

The single idea

Scale turns β€œrare” into β€œroutine”

A component with 99.9% reliability sounds excellent.

Series

Weakest link

All must work.

Parallel

Redundancy

At least one path up.

Availability

Repair matters

High MTBF with long MTTR still means painful outages.

Cascade

Fault propagation

One bad hop poisons downstream steps.

Interactive

Resilience Lab β€” redundancy math

Compare series vs parallel.

System reliability 99.50%

Series of 5 Γ— 99.9% stages β€” one weak hop dominates.

Rsystem β€”
P(failure) β€”
Availability A β€”
Expected fails / 1k β€”

Rsys = Rn

Fault propagation

Cascade simulator

A linear AI pipeline.

What you should see

Real systems patterns

Microservice chain

Ten services at 99.9%.

Multi-AZ redundancy

Parallel paths help until common-mode.

LLM agent pipeline

Hallucination at step 2.

On-call reality

MTTR dominates user pain.

Cross-domain transfer

Same law β€” different context

After the lab, map the model to your stack.

HW

Components & derating

Chain of Rα΅’ blocks.

FW

Watchdog & safe state

Task failure β†’ safe mode.

FPGA

Redundancy / TMR

Voting modules = parallel redundancy.

SW

Microservice cascade

10 Γ— 99.9% β‰ˆ 99% end-to-end.

Self-check

Did I get it?