Module 2 ยท Foundation

Control Theory โ€” close the loop

An autoscaler is a feedback controller.

Open the Autoscaler Lab First, the idea

The single idea

Measure the gap. Adjust. Repeat.

Every feedback controller works the same way.

Too slow

Underdamped

The controller reacts lazily.

Just right

Critically damped

Fastest approach without overshooting.

Too aggressive

Oscillatory

Overshoot and oscillation.

Delay kills

Delayed feedback

Stale measurements cause wrong decisions.

Interactive

Autoscaler Lab

Tune gain and delay.

Controller status Settling

Watch convergence

Overshoot 0%
Settle time โ€” s
Steady-state error 0.0
Oscillation count 0
load your servers ref: under ref: critical ref: oscillatory

Frequency domain

Why does a system oscillate? โ€” The Nyquist answer

๐Ÿš—

The car on a highway

You see the car drift right. It takes a second. You turn the wheel left. That takes another second. By the time your correction reaches the wheels, the car has already drifted further โ€” and if you turn too hard, you overshoot and have to correct again, harder, faster. The car starts violently weaving. That's instability.

The problem: phase delay (lag). Sensors, motors, and your brain don't work instantly.

๐ŸŽค

The microphone and speaker

You speak into a mic. Sound comes out of the speaker, slightly louder, slightly delayed. Normal. Then you step closer. The speaker feeds back into the mic โ€” the same signal comes around again, amplified. At some frequency, it arrives exactly out of phase (180ยฐ) and stronger than the original โ€” a piercing screech. That frequency is the point (โˆ’1, 0) on the Nyquist diagram.

The rule: if the curve stays left of โˆ’1, no screech. If it crosses or encircles โˆ’1, the system screams.

Stable Curve passes left of โˆ’1
Car tracks straight. No feedback howl.
At the margin Curve touches โˆ’1
One sneeze away from oscillation.
Unstable Curve encircles โˆ’1
Wheel jerks out of hands. Screech.

How the criterion works, very simply: take the open-loop system โ€” sensor disconnected, no corrections. Study how "steering wheel" affects "wheels" without your input. Draw a curve showing, at every frequency: how much the signal changes (magnitude) and how much it lags (phase). Then look at the red point (โˆ’1, 0) โ€” the "danger button." It means: "the signal came back exactly opposite in phase and just as strong as it was." If the curve goes left of that point โ€” stable. If it wraps around it โ€” unstable. If it passes through โ€” on the edge.

G(s) = Kp ยท eโˆ’ฯ„s (s + 1)(s + 2)
|G| = โ€”   โˆ  โ€”ยฐ   ฯ‰ = โ€”
stable region approaching โˆ’1 near danger tracer โˆ’1 point
Stable โ€” no screech

The mic and speaker are far apart. Signal decays with each loop. The curve stays to the left of โˆ’1.

Beyond classic PID

AI & ML approaches โ€” controllers that learn

Classic PID requires manual tuning. Modern systems use artificial intelligence to auto-tune gains or even replace the PID equation entirely. Below are the key approaches that are changing how we build controllers.

๐Ÿง 

Reinforcement Learning

An RL agent discovers optimal gains through trial and error in a simulator. Reward signal: minimize error, overshoot, and settle time.

How it works: Agent observes error โ†’ takes action (adjust gains) โ†’ gets reward โ†’ learns policy that maximizes cumulative reward.

Real example: Google DeepMind used RL to tune cooling in data centers โ€” 40% energy reduction.

๐Ÿ”—

Neural Network Controller

A neural network directly maps sensor readings to control actions โ€” no PID formula needed. Trained on historical data or in simulation.

How it works: Input = [error, error history, system state] โ†’ Hidden layers โ†’ Output = control signal u(t). Learns nonlinear policies.

Real example: Tesla Autopilot โ€” NN processes camera data and outputs steering angle, bypassing any fixed PID structure.

๐ŸŽฏ

Bayesian Optimization

Searches the gain space (Kp, Ki, Kd) intelligently. Builds a probability model of performance vs gains, tests only the most promising candidates.

How it works: Gaussian Process models the unknown performance landscape โ†’ picks next trial where uncertainty + expected gain is highest.

Real example: Tuning a chemical reactor controller โ€” each experiment costs thousands. Bayesian Opt finds optimal gains in 10โ€“30 trials instead of hundreds.

๐Ÿ”ฎ

Model Predictive Control + ML

Uses a learned model of the system to look ahead and optimize control actions over a future horizon. ML learns the model from data.

How it works: ML predicts future states โ†’ optimization engine computes best sequence of actions โ†’ applies first action, repeats.

Real example: Energy grid balancing โ€” predicts demand spikes 30 min ahead, pre-emptively dispatches generation capacity.

๐Ÿงฌ

Genetic Algorithms

Mimics natural selection: a population of gain sets evolves over generations. Weak controllers die, strong ones mutate and recombine.

How it works: Population of (Kp, Ki, Kd) โ†’ evaluate fitness on simulator โ†’ select best โ†’ crossover & mutate โ†’ repeat.

Real example: Tuning flight controllers for delivery drones in turbulent wind โ€” finds robust gains that survive real-world noise.

โš™๏ธ

AutoML for Control

End-to-end automated pipeline: from raw sensor logs, through feature engineering, to a trained controller model. Discovers control structures a human might never invent.

How it works: Data โ†’ Auto-preprocessing โ†’ Feature selection โ†’ Model architecture search โ†’ Hyperparameter tuning โ†’ Deployed controller.

Real example: Autonomous racing โ€” a learned throttle+steering policy discovered by AutoML beats hand-tuned PID by 0.4s per lap.

Classic PID vs ML-based control โ€” side by side

Aspect Classic PID ML-based Control
Interpretability Every term has physical meaning Neural nets are black boxes
Tuning effort Manual, trial-and-error Automated (RL, Bayesian Opt)
Handles nonlinearity Linear controller only Any nonlinear policy
Data requirements Low โ€” just error signal High โ€” needs training data
Safety guarantees Well-studied stability margins Hard to formally verify
Adaptability Fixed gains, needs retuning Can adapt online
Where used 95% of industrial control loops Robotics, autonomous driving, data centers
๐Ÿ”€

The hybrid future: AI-tuned PID

The most practical path today is not to replace PID, but to let AI tune it continuously. A reinforcement learning agent watches the system and adjusts Kp, Ki, Kd in real time as conditions change. The controller stays interpretable and safe (PID), while the tuning becomes adaptive and optimal (AI). This is called adaptive PID or neural-PID โ€” it's already running in industrial plants, drone autopilots, and cloud autoscalers.

What you should see

Five behaviors, one controller

Click each preset above.

Underdamped

Low gain.

Critically damped

Balanced gain.

Oscillatory

High gain.

Delayed feedback

Stale metrics.

Integral term

Ki eliminates error.

Cross-domain transfer

Same law โ€” different context

After the lab, map the model to your stack.

SW

Autoscaling

Gain and delay in the lab.

HW / EE

Thermal / power PID

Power stage with sensor delay.

FW

Speed loop

PWM + encoder sampling delay.

EE

PLL lock

Phase-locked loop is feedback.

Self-check

Did I get it?