Networking II — The Web Stack: HTTP, TLS, Load Balancing & Caching

Analogy: Think of the whole web stack as a busy restaurant. The CDN/edge is a food truck on your street with popular dishes pre-made (no trip downtown). The reverse proxy is the host at the door who checks your ID and points you the right way. The load balancer seats you at whichever kitchen line is shortest. The kitchens are the backend servers. And statelessness means the kitchen forgets who you are between courses unless you hand back your table number (a cookie).

1. The HTTP request/response model

HTTP (HyperText Transfer Protocol) is a text-based, request/response protocol. "Request/response" means a client (a browser, a phone app, or a command-line tool like curl) sends a request, and the server sends back a response. Crucially, it is client-driven: the server never speaks first. Nothing happens until the client asks. (That one-way limitation is exactly why WebSockets and Server-Sent Events were later invented — to let servers push data.)

A request has four parts: (1) a request line — the method, path, and version; (2) headers — key:value metadata about the request; (3) a blank line; (4) an optional body (form data, JSON, an uploaded file). A response mirrors this with a status line, headers, a blank line, and a body.

REQUEST                          RESPONSE
GET /products/5 HTTP/1.1         HTTP/1.1 200 OK
Host: shop.com                   Content-Type: application/json
Accept: application/json         Content-Length: 48
<blank line>                     <blank line>
                                 { "id": 5, "name": "Mug" }

Methods (also called verbs) tell the server your intent:

• GET — read/fetch a resource (no body).
• POST — create something or submit data.
• PUT — replace a whole resource with what you send.
• PATCH — partially update a resource.
• DELETE — remove a resource.
• HEAD — like GET but returns headers only, no body (handy for "did this change?").
• OPTIONS — ask what the server allows (used in CORS preflight checks).

Status codes — grouped by their first digit

The server's response always starts with a 3-digit code. The first digit tells you the family:

• 1xx informational — e.g. 103 Early Hints.
• 2xx success — 200 OK, 201 Created, 204 No Content.
• 3xx redirection — 301 (moved permanently), 302/307 (temporary), 304 Not Modified (used by caches, explained later).
• 4xx client error — you made a mistake: 400, 401, 403, 404, 409 Conflict, 422, 429 Too Many Requests.
• 5xx server error — the server failed: 500, 502, 503, 504.

2. Statelessness, cookies, sessions & tokens

HTTP is stateless: the server keeps no memory of your previous requests. Each request must carry everything needed to handle it. This sounds like a limitation, but it's the secret to horizontal scaling — running many identical servers side by side. Because no server "remembers" you, any server can handle any request, so you can add more servers freely.

But real apps need to remember things like "I'm logged in." The trick: the client carries an identifier on every request. The most common way is a cookie. The server sends Set-Cookie: session=abc123; the browser stores it and automatically attaches Cookie: session=abc123 to every later request to that site.

Term	What it is	Trade-off
Cookie	A small value the browser stores and auto-sends	Easy, but you must protect it
Session	Server-side data (in Redis/DB) keyed by the cookie's id	Needs shared storage if you have many servers
Token (JWT)	A signed token that carries the user's claims itself	No server lookup, but hard to revoke early

Three cookie flags are essential for security: HttpOnly (JavaScript can't read it, which blocks theft via XSS attacks), Secure (only sent over HTTPS), and SameSite (limits cross-site sending, which blocks CSRF attacks). XSS = injecting malicious scripts into a page; CSRF = tricking your browser into sending a request you didn't mean to.

3. HTTP/1.1 vs HTTP/2 vs HTTP/3

HTTP has evolved to be faster. The first big enemy is head-of-line (HOL) blocking — when one slow item stuck at the front of a line holds up everything behind it (like one slow shopper jamming the only checkout lane).

HTTP/1.1 (1997): text-based; one connection handles one request at a time. Connection: keep-alive lets a connection be reused for several sequential requests (avoiding the cost of re-opening it), but they still go one after another. To get parallelism, browsers cheated by opening about 6 connections per host.

HTTP/2 (2015): same meaning, new wire format. Its wins: (a) binary framing (machine-friendly, not text); (b) multiplexing — many requests (called streams) interleaved over one TCP connection, killing the 6-connection hack; (c) HPACK header compression, which removes repeated header bytes. Server Push existed but is now deprecated (Chrome removed it in 2022) — use 103 Early Hints instead.

HTTP/3 (2022): runs over QUIC, a new transport built on UDP instead of TCP. QUIC does reliability and ordering per stream, so one lost packet only stalls its own stream. It also folds in the TLS handshake (faster setup) and supports connection migration — a phone switching Wi-Fi to cellular keeps the same connection via a Connection ID rather than its IP address.

HTTP/1.1:  [Req A]--wait--[Req B]--wait--[Req C]   one at a time

HTTP/2:    one TCP pipe:  A1 B1 A2 C1 B2 ...   interleaved
           BUT a dropped packet stalls the WHOLE pipe (TCP)

HTTP/3:    lane A | lane B | lane C   independent
           a drop in lane B stalls ONLY lane B

4. REST, APIs & idempotency

REST is a style for designing web APIs. Its rules: resources are named by URLs (/users/42), you act on them with HTTP methods, you transfer representations (usually JSON), and it's stateless. Use nouns in paths and verbs as methods — write GET /users/42, never GET /getUser?id=42.

Two properties drive safe API design:

• Safe = read-only, changes nothing on the server: GET, HEAD, OPTIONS.
• Idempotent = doing it many times has the same effect as doing it once: GET, HEAD, OPTIONS, PUT, DELETE. NOT idempotent: POST, PATCH.

Why care? Because networks fail. If a request times out, you don't know if it succeeded. You can safely retry an idempotent call. Retrying a POST, though, might run it twice.

5. TLS / HTTPS — keeping it private and trustworthy

HTTPS is just HTTP carried inside a TLS-encrypted channel. TLS (Transport Layer Security) gives three guarantees:

• Confidentiality — eavesdroppers see only scrambled ciphertext.
• Integrity — if anyone tampers with the data, it's detected.
• Authentication — you're really talking to the right server, proven by its certificate.

A server presents an X.509 certificate — a document binding its domain name to a public key, signed by a Certificate Authority (CA) that browsers already trust. This forms a chain of trust up to a "root" CA stored in your operating system. Let's Encrypt made these certificates free and automatic.

TLS 1.3 handshake (1 round trip):

Client --ClientHello: ciphers + key share-->        Server
Client <--ServerHello: cert + key share, encrypt now-- Server
        both derive the SAME session key (ECDHE),
        then application data flows. Done in 1 RTT.

The key idea: slow asymmetric crypto (public/private keys) is used only to agree on a shared secret key. Then fast symmetric crypto (AES-GCM, ChaCha20) encrypts the actual bulk data. TLS 1.3 finishes the handshake in one round trip (TLS 1.2 needed two). It also requires forward secrecy (via ephemeral ECDHE keys): even if someone later steals the server's long-term key, they can't decrypt traffic they captured in the past. TLS 1.3 also offers 0-RTT resumption for returning clients (even faster), but it's replay-vulnerable, so use it only for idempotent requests.

6. Latency vs throughput vs bandwidth

These three are constantly confused:

• Latency — time for one trip (delay), measured in milliseconds.
• Throughput — actual work done per second (e.g. requests/sec achieved).
• Bandwidth — the maximum capacity of the pipe (bits/sec).

Latency has a hard floor: the speed of light is ~1ms per ~100km over fiber, so a cross-continent round trip is naturally ~150ms. A rough order-of-magnitude "latency canon" every engineer should feel:

Operation	Rough time
Main memory read	~100 nanoseconds
Same-datacenter round trip	~0.5 ms
SSD random read	~16–20 microseconds
Cross-continent round trip	~150 ms

7. Caching everywhere

Caching means storing a copy of a result closer to whoever needs it, so you avoid redoing the work. It exists at every layer of the stack:

User
  -> [Browser cache]        governed by Cache-Control: max-age
  -> [CDN edge PoP]         governed by Cache-Control: s-maxage
  -> [Reverse proxy/Varnish]
  -> [App + Redis cache]
  -> [DB buffer pool]

HTTP gives you headers to control caching:

• Cache-Control: max-age=N — fresh for N seconds.
• s-maxage — overrides max-age, but only for shared caches like a CDN.
• no-cache — may store, but must revalidate before using.
• no-store — never store at all (for private/sensitive data).
• private — browser only, never a shared cache; public — anyone may cache.

When a cached copy might be stale, the cache revalidates using a validator. An ETag is a fingerprint of the content. The cache asks If-None-Match: "v7"; if nothing changed, the server replies 304 Not Modified with no body — saving bandwidth. The stale-while-revalidate directive serves the old copy instantly while quietly refreshing in the background, hiding the delay from the user.

8. Load balancers

A load balancer (LB) spreads incoming traffic across many backend servers. This gives you scale (more servers = more capacity) and high availability (route around a dead server).

The big distinction is L4 vs L7 (the OSI layer numbers):

	L4 (transport)	L7 (application)
Routes by	IP + port only	URL path, host, header, cookie
Reads payload?	No (payload-blind)	Yes (content-aware)
Speed	Very fast, low CPU	Smarter, more CPU
Can it terminate TLS?	No	Yes
Example	AWS NLB	AWS ALB, Nginx, Envoy

Balancing algorithms: Round Robin (cycle through servers evenly — fine when requests are uniform); Weighted (bigger servers get more); Least Connections (send to the server with the fewest active requests — best when request durations vary); IP Hash / Consistent Hashing (same client always lands on the same server, with minimal reshuffling when servers are added/removed — vital for sharded caches).

Health checks ensure the LB only sends traffic to healthy servers. Active checks probe on a schedule ("does GET /health return 200?"). Passive checks watch real traffic for errors. Use both. Sticky sessions pin a client to one server so its session state stays reachable — but this unbalances load and breaks when that server dies.

9. Reverse proxies, CDNs & the edge

A forward proxy sits in front of clients (e.g. a company's outbound gateway). A reverse proxy sits in front of servers (Nginx, HAProxy, Envoy). The reverse proxy is your single "front door": it terminates TLS, caches, compresses, routes requests, load-balances, and filters attacks. Clients never touch your app servers directly.

A CDN (Content Delivery Network — Cloudflare, Fastly, CloudFront) is a globally distributed network of reverse-proxy caches. Users connect to the nearest edge PoP ("point of presence" — a server cluster near them). A cache hit is served straight from the edge; a miss is fetched from your origin (your main server), cached, then served. Benefits: lower latency (content is physically near users), less load on your origin, DDoS absorption, and TLS handled at the edge. "Edge computing" pushes not just caching but actual code (edge functions) out to those PoPs so even dynamic logic runs close to users.

10. Rate limiting, timeouts, retries & backoff

Rate limiting caps how many requests a client may make in a time window. It protects you from abuse, buggy clients, and overload. Over-limit requests should return 429 Too Many Requests with a Retry-After header telling the client when to try again.

TOKEN BUCKET (allows bursts):
  +1 token / 100ms refills the bucket
  10 saved tokens -> a burst of 10 passes instantly,
  then throttles to the refill rate.

LEAKY BUCKET (smooths bursts):
  pour in fast -> drips out exactly 1 / 100ms,
  a steady constant stream no matter the input.

Other algorithms: Fixed Window (count per clock window — simple, but allows a 2x burst at the boundary), Sliding Window (smooths that boundary problem). Token bucket allows controlled bursts and is the most common API choice.

Timeouts are mandatory. Never wait forever — every network call needs a deadline. Without one, a single hung dependency can use up all your threads/connections and take the whole system down.

Retries must be careful. Only retry idempotent requests (or those with an Idempotency-Key), and only on transient errors (timeouts, 502/503/504, 429-with-Retry-After) — never on permanent client errors like 400/401/404. Use exponential backoff (wait 1s, 2s, 4s, 8s…) plus jitter (randomize the delay).

THUNDERING HERD (no jitter):
  1000 clients fail at t=0, ALL retry at t=1,2,4...
  -> each wave re-crashes the recovering server

WITH JITTER:
  each client picks a random delay in [0, backoff]
  -> retries spread out, the server recovers