Relational Databases
ACID guarantees, indexing, and scaling the workhorse of most systems.
What Relational Databases Are
Relational databases have been the default choice for persistent storage for half a century. Not because engineers lack imagination — the industry has tried alternatives several times per decade — but because relational databases solve the hardest problem in data management (correctness under concurrency) better than any of those alternatives. The model is simple: data lives in tables of rows and columns; relationships between tables are expressed through foreign keys; queries are written in a declarative language called SQL.
What makes them trustworthy is not the table model. It is the four-letter promise: ACID. Atomicity, consistency, isolation, and durability are the guarantees that let you charge a credit card, transfer money, or book a seat without the system silently corrupting itself under load. Newer database categories trade pieces of ACID for scale or flexibility — sometimes worth it, often not.
A — Atomicity
Definition: all operations in a transaction succeed or none do.
Without it: a crash mid-transfer leaves money debited from one account but never credited to the other.
Example: bank transfer — the debit and credit are one atomic unit.
C — Consistency
Definition: every transaction leaves the database in a valid state per all defined rules.
Without it: foreign key violations, broken invariants, orphaned rows.
Example: an order row cannot reference a non-existent customer.
I — Isolation
Definition: concurrent transactions don't see each other's partial work.
Without it: double-spends, lost updates, dirty reads under concurrency.
Example: two users buying the last seat — only one transaction commits.
D — Durability
Definition: once committed, data survives crashes, power loss, restarts.
Without it: “your order was confirmed” turns into “sorry, we lost it.”
Example: WAL (write-ahead log) flushed to disk before COMMIT returns.
The mental model: a relational database is a correctness machine wearing a query language as a coat. SQL is the syntax; ACID is the substance. The reason this category has not been displaced in 50 years is that almost every alternative trades a piece of ACID for some other property, and most applications care about correctness more than they admit.
- Relational databases store data in tables and express relationships through foreign keys.
- SQL is the declarative query language — what you want, not how to get it.
- ACID — atomicity, consistency, isolation, durability — is what makes the category trustworthy.
- Real implementations: PostgreSQL, MySQL, SQLite, Oracle, SQL Server, Aurora, CockroachDB.
How They Work Internally
Underneath the SQL, a relational database is a careful dance between memory and disk. Data lives in fixed-size pages (typically 8 KB in PostgreSQL, 16 KB in MySQL/InnoDB). Pages are loaded into a shared buffer pool; queries operate on those in-memory pages. The trick that makes durability work is the write-ahead log — every change is appended to a sequential log file before the modified page is written back to its data file. On crash, replay the log; the database is consistent again. This is also the foundation of replication: streaming the WAL is how replicas stay in sync.
Indexes are the difference between a query that runs in 1 ms and one that runs in 30 seconds. The default in every mainstream database is the B-Tree — a balanced tree where every leaf is at the same depth, giving O(log n) lookups and efficient range scans. Hash indexes exist, are O(1) for equality, and are nearly useless for anything else.
B-Tree Index
Structure: balanced tree, sorted keys at every node, all leaves equidistant.
Use case: equality, range scans, prefix matches, ORDER BY.
Example query: WHERE created_at > '2026-01-01' — range scan walks adjacent leaves.
Limitation: write amplification — every INSERT/UPDATE on indexed columns updates the tree.
Hash Index
Structure: hash table mapping key → row pointer.
Use case: equality only, when you need maximum speed for exact-match lookups.
Example query: WHERE user_id = 12345.
Limitation: no range queries, no ordering, no prefix matching. PostgreSQL has hash indexes; you almost never need them.
Two more index concepts pay outsize dividends in production. Composite indexes cover multiple columns in one structure, with the leftmost-prefix rule: an index on (tenant_id, created_at) can serve queries on tenant_id alone or on both columns, but not on created_at alone. Covering indexes include all columns the query needs, so the engine never has to visit the table itself — index-only scans are dramatically faster than index-then-heap lookups.
Every query goes through three phases: parsed into a syntax tree, planned by the optimiser (which considers many possible execution paths and picks one based on collected statistics), and executed. The planner is the part that surprises engineers. Two queries that look identical can have wildly different plans depending on data distribution — a small table gets a sequential scan, a large one gets an index scan. EXPLAIN shows you the chosen plan; EXPLAIN ANALYZE runs the query and shows actual timings. Read these. Profile slow queries with them. They tell you the truth.
An index is a trade-off. It speeds up reads at the cost of write performance and storage. Index what you query — not everything. The cost of a wrong index is rarely visible in benchmarks; the cost of a missing one is a 30-second query in production.
- Pages are the storage unit; the WAL is the durability mechanism and the foundation of replication.
- B-Tree indexes are the default — O(log n), good for equality and ranges. Hash indexes are equality-only and rarely needed.
- Composite indexes follow leftmost-prefix; covering indexes avoid heap lookups entirely.
- Use
EXPLAIN ANALYZEto see what the planner actually does — never guess query plans.
When to Use — and When Not To
The most expensive technical decision I see junior architects make is choosing “something else” over a relational database for vague reasons. “It won't scale.” “Schema is too rigid.” “Mongo is more modern.” Most of these reasons evaporate on contact with actual numbers. PostgreSQL on a single beefy server handles tens of thousands of QPS for years before you need to think about anything more exotic. The cases where you genuinely need to move away from relational are real, but they are specific and measurable.
USE Relational When…
Data has clear relationships. Customers, orders, line items, products — the canonical tabular model.
Transactions matter. Money, inventory, bookings, anything where partial failure is unacceptable.
Complex queries with joins. Reporting, analytics, ad-hoc business questions.
Schema is relatively stable. Migrations are real work, but they're solved work.
Team knows SQL. Almost universal — lower training cost than learning a query DSL.
DO NOT Use When…
Data has no tabular shape. Tree-shaped documents, graphs, time-series at scale — force-fitting them into rows is painful.
Extreme write throughput. Sustained millions of writes per second — relational caps out long before this on a single primary.
Schema churns daily. Early-stage product where every week brings new fields. Document stores are kinder here.
Massive horizontal sharding from day one. If you genuinely need to start with 50 shards, the operational shape is different.
Pure unstructured blobs. Use object storage, not the database, for files.
Default to Relational
The principle: most applications should start with a relational database. The reasons to move away are specific and measurable — not vague feelings about scale.
Test before you migrate: if your DB is the bottleneck, profile first. 80% of the time the answer is missing indexes, N+1 queries, or unnecessary joins — not “we need NoSQL.”
The JSON Escape Hatch
PostgreSQL JSONB: indexed binary JSON columns let you store semi-structured data inside a relational row. You get rigid schema where it matters, flexible documents where it doesn't.
When useful: per-tenant custom fields, audit logs, event payloads. Far more often than you'd think.
Most applications should start with a relational database. The reasons to move away from one are specific and measurable — not vague feelings about scale. “PostgreSQL won't handle this” said before measuring is the most expensive sentence in modern systems engineering.
- Relational databases excel where data has relationships, transactions matter, and complex queries are needed.
- Move away from them only for measurable reasons: extreme write throughput, naturally non-tabular data, or schema that churns daily.
- JSONB columns in PostgreSQL bridge the gap between rigid schema and flexible documents.
- Most “we need NoSQL” conversations end after a profiling session reveals a missing index.
Trade-offs & Comparisons
Database connections are not free. Each PostgreSQL connection consumes around 5 MB of server memory and a backend process. A web app with 200 concurrent requests and a naive “one connection per request” pattern will exhaust your database before it exhausts anything else. Connection pooling — reusing a fixed set of connections across many requests — is non-negotiable in production.
The pool size formula that most teams get wrong: pool_size ≈ cpu_cores × 2 on the database server, not on the app server. A pool of 200 connections does not give you 200× throughput; it gives you contention. The database can only do so much real work in parallel; the pool should match that capacity. Use PgBouncer (PostgreSQL) or HikariCP (Java) and size the pool to the database, not to your app.
You ask the ORM for 100 users. Then for each user, you access their orders. The ORM dutifully runs 1 + 100 = 101 queries instead of one JOIN. Multiplied by every endpoint and every developer, you have a database doing tens of thousands of queries when it should be doing hundreds. The fix is eager loading (JOIN, preload, includes, depending on the ORM). The detection is your slow query log — or better, an APM tool that shows query counts per request.
When the primary is bottlenecked on reads, the standard answer is to add replicas: the primary handles all writes; replicas serve reads. This works because most production workloads are read-heavy (5:1 to 100:1). The catch: replication lag. Replicas are eventually consistent — usually milliseconds behind, occasionally seconds, rarely minutes during heavy load. The classic bug is the read-your-own-writes problem: a user creates a post, immediately reads from a replica, and sees nothing.
The standard fix: route reads to the primary for a short window (5–30 seconds) after any write from the same user, then go back to replicas. Some systems use replication slots and committed-LSN tracking to be more precise, but the simple time-based heuristic catches the vast majority of cases.
PostgreSQL
Strengths: SQL standards compliance, complex queries (CTEs, window functions, lateral joins), JSONB, partial & expression indexes, mature analytical features, extensions (PostGIS, pgvector, TimescaleDB).
Weaknesses: slightly more complex to operate, vacuum tuning required at scale, replication setup more involved than MySQL.
Use: default for new projects in 2026.
MySQL / MariaDB
Strengths: simpler operational model, slightly faster simple-read workloads, near-universal hosting support, InnoDB engine is rock-solid, easier replication out of the box.
Weaknesses: weaker SQL support (no array types, limited window functions until recently), JSON less powerful than JSONB.
Use: when hosting/operational constraints make it the practical choice, or for legacy compatibility.
N+1 queries are silent killers. Your ORM is generating 500 queries for what should be one. Always check your slow query logs and your APM query counts — the worst latency bug in your system is almost certainly an N+1 you haven't found yet.
- Connection pools are mandatory; size them to the database's real capacity, not your app's concurrency.
- N+1 queries are the most common cause of database overload — eager-load and audit query counts.
- Read replicas scale reads but introduce replication lag — route post-write reads to the primary briefly.
- PostgreSQL is the default for new projects in 2026; MySQL when operational constraints demand it.
Production Patterns & Common Mistakes
The single most dangerous thing you can do to a production relational database is run a destructive migration in one step. Dropping a column while the old code still reads it crashes your service; adding a NOT NULL column without a default crashes inserts during deploy; renaming a table breaks every replica. Zero-downtime migrations follow a discipline: every change is split into stages where the schema and the code are simultaneously correct.
Pattern: Add a Column Safely
Step 1: ALTER TABLE ADD COLUMN as nullable, with no default. Fast metadata change.
Step 2: deploy code that writes both old and new columns; reads ignore new.
Step 3: backfill the new column in batches (avoid table-wide UPDATE).
Step 4: deploy code that reads the new column.
Step 5: add NOT NULL constraint after backfill is verified.
Pattern: Drop a Column Safely
Step 1: deploy code that no longer reads or writes the column.
Step 2: wait at least one full release cycle. No code path touches the column.
Step 3: ALTER TABLE DROP COLUMN.
Step 4: clean up application code references in the next release.
Tools: pgmigrate, Flyway, Liquibase, gh-ost, pt-online-schema-change.
Most production systems use soft deletes: a deleted_at timestamp marks a row as deleted instead of physically removing it. This gives you audit trails, recovery, and foreign-key integrity. The cost: every query must filter WHERE deleted_at IS NULL, and forgetting that filter shows deleted data to users. The non-obvious requirement: a partial index on WHERE deleted_at IS NULL — otherwise the planner scans deleted rows on every query and your indexes get bigger and slower over time.
Mistake 1 — No Indexes on Foreign Keys
JOINs do full table scans on the “many” side. Catastrophic at scale; ON DELETE CASCADE becomes an outage. Fix: index every foreign key column. Most DBs do not do this automatically.
Mistake 2 — SELECT * in Production
Fetches columns you don't need; defeats covering indexes; breaks when schema evolves. Fix: select specific columns, always. Lint for SELECT * in code review.
Mistake 3 — Long Transactions
Hold locks; block other transactions; bloat MVCC; prevent vacuum. Fix: keep transactions short. Never wrap external API calls or user input loops inside one.
Mistake 4 — Files / Blobs in DB
Database size explodes; backups become unmanageable; vacuum slows. Fix: store files in object storage (S3); store the URL in the DB.
Mistake 5 — No Slow Query Monitoring
Bad queries hide for months until traffic doubles. Fix: set log_min_duration_statement (e.g. 100 ms). Review the slow query log weekly. Alarm on regressions.
Bonus — No Backup Restoration Drills
You have backups. You have never restored one. The first time will be the worst. Fix: quarterly drill: restore to a fresh instance from the latest backup. Time it. Treat results as a real metric.
The relational database is the most-tested, best-understood persistent store in software engineering. Almost all production failures come from violating well-known patterns: indexing too little, transactions too long, schema migrations too eager, slow queries unmonitored. Discipline beats cleverness.
- Schema migrations are multi-step processes — never destructive in one deploy.
- Soft deletes are useful with audit trails but require partial indexes on
deleted_at IS NULL. - Index foreign keys, never
SELECT *, keep transactions short, store files in object storage, monitor slow queries. - Test your backups by restoring them. The drill is the only thing that proves they work.