"NoSQL" is not one technology. It is a label stuck on a group of databases that, since the late 2000s, made different choices from the relational model. The honest way to learn them is to ask what pressure produced each choice and what was given up in return. Every feature in this section is a trade, not a free improvement.

The best-known origin stories are Google Bigtable and Amazon Dynamo, both described in papers (Dynamo in 2007), which inspired open-source systems such as HBase, Cassandra and Riak. Document stores such as MongoDB and key-value stores such as Redis and DynamoDB (the managed AWS service, which borrows ideas from the Dynamo paper but is a different system) came from the same era. Today most families also offer features once considered "relational only", so the line has blurred.

CAP is the most quoted and most misquoted idea in this field. Stated precisely (Gilbert and Lynch proved the formal version in 2002, after Eric Brewer's conjecture): in a distributed system that stores data on several machines, when a network partition occurs, you must choose between consistency and availability.

Because partitions are a fact of life rather than a feature you can decline, the "pick two of three" slogan misleads. The real statement is: partitions will happen; during one, choose C or A. When the network is healthy you can have both. A single-node database is not distributed, so CAP does not apply to it at all.

Partitions are rare, but latency is constant. Daniel Abadi's PACELC extension says: if there is a Partition, choose Availability or Consistency; Else, when running normally, choose Latency or Consistency. Even with a healthy network, keeping replicas in step costs round trips. A system that waits for every replica before acknowledging a write is consistent and slow; one that acknowledges after one replica is fast and may serve stale reads.

Relational databases popularised ACID. Many NoSQL systems describe themselves with the playful opposite, BASE. These are design philosophies, not exact specifications, and the contrast is easy to overstate.

Two cautions. First, BASE does not mean "no durability": a write that was acknowledged is normally still safely stored (on at least one node, often more). What is relaxed is agreement between replicas, not safety of the write. Second, "ACID" is not binary either: isolation has levels (read committed, snapshot, serializable), and several NoSQL databases do offer ACID transactions within a single item, a single partition or, since recent versions, across documents (chapter 9).

"Consistency" is a spectrum of promises about which value a read may return after writes. The diagram shows the situation that makes the spectrum necessary: replicas receive the same write at different moments, and a read can land on any of them.

Linearizable and eventual are the two ends; the middle models are the useful ones for applications. Read-your-writes and monotonic reads are called session guarantees and fix the cases users actually notice: posting a comment and not seeing it, or refreshing a page and watching it revert.

Replication keeps several copies of each piece of data for durability and availability, and lets reads spread across machines. How the copies are kept in step decides most of a database's behaviour. There are three main shapes.

In a leaderless system with N replicas of each item, a write is considered successful once W replicas acknowledge it, and a read asks R replicas and returns the freshest answer. If **R + W > N**, the set of replicas read must overlap the set written in at least one node, so the read sees the latest acknowledged write. The overlap guarantee is the whole idea.

The quorum overlap is a statement about sets of nodes, and the hands-on in chapter 11 enumerates it exhaustively. In practice R + W > N is a necessary ingredient, not a full guarantee of linearizability. Concurrent writes, sloppy quorums, clock-based conflict resolution and partial write failures can still produce surprises, which is why Cassandra documentation, for example, talks about "strong consistency" with quorum reads and writes under stated conditions rather than as an unconditional guarantee.

Cassandra's architecture documentation describes hinted handoff in the write path and read repair in the read path, following the Dynamo paper. The exact timings, time-to-live of hints and tuning flags are product- and version-specific, so check the documentation for the version you run.

Replication copies the same data to several machines. Partitioning (also called sharding) splits different data across machines so no single machine holds or serves everything. Every scale-out database does both: data is cut into partitions, and each partition is replicated.

Even with a hash, a hot key can overload one partition: a celebrity profile, a viral product, a counter that every request increments, a "current date" key. The database cannot spread a single key over many machines, so the remedy is in the data model: add a random or computed suffix to spread the key (product-7#0 to product-7#9) and sum or merge on read, cache the hot item, or batch counter updates.

The naive scheme, hash(key) mod N, reassigns almost every key when N changes. Consistent hashing places both nodes and keys on a ring; a key belongs to the first node clockwise from it. When a node joins or leaves, only the keys in its arc move.

Not every database uses a literal ring. Fixed-count partition schemes (a database that creates, say, a few hundred logical partitions up front and assigns them to nodes, as Redis Cluster does with 16384 hash slots) give the same property: adding a node moves whole partitions, not individual keys. DynamoDB hides partitioning entirely and splits partitions for you. The concept, not the mechanism, is what carries across products.

A partitioned store finds data fast by its partition key. Asking "all orders with status SHIPPED" is a different question: the answer is scattered across every partition. A secondary index makes such a query possible, and how it is partitioned decides its cost.

DynamoDB offers both under these names: local secondary indexes share the partition key of the table (and need the table's partition key plus a different sort key), while global secondary indexes can have an entirely different key and have their own provisioned or on-demand capacity. Names differ elsewhere, but the trade is the same.

• Every write is amplified: an item with three indexes is written four times (once to the table, once per index).
• Storage grows with each index, and a global index copies the projected attributes.
• An index can lag: a global index that updates asynchronously can show a stale view, so never use it to check a uniqueness rule.
• Low-cardinality indexes help little: indexing a boolean returns half the table.

This is the biggest mental shift from relational design. In a relational database you model the data (entities, normal forms) and write whatever queries you later need. In most NoSQL databases you list the queries first and shape the data so each query reads one place.

A schemaless store does not remove the schema, it moves it into the application code. Every reader must cope with every shape that ever existed. Practise it deliberately:

• Put a **schemaVersion** field on every item, and let readers upgrade old shapes on read (and optionally write them back).
• Make new fields optional with defaults; never rely on a field being present in old items.
• Prefer additive changes; for renames and splits, write both shapes for a while, then backfill, then drop the old one.
• Where the database supports it, add validation (MongoDB supports JSON Schema validation on collections) to catch typos, because a store that accepts anything also accepts custmerId.

Two users edit the same item. Without protection, the second write silently overwrites the first (a lost update). The standard fix needs no locks: every item carries a version, and a write succeeds only if the version is still the one the writer read.

In a Spring application with MongoDB or JPA this appears as an @Version field and an optimistic-locking exception to catch and retry; in DynamoDB it is a ConditionExpression on the version attribute. For the same idea in relational databases see the SQL section; for the transaction boundaries on the Spring side see Spring: Transactions.

A distributed database needs to decide which of two writes to the same item is "newer". Wall clocks are the obvious tool and the dangerous one: clocks on different machines drift and jump, even with synchronisation, so "later" by clock is not "later" in reality.

Scale-out databases make cost visible in terms of what you ask, not just what you store. Do the arithmetic before choosing.

Worked example (illustrative numbers): 5,000 writes per second of 2 KB items is about 10 MB per second of ingest, roughly 860 GB per day before replication; with N = 3 copies that is about 2.6 TB per day of raw storage and three times the network traffic inside the cluster. If each item also has two global secondary indexes, writes become three physical writes before replication. Numbers like these quickly separate "a single PostgreSQL" from "a partitioned store" and show how fast storage grows. Always compute the volume before arguing about brands.

Reading R + W > N is easy to nod at and hard to feel. This small program models N = 3 replicas of one item. Each replica holds a versioned value. A writer updates W replicas to version 2 (the others have not received it yet), then a reader asks R replicas and returns the highest version it sees. The program tries every possible choice of which replicas were written and which were read, and counts the reads that returned the stale value.

Compile and run it with javac QuorumSim.java then java QuorumSim. The output below is the real output of that run:

Notice what the program does not model, because the gaps are the lesson: it has no failures during a write (a write that reached only some replicas but was never acknowledged), no concurrent writers, no clocks, and no repair. Each of those is a place where a real system can still surprise you, and where hinted handoff, read repair and version vectors earn their keep.

Crisp answers for the questions that come up in interviews on this topic.

• Q: State the CAP theorem precisely.
  In a replicated system, when a network partition occurs, you must choose between consistency (linearizability: every read sees the latest write) and availability (every request to a live node gets a non-error answer). Partition tolerance is not a free choice, since networks fail.
• Q: Why is "CA" not a real option for a distributed database?
  A system that gives up partition tolerance has to stop working when the network splits, which in practice means it is not distributed (a single node) or it is not actually available during partitions. Partitions happen whether you plan for them or not.
• Q: What does PACELC add?
  Even without a partition there is a trade between latency and consistency: waiting for more replicas to acknowledge makes writes and reads slower but more consistent. It describes the everyday behaviour that CAP ignores.
• Q: Compare ACID and BASE.
  ACID gives atomic, isolated, durable transactions with enforced constraints, at the cost of coordination. BASE (basically available, soft state, eventually consistent) accepts temporary disagreement between replicas for availability and speed. The choice is per piece of data, and many databases offer both within limits.
• Q: What is eventual consistency, and what can a user see?
  If writes stop, all replicas converge. Meanwhile reads can return old values, go backwards in time, or miss a user's own write. Session guarantees (read-your-writes, monotonic reads) remove the most visible anomalies.
• Q: Explain R + W > N.
  With N replicas, a write acknowledged by W and a read from R replicas must overlap in at least one replica when R + W is greater than N, so the read sees the latest acknowledged write. It is necessary for strong reads in a leaderless system but not sufficient against concurrent or partly failed writes.
• Q: What are hinted handoff and read repair?
  Hinted handoff: another node holds a write meant for a temporarily down replica and hands it over when it returns. Read repair: a read that finds replicas disagreeing writes the freshest value back to the stale ones. Both reduce divergence; anti-entropy repair covers data that is never read.
• Q: Hash or range partitioning?
  Hash for even load and point lookups; range for ordered scans but with hot-spot risk on sequential keys. A common compromise is a hashed partition key with a sorted key inside the partition.
• Q: What is consistent hashing and why use virtual nodes?
  Nodes and keys are placed on a ring and a key belongs to the next node clockwise, so adding or removing a node moves only about 1/N of the keys. Virtual nodes give each machine many ring positions, which evens out load and spreads rebuild work across peers.
• Q: Local vs global secondary index?
  A local index lives with each partition and is cheap to write but a query by the indexed value must ask every partition. A global index is partitioned by the indexed value, so a lookup is fast, but writes are costlier and the index is often eventually consistent.
• Q: How do you model one-to-many in a document or key-value store?
  Embed when the children are few, bounded and read with the parent. Reference when they are many or shared: separate items sharing the parent's partition key and sorted by a sort key, so a single range query returns them.
• Q: How do you prevent lost updates without transactions?
  Optimistic concurrency: keep a version on the item and make the write conditional on the version read. A failed condition means someone else wrote first, so re-read and retry.
• Q: Why is last-write-wins dangerous?
  Clocks drift, so the "latest" timestamp may not be the latest event, and concurrent updates are silently discarded. It is acceptable only for idempotent whole-value overwrites. Otherwise use versions, version vectors with merge logic, or CRDTs.
• Q: When would you choose a relational database over NoSQL?
  When queries are unknown or ad-hoc, when multi-row transactions and constraints are central, when data is highly relational, or when it fits on one server. NoSQL is justified by a measured need: scale, latency, flexible schema, or a specific data shape.
• Q: How would you migrate a service from a relational database to a NoSQL store safely?
  Strangle one slice at a time, load history by backfill, keep stores in sync with change data capture or an outbox rather than naive dual writes, compare results in shadow reads, cut reads over first and writes next, and keep a rollback path.