NoSQL · 04

MongoDB Documents at Scale

MongoDB stores data in the shape your application uses. This page explains the document model and BSON, how to decide between embedding and referencing, the schema design patterns, how indexes and the ESR guideline decide query speed, how to read explain output (with a real run), the aggregation pipeline, replica sets with write and read concern, sharding and shard key choice, transactions, change streams, validation, Spring Data MongoDB and a Docker lab.

13
Chapters
8.x
MongoDB line
ESR
Index rule
1
Docker lab
01
Chapter 1 - The Model

Documents, BSON, Collections and ObjectId

MongoDB is a document database. The unit of storage is a document: an ordered set of field/value pairs, written in a JSON-like form and stored in a binary format called BSON. Documents live in collections (roughly tables), and collections live in databases. Unlike a table row, a document can contain arrays and nested documents, so data that a relational design would spread over several tables can be stored, read and written as one unit.

One order as a document: nested, typed, self-describing
orders collection (one document) _id: ObjectId("6ae6...") customerId: 42 status: "shipped" total: Decimal128("19.90") createdAt: ISODate(...) address: { city, zip } lines: [ {sku, qty}, {sku, qty} ] BSON Binary on the wire and on disk length + typed fields + terminator Why BSON and not plain JSON: More types: date, int32/64, double, Decimal128, ObjectId Length-prefixed, so the server can skip fields quickly Field names are stored in every document One document is the unit of atomicity and the unit of I/O Hard limit: one document is at most 16 MiB

This page targets the MongoDB 8.x line. Where a fact is version-sensitive it says so. All facts were checked against the MongoDB manual on 2026-10-31; the hands-on output comes from a real server run described in chapter 11.

Relational termMongoDB termNotes
DatabaseDatabaseA namespace for collections; also the unit of access control
TableCollectionNo fixed columns by default; optional validation rules (chapter 10)
RowDocumentNested objects and arrays allowed; at most 16 MiB (BSON document size limit)
ColumnFieldDifferent documents in one collection can have different fields
Primary key_idRequired, unique and immutable; an index on it always exists
JOIN$lookup or embeddingPrefer embedding data that is read together (chapter 3)
TransactionSingle-document atomicity, or multi-document transactionChapter 10
BSON types you will actually use
TypeExample / aliasGotcha
String"shipped"UTF-8; comparisons are binary unless you set a collation
Int32 / Int64 / DoubleNumberInt(5), NumberLong(5), 5.5In mongosh a plain number literal is a double, which can surprise $jsonSchema bsonType: "int" checks
Decimal128NumberDecimal("19.90")Use for money; doubles cannot represent 0.1 exactly
DateISODate("2026-10-31T10:00:00Z")Stored as milliseconds since the epoch, always UTC. Store dates as Date, not strings
ObjectIdObjectId()12 bytes, explained below
Boolean, Nulltrue, nullA missing field and a field set to null are different to queries and to sparse indexes
Array["vip", "beta"]Queries match if any element matches; indexing an array makes a multikey index (chapter 4)
Embedded document{ city: "Pune", zip: "411001" }Query nested fields with dot notation: "address.city"
BinData, UUIDUUID("...")Drivers differ in the legacy UUID representation; set it explicitly in your driver
What is inside an ObjectId

If you do not supply _id, the driver generates an ObjectId on the client before sending the insert. It is 12 bytes: a 4-byte timestamp (seconds since the Unix epoch), a 5-byte random value generated once per client process, and a 3-byte incrementing counter initialised to a random value. That makes ids unique without a central sequence, roughly ordered by creation time, and cheap to generate in any application instance.

mongosh - an ObjectId and the time hidden inside it (real run, MongoDB 7.0.45 shell)
> const oid = new ObjectId() > oid.toString() 6ae6297bb6687623238282f5 > oid.getTimestamp().toISOString() 2026-10-31T17:46:35.000Z
  • The first 4 bytes (6ae6297b) are the timestamp, the next 5 identify the process, the last 3 are the counter. Ids from one process increase, ids across processes are only ordered to the second.
  • _id can be any immutable unique value: a string, a number, a UUID, a compound document. A natural key (an order number, an email) is fine if it never changes. You cannot update _id.
  • Do not treat the timestamp inside an ObjectId as your business "created at". Store a real createdAt field if you query by it.
⚠ Treating MongoDB as schemaless

"Schemaless" really means "the database does not force a schema by default". Your application still has one, and every code path that reads the collection depends on it. Without validation (chapter 10) and a versioning habit, a collection ends up with five spellings of the same field and every reader needs defensive code. Decide the shape, validate it, and evolve it deliberately.

Key insight: A relational row is one cell of a spreadsheet; the entity (an order) is rebuilt by joining sheets. A MongoDB document is the whole order folder, with its lines and address inside. You read the folder in one motion. The price is that the folder has a size limit and that data shared between folders must be copied or referenced.
02
Chapter 2 - CRUD

Working with Documents in mongosh

mongosh is the MongoDB shell, a JavaScript environment. Every database operation is a method on a collection. The examples below were run against a real server (see chapter 11); shop is the database and orders and users the collections. Collections and databases are created implicitly on first write.

OperationMethodsNotes
CreateinsertOne, insertManyinsertMany is ordered by default: the first error stops the batch. Pass {ordered: false} to continue
Readfind, findOne, countDocumentsfind returns a cursor; the shell prints the first 20
UpdateupdateOne, updateMany, replaceOne, findOneAndUpdateUpdates use operators ($set, $inc, $push), which are atomic per document
DeletedeleteOne, deleteManyA filter of {} matches everything. Check your filter with find first
Create and read
mongosh - insert, query, project, sort, paginate
use shop db.users.insertOne({ email: "a@x.io", name: "Ada", tags: ["vip"], address: { city: "Pune" } }) db.users.find({ "address.city": "Pune" }) // dot notation for nested fields db.users.find({ tags: "vip" }) // matches if ANY array element equals "vip" db.users.find({ tags: { $in: ["vip", "beta"] } }) db.orders.find({ total: { $gte: 100 }, status: { $ne: "cancelled" } }) db.orders.find({ $or: [{ status: "new" }, { total: { $gt: 500 } }] }) // projection (1 = include, 0 = exclude), sort, skip/limit db.orders.find({ customerId: 42 }, { _id: 0, status: 1, total: 1 }) .sort({ createdAt: -1 }).limit(10)

Filters combine with an implicit AND. The most useful operators are the comparison family ($eq $ne $gt $gte $lt $lte $in $nin), logical ($and $or $not $nor), element ($exists, $type), array ($all, $elemMatch, $size) and $regex. Large skip values are slow because the server still walks past the skipped entries; paginate with a range on an indexed field instead (createdAt < lastSeen).

Update with operators
mongosh - update operators (the result object below is from a real run)
db.users.updateOne({ email: "a@x.io" }, { $push: { tags: "beta" }, $set: { plan: "pro" }, $inc: { logins: 1 } }) // { acknowledged: true, matchedCount: 1, modifiedCount: 1, upsertedCount: 0 } db.users.findOne({ tags: "beta" }, { _id: 0 }) // { email: 'a@x.io', name: 'Ada', tags: [ 'vip', 'beta' ], plan: 'pro' } // upsert: update if it exists, insert if not (use a unique index to make it race-safe) db.counters.updateOne({ _id: "page-views" }, { $inc: { n: 1 } }, { upsert: true }) // update with an aggregation pipeline (computed from other fields) db.orders.updateMany({ status: "paid" }, [ { $set: { totalWithTax: { $multiply: ["$total", 1.18] } } } ])
OperatorDoesOperatorDoes
$set / $unsetSet or remove a field$push / $addToSetAppend / append if absent
$inc / $mulAdd / multiply atomically$pull / $popRemove matching / first or last element
$min / $maxKeep the lower / higher value$renameRename a field
$currentDateSet a Date or timestamp$[<id>] positional filtersUpdate specific array elements
Atomicity you can rely on
  • A write to one document is atomic, including all of its nested fields and array elements. $inc is a safe counter without a transaction. This is why good schema design (chapter 3) keeps data that must change together inside one document.
Write concern in one paragraph

Every write has a write concern: how many replica set members must acknowledge it before the call returns. The implicit default is w: "majority" (with one arbiter-related exception, chapter 8). You can override it per operation, for example db.orders.insertOne(doc, {writeConcern: {w: 1}}) for a fire-and-mostly-forget write, or {w: "majority", j: true} to be explicit. Chapter 8 explains what these mean and what can still be lost.

⚠ Updating without a filter safeguard
  • updateMany({}, ...) and deleteMany({}) touch every document. Run the same filter through find(...).count() or countDocuments(filter) first, and in applications build filters from validated parameters. Unvalidated user input placed directly in a filter lets a client send operators ({"$ne": null}) instead of values: this is NoSQL injection.
03
Chapter 3 - Schema Design

Embed or Reference

In a relational database you normalise first and then write queries. In MongoDB you start from the queries and updates the application performs and shape documents to serve them. The central decision is for each relationship: embed the related data inside the parent document, or reference it by storing its _id.

Same data, two shapes: embedded versus referenced
Embedded: one read returns everything order { _id, status, address: { city, zip }, lines: [ {sku, qty, price}, {sku, qty, price} ] } 1 query, 1 document, atomic update Grows with the array: keep it bounded Data duplicated if shared with others Referenced: two reads or a $lookup orders { _id: 7, customerId: 42 } customers { _id: 42, name, plan } ref Shared data lives once Unbounded children stay in their own collection and can be indexed alone Cost: extra round trip or $lookup Rule of thumb: embed what is read together and bounded in size. Reference what is shared, large, unbounded or updated independently.
QuestionPoints to embeddingPoints to referencing
Are they always read together?Yes (order and its lines)No, the child is often read alone
How many children?Few and bounded (an address, up to dozens of lines)Many or unbounded (all orders of a customer, all comments of a post)
Does the child change independently?RarelyOften, or it is updated by another service
Is the child shared by many parents?No, it belongs to one parentYes (a product referenced by many orders)
Do you need atomic updates across both?Yes: one document is atomic for freeNeeds a multi-document transaction
Size of the wholeStays far below the 16 MiB limitParent would grow large, slowing every read and write of it
Is the child queried on its own?RarelyOften, with its own filters and indexes

The three classic relationship shapes map to these answers: one-to-one and one-to-few usually embed; one-to-many (thousands) references from the many side, or embeds a small summary (extended reference, below); many-to-many stores arrays of ids on one or both sides.

mongosh - the same order, embedded lines and a referenced customer
// Shape: embed what belongs to the order, reference what is shared db.orders.insertOne({ _id: 7, customerId: 42, // reference: customers are shared, updated on their own status: "new", shipTo: { city: "Pune", zip: "411001" }, // embedded: belongs to this order lines: [ { sku: "A-1", qty: 2, price: Decimal128("9.95") }, { sku: "B-9", qty: 1, price: Decimal128("5.00") } ], createdAt: new Date() }) // join on demand with an aggregation $lookup (chapter 7) db.orders.aggregate([ { $match: { _id: 7 } }, { $lookup: { from: "customers", localField: "customerId", foreignField: "_id", as: "customer" } }, { $unwind: "$customer" } ])
⚠ The unbounded array
  • Embedding "all comments", "all events" or "all orders" inside a parent document looks convenient until the array reaches thousands of elements. Documents are rewritten and cached whole, so a hot array makes every update heavier, and the 16 MiB limit is a hard stop that fails the write. If a list can grow without a known cap, give its elements their own collection (or use the bucket pattern, chapter 4).
⚠ Copying everything because "joins are slow"

Denormalising a field that changes (a customer name copied into every order) buys read speed at the cost of fan-out writes and stale data. Copy values that are historical facts (the price and name at purchase time) freely; copy values that must stay current only with a plan to update every copy.

04
Chapter 4 - Schema Design

Design Patterns for Real Workloads

MongoDB published a series of blog posts called Building with Patterns, and the manual now documents several of them under Schema Design Patterns. They are named solutions to recurring problems. Know the name, the problem and the trade-off; interviewers ask for exactly these.

PatternProblem it solvesHowTrade-off
BucketHuge numbers of small records (events, readings, reviews) and slow paginationGroup a series into one document per time span or fixed count, with a capSlightly more complex writes. Time series collections apply this automatically and suit most cases
SubsetThe working set is too big because documents carry rarely used dataKeep the hot part (latest 10 reviews) in the main document, the rest in another collectionDuplicated data; two places to update
Extended referenceFrequent $lookup just to show a name or cityCopy the few fields you always display into the referencing document, keep the idCopies go stale if the source changes; update them, or accept staleness
ComputedThe same expensive calculation (sum, average, count) on every readCompute on write or periodically, store the result next to the dataExtra write work; results can lag
OutlierA few documents are far bigger than the rest (a viral book with 1M reviews)Flag the outlier and move its overflow to a separate document or collectionApplication logic for two paths
ApproximationExactness is not worth a write per event (page views)Update only every N events or by samplingNot precise, by design
ArchiveOld data slows hot queries and fills expensive storageMove cold documents to another collection or storage tierQueries must know where to look
Schema versioningDocuments of several shapes coexist during a migrationAdd a schemaVersion field and handle each version in codeCode carries old versions until migrated
Bucket pattern: many small events become one document per hour
Before: one document per reading r1 r2 r3 r4 r5 r6 3600 documents per sensor per hour, each with its own _id, index entries and overhead group by sensor + hour After: one bucket document { sensorId: 7, start: ISODate(10:00), count: 3600, sum: 81234, readings: [ {t, v}, {t, v}, ... ] } Cap the bucket (by count or time span). Fewer documents, smaller indexes, cheap range reads and pre-computed totals. For plain time series data use a native time series collection (it buckets for you).
Extended reference and computed, in code
mongosh - extended reference: copy only what the order list needs
db.orders.insertOne({ _id: 8, customer: { _id: 42, name: "Ada", city: "Pune" }, // reference id PLUS the fields shown in lists status: "paid", total: Decimal128("24.90") }) // Order list screens now need no join. If a customer is renamed: db.orders.updateMany({ "customer._id": 42 }, { $set: { "customer.name": "Ada L." } })
mongosh - computed pattern: keep running totals with $inc on write
db.stats.updateOne({ _id: "2026-10-31" }, { $inc: { orders: 1, revenue: 24.90 } }, { upsert: true }) // Reading today's totals is one document lookup instead of an aggregation over all orders.
Patterns follow access patterns
  • Pick a pattern because a query is slow, a document is too big, or a hot path needs a join, not because the name sounds good. Measure with explain (chapter 6) before and after.
05
Chapter 5 - Indexes

Single, Compound, Multikey and Friends

Without an index MongoDB reads every document of the collection (a collection scan, COLLSCAN). An index is a B-tree of key values pointing to documents, kept sorted so equality, range and sort queries can walk a slice. Indexes cost memory and slow every write (each insert must update each index), so you want the few that serve your real queries. A collection can have at most 64 indexes and a compound index at most 32 fields.

Index typeCreate withUse it forNotes
Single fieldcreateIndex({email: 1})Lookups and sorts on one fieldDirection matters little for a single field
CompoundcreateIndex({customerId: 1, status: 1, createdAt: -1})Queries on several fields; a prefix of the keys is also usableKey order is everything; see ESR below
Multikeyautomatic on an array fieldQuery array elements (tags: "vip")One entry per element; a compound index may include at most one array field
Unique{unique: true}Enforce uniqueness; make upserts safeDuplicate insert fails with error code 11000 (verified in the lab)
Partial{partialFilterExpression: {status: "open"}}Index only a subset of documentsSmaller and cheaper; the query filter must imply the expression to use it
Sparse{sparse: true}Skip documents lacking the fieldPartial indexes are the more expressive tool and are generally preferred
TTL{expireAfterSeconds: 3600} on a Date fieldAuto-delete sessions, logs, tokensA background task deletes expired documents, so deletion is not instant
TextcreateIndex({body: "text"})Basic $text word searchOnly one text index per collection; for rich search use a search engine (page 05)
Hashed / geospatial / wildcard{id: "hashed"}, 2dsphere, {"$**": 1}Hashed shard keys, geo queries, unpredictable field namesWildcard indexes are a tool for flexible shapes, not a substitute for designed indexes
The ESR guideline

For a compound index MongoDB recommends ordering keys by Equality, Sort, Range (the diagram below). Equality fields first keep the remaining keys sorted; then the sort field so the index delivers order; then range fields. It is a guideline, not a law: always confirm with explain.

ESR: order compound index keys as Equality, Sort, Range
Query: customerId = 42, status = "shipped", sort createdAt desc, total >= 50 Index { customerId: 1, status: 1, createdAt: -1, total: 1 } customerId Equality status Equality createdAt Sort total Range then fetch Why this order works Equality keys first: the index narrows to one contiguous slice of entries. Inside that slice entries are already ordered by createdAt, so the sort is free (no SORT stage). A range placed before the sort would split the slice into pieces that are not globally sorted, forcing an in-memory sort. Range last filters the remaining entries without breaking order. Result: fewer keys examined, no in-memory sort, often a covered query (projection only on index fields).
mongosh - indexes for the ESR example, partial, TTL and unique
db.orders.createIndex({ customerId: 1, status: 1, createdAt: -1 }) // E, E, S db.orders.createIndex({ createdAt: 1 }, { partialFilterExpression: { status: "open" } }) db.sessions.createIndex({ lastSeen: 1 }, { expireAfterSeconds: 1800 }) // TTL, 30 minutes db.users.createIndex({ email: 1 }, { unique: true }) db.users.insertOne({ email: "a@x.io" }) // second time: E11000 duplicate key error (code 11000) db.orders.getIndexes() db.orders.dropIndex("customerId_1_status_1_createdAt_-1")
Covered queries and index intersection
  • A covered query is answered from the index alone: every filtered field and every returned field is in one index, and _id is excluded from the projection (unless it is in the index). explain then shows docsExamined: 0. Chapter 6 shows a real one.
  • Be careful with index intersection: in principle the planner can combine two single-field indexes, but you should not design for it. A purpose-built compound index is almost always faster and more predictable, so create the compound index your query needs.
  • Every index must fit comfortably in memory with the hot data, or lookups hit disk. Drop indexes nothing uses ($indexStats shows usage).
⚠ Wrong key order and the "index on everything" reflex
  • An index on {status: 1, createdAt: -1} does not help a query on createdAt alone (it is not a prefix), and low-cardinality status first gives poor selectivity for the other keys unless the query always filters it. Indexing every field slows all writes and wastes memory. Start from the query list, apply ESR, and prove it with explain.
06
Chapter 6 - Query Plans

Reading explain("executionStats")

explain shows how the server runs a query. Use the verbosity "executionStats": it actually runs the query and reports counts. The three numbers to read are **nReturned (documents returned), totalKeysExamined (index entries read) and totalDocsExamined** (documents fetched). A healthy query examines about as many keys and documents as it returns.

Reading explain: what the plan stages tell you
COLLSCAN reads every document IXSCAN > FETCH index slice, then documents IXSCAN (covered) no documents read docsExamined = N keys = docs = nReturned keys = nReturned, docs = 0 Healthy ratios (from the lab in this chapter, 100,000 documents, 100 results): No index: docsExamined 100000 -> scans 1000x more than it returns Index {customerId}: keys 100, docs 100, but a SORT stage still runs in memory Index {customerId,status,createdAt}: keys 100, docs 100, no SORT stage Same index + projection on indexed fields: keys 100, docs 0 (covered) Rule: keysExamined and docsExamined should be close to nReturned. A big gap = wasted work.

The run below used a collection of 100,000 orders with 1,000 customers, four statuses and a timestamp. The query is find({customerId: 42, status: "shipped"}).sort({createdAt: -1}), which returns 100 documents. These are real numbers from MongoDB 7.0.45 (the 8.x shell output has the same structure). The ms values are from one run on a laptop and only show direction.

Index presentWinning plannReturnedkeysExamineddocsExaminedms
noneSORT > COLLSCAN100010000085
{customerId: 1}SORT > FETCH > IXSCAN1001001003
{customerId: 1, status: 1, createdAt: -1}FETCH > IXSCAN (no SORT)1001001005
same, projection {_id:0, createdAt:1, customerId:1, status:1}PROJECTION_COVERED > IXSCAN10010002
same index, but query {status: "shipped"} sorted by createdAt, limit 5SORT > COLLSCAN50100000169

Read the table as a story. With no index the server examined all 100,000 documents to return 100. The single-field index cut that to 100 keys and 100 documents but still sorted the 100 in memory (the SORT stage). The ESR index made the sort free. The projection made it covered. The last row is the lesson about prefixes: the query does not filter customerId, so the compound index cannot be used and the planner falls back to a collection scan.

mongosh - the helper used to print the plan (real run)
function show(e) { const x = e.executionStats, w = e.queryPlanner.winningPlan const stages = p => [p.stage].concat(p.inputStage ? stages(p.inputStage) : []) print(JSON.stringify({ plan: stages(w).join(" > "), nReturned: x.nReturned, keysExamined: x.totalKeysExamined, docsExamined: x.totalDocsExamined, ms: x.executionTimeMillis })) } show(db.orders.find({ customerId: 42, status: "shipped" }).sort({ createdAt: -1 }) .explain("executionStats")) // {"plan":"SORT > COLLSCAN","nReturned":100,"keysExamined":0,"docsExamined":100000,"ms":85}
Plan stageMeaningWorry when
COLLSCANReads the whole collectionThe collection is large and the query is frequent
IXSCANWalks an index range (see indexBounds)Bounds are [MinKey, MaxKey]: the index is scanned fully
FETCHLoads documents the index pointed todocsExamined is far above nReturned (a second filter runs after fetch)
SORTIn-memory sortLarge input; sorts have a memory limit and can spill or fail
PROJECTION_COVEREDResult built from the index onlyThis is the good case
SORT_MERGE, OR, SHARD_MERGECombine several index scans or shardsCheck each branch has its own index
Other tools
  • Use explain("queryPlanner") for the plan without running it, db.collection.find().hint({...}) to compare a specific index, $indexStats to find unused indexes, and the profiler or slow query log (slowms, default 100 ms) to discover which queries to explain in the first place.
07
Chapter 7 - Aggregation

The Aggregation Pipeline

find filters and projects. When you need grouping, joins, reshaping or computed output, use the aggregation pipeline: an array of stages, where each stage takes the documents from the previous one and passes new documents on. It is MongoDB's answer to GROUP BY, JOIN and subqueries.

Aggregation pipeline: documents flow through stages
$match filter early, use an index $group bucket and accumulate $sort order the results $limit top N $lookup join another collection collection -> stage -> stage -> stage -> result documents The server reorders safe stages: a $match placed after $group on the grouping key is moved before it. Lab check: explain of {$group by customerId} then {$match _id: 5} returned stages [ $cursor, $group ]: the filter was pushed down into the initial scan, not run after grouping. Put $match and $limit first, $project to drop big fields early, and $lookup on an indexed foreign field.
StageDoesNotes
$matchFilter documentsFirst in the pipeline it can use indexes. Same syntax as find
$project / $addFields / $setSelect, rename, compute fieldsThe server already prunes unused fields automatically in many cases
$groupGroup by a key and accumulate ($sum, $avg, $min, $max, $push, $addToSet)Blocks until all input is read; memory heavy for many groups
$sort, $limit, $skipOrder and page$sort followed by $limit is optimised into a top-N sort; a $sort on an indexed prefix is free early on
$unwindOne output document per array elementMultiplies documents; filter first
$lookupLeft outer join to another collectionIndex the foreign field. Works with sharded collections since 5.1
$facetRun several sub-pipelines on the same input (counts plus a page of results)Returns one document, so its total output is bound by the 16 MiB document limit
$unionWith, $merge, $outCombine collections, write results to a collectionBasis for materialised views and ETL
mongosh - top customers by revenue, then a join (the first pipeline ran in the lab)
db.orders.aggregate([ { $match: { status: "paid" } }, // early, index-friendly { $group: { _id: "$customerId", n: { $sum: 1 }, rev: { $sum: "$total" } } }, { $sort: { rev: -1 } }, { $limit: 2 } ]) // [ { _id: 81, n: 100, rev: 25497 }, { _id: 189, n: 100, rev: 25493 } ] // join customer names on, and report totals plus a page in one round trip db.orders.aggregate([ { $match: { status: "paid" } }, { $facet: { total: [ { $count: "orders" } ], page: [ { $sort: { createdAt: -1 } }, { $skip: 0 }, { $limit: 20 }, { $lookup: { from: "customers", localField: "customerId", foreignField: "_id", as: "c" } }, { $unwind: "$c" } ] } } ])
Optimising a pipeline
  • Filter and cut early. $match and $limit first. The optimiser moves some stages for you (for example $match before $sort, or before a $project), but it cannot fix a design that groups everything and filters later.
  • Check db.collection.explain("executionStats").aggregate([...]) for the real plan, as with find.
  • Memory: stages that need more than 100 megabytes write temporary files to disk by default since 6.0 (allowDiskUseByDefault), which makes a big group slow rather than failing. Pass {allowDiskUse: false} to forbid it per command.
  • A $lookup on an unindexed foreign field performs a scan per input document. Index the foreign key, or denormalise with extended reference (chapter 4).
  • If you aggregate the same big data constantly, precompute (computed pattern) or write a summary collection with $merge.
⚠ Using the pipeline as an application join layer
  • Frequent multi-collection $lookup chains over big collections suggest that the schema is relational data in a document store. Rework the model (embed or extended reference) before tuning the queries.
08
Chapter 8 - Storage and Replication

WiredTiger, Replica Sets, Write and Read Concern

WiredTiger in one page

WiredTiger is the default storage engine. You rarely tune it, but you should know what it does for you.

TopicWhat to knowVersion / default
ConcurrencyDocument-level concurrency control: writers to different documents do not block each otherSince 3.0
CacheWiredTiger keeps an internal cache of hot data and indexes; the OS file cache holds more. Do not set the cache above its default without a reasonDefault is the larger of 50% of (RAM minus 1 GB) or 256 MB (docs, current)
JournalA write-ahead log: changes are journaled so a crash replays everything since the last checkpoint. j: true waits for the journal flushJournal is flushed about every 100 ms (storage.journal.commitIntervalMs) and for j: true writes
CompressionBlock compression (snappy by default) for collections, prefix compression for indexes; journal is compressed tooAlternatives such as zstd are configurable
CheckpointsPeriodic consistent snapshots of the data filesJournal fills the gap between checkpoints
TransactionsLong open transactions keep old versions alive and put pressure on the cacheSee chapter 10
Replica sets

A replica set is a group of mongod processes that hold the same data: one primary takes all writes; secondaries copy them. Replication works through the oplog (local.oplog.rs), a capped collection of idempotent operations that secondaries tail and apply. The default oplog size on WiredTiger is 5% of free disk space, between 990 MB and 50 GB; it must cover the time a secondary can be offline and still catch up without a full resync.

Replica set: one primary, write concern decides when the client hears back
PRIMARY accepts all writes SECONDARY A applies oplog SECONDARY B applies oplog oplog entries oplog entries CLIENT write {w: majority} 1 insert Acknowledgement timeline for w:"majority" in a 3-member set (majority = 2): 1. primary applies the write and appends it to its oplog (and journals it) 2. at least one secondary replicates and applies it, so 2 of 3 members have it 3. primary tells the client "acknowledged": the write now survives a primary failure If the primary dies earlier, an unreplicated write can be rolled back on re-election.
Election after the primary becomes unreachable
Primary lost Secondary A no heartbeat for 10 s Secondary B vote? yes A is the new primary A member that has not heard from the primary within electionTimeoutMillis (default 10 s) calls an election. It needs votes from a majority of voting members and a log at least as current as theirs. Drivers with retryable writes (on by default) find the new primary and retry once; writes can pause during the election. Use an odd number of voting members (3 or 5). Two members cannot elect anyone if one fails.

Production sets have three or more voting members in different failure domains. An arbiter votes but holds no data; avoid it when you can, because it weakens majority write concern (see the default rule below).

Write concern, read concern and read preference
SettingOptionsDefault (current manual)What it controls
Write concern w1, a number, "majority", a tagw: "majority" (implicit default; w: 1 only if the set has an arbiter and data-bearing members are not more than the voting majority)How many members must have the write before the client is told
Write concern jtrue / falsew: "majority" waits for on-disk journaling by default (writeConcernMajorityJournalDefault)Wait for the journal flush on the node(s) that count
Write concern wtimeoutmillisecondsnoneReturn an error (not a rollback) if the concern is not met in time
Read concernlocal, available, majority, linearizable, snapshotlocal for reads against primary and secondariesHow "committed" the data you read must be
Read preferenceprimary, primaryPreferred, secondary, secondaryPreferred, nearestprimaryWhich member serves the read
  • local may return data that is later rolled back. majority returns only data acknowledged by a majority, so it survives failover. linearizable gives the strictest single-document reads at a latency cost. snapshot is for transactions.
  • Reading from secondaries lowers load on the primary but returns data that lags behind, sometimes by a lot. Use it for analytics and reports, not for read-your-writes flows. Causally consistent sessions can give read-your-own-writes across members.
  • Unrelated to the above: retryable writes and reads let the driver resend an operation once after a network error or failover, so a single election usually does not surface as an application error.
mongosh - writes with explicit concerns (the last error is real: a 1-node set cannot satisfy w:2)
db.orders.insertOne(doc, { writeConcern: { w: "majority", j: true, wtimeout: 5000 } }) db.orders.find().readConcern("majority") db.getMongo().setReadPref("secondaryPreferred") // shell session default db.orders.insertOne({ a: 1 }, { writeConcern: { w: 2, wtimeout: 500 } }) // MongoWriteConcernError: Not enough data-bearing nodes
⚠ Assuming w:1 survives failover
  • A write acknowledged by the primary alone (w: 1) can be lost if the primary fails before any secondary has it, because that write is rolled back when the old primary rejoins. For data you cannot lose use w: "majority" (already the default) and read with majority when you need to see only durable data.
09
Chapter 9 - Sharding

Scaling Out with Shards

A replica set copies the same data to every member, so it scales reads and availability but not total data size or write throughput of the primary. Sharding splits a collection across several replica sets (shards). Reach for it when data or write load no longer fits one replica set: not before, because a sharded cluster is much more work to operate.

Sharded cluster: mongos routes by shard key using config server metadata
App / driver mongos (router) Config servers (replica set) chunk ranges per shard metadata Shard 1 (a replica set) chunks userId 0 .. 3333 Shard 2 (a replica set) chunks userId 3334 .. 6666 Shard 3 (a replica set) chunks userId 6667 .. 9999 Query with the shard key (userId = 4200): mongos targets one shard. Query without the shard key: mongos broadcasts to every shard and merges (scatter-gather). The balancer moves chunks between shards in the background to even out data. Chunk ranges shown are an example for a ranged key, not real defaults.
ComponentRole
ShardA replica set holding a subset of the collection's data (in production each shard is itself a replica set)
mongosRouter. Applications connect to it, it looks up which shards own the keys in the query and fans out. Stateless; run several
Config serversA replica set that stores the cluster metadata: which chunk range lives on which shard
Chunk / rangeA contiguous range of shard key values. Default range size is 128 MB; a chunk is the unit that moves
BalancerBackground process that migrates chunks so shards hold similar amounts of data
The shard key is the whole decision

You choose a shard key (one field or a compound of fields) per collection. Every query is then either targeted (it includes the key, one shard answers) or scatter-gather (every shard answers and mongos merges). Good keys have high cardinality, even write distribution and match your most frequent query filter.

StrategyHow it distributesGood forWeak spot
RangedContiguous key ranges per chunkRange queries on the key (createdAt between ...)A monotonically increasing key (timestamp, ObjectId) sends all new inserts to one shard: a hot shard
HashedHash of the key value spreads documents evenlyEven write distribution for ids and monotonic keysRange queries on the key become scatter-gather
ZonedYou tag shard key ranges and pin them to shardsData locality (EU users on EU shards), tiered hardwareMore configuration to maintain
mongosh on a mongos - enable sharding for a collection (commands shown, not run in the lab)
sh.shardCollection("shop.orders", { customerId: "hashed" }) // hashed key sh.shardCollection("shop.events", { tenantId: 1, createdAt: 1 }) // compound ranged key sh.status() db.orders.explain().find({ customerId: 42 }) // SINGLE_SHARD when targeted, SHARD_MERGE otherwise
  • Jumbo chunks: a chunk larger than the range size that cannot be split, most commonly because it holds a single shard key value. It cannot be divided or moved easily and becomes a bottleneck. The cure is a higher-cardinality key (add a suffix field).
  • Changing the key: since 5.0 you can reshard a collection to a different key (reshardCollection), and you can refine a key by adding suffix fields. Resharding needs free storage on the recipient shards (the docs give a formula of about twice the collection plus index size spread over the shards) and a brief write block at the end; read the requirements on the reshard page before attempting it. Treat it as an escape hatch, not a plan.
  • Transactions and $lookup work on sharded collections, but cross-shard operations cost more than single-shard ones.
⚠ A monotonic shard key
  • Sharding orders on {createdAt: 1} or _id (an ObjectId, ordered by time) makes every insert land in the newest range on a single shard while the others idle. Use a hashed key, or a compound key whose first field spreads the writes ({tenantId: 1, createdAt: 1}).
⚠ Sharding too early or on a key you never query
  • A key that your queries do not filter on turns every read into scatter-gather across all shards, which is slower than a single replica set. Confirm you really need sharding, pick the key from the real query mix, and test with representative data.
10
Chapter 10 - Consistency Tools

Transactions, Change Streams and Schema Validation

Multi-document ACID transactions

Since 4.0 (replica sets) and 4.2 (sharded clusters) MongoDB supports multi-document ACID transactions. Inside a transaction, multiple operations on multiple documents and collections commit together or not at all, with snapshot isolation.

mongosh - a transfer between two documents (real run on a one-node replica set)
const session = db.getMongo().startSession() const accts = session.getDatabase("shop").acct session.startTransaction({ readConcern: { level: "snapshot" }, writeConcern: { w: "majority" } }) accts.updateOne({ _id: "A" }, { $inc: { bal: -30 } }) accts.updateOne({ _id: "B" }, { $inc: { bal: 30 } }) session.commitTransaction() // balances afterwards: A 70, B 30 (starting from 100 and 0)
  • Cost. The manual states that in most cases a distributed transaction costs more than single-document writes and should not replace good schema design: denormalised, embedded models usually avoid the need.
  • Limits. A transaction has to finish in under 60 seconds by default (transactionLifetimeLimitSeconds), otherwise it is aborted. Long transactions hold snapshot history and put pressure on the WiredTiger cache. Since 4.2 a transaction's writes are spread over several oplog entries, so there is no 16 MB total limit, but each entry must still fit the document limit.
  • Errors. Drivers expose a callback API (withTransaction) that retries the whole transaction on TransientTransactionError and retries the commit on UnknownTransactionCommitResult. Prefer it to hand-written retry logic.
  • Read preference inside a transaction must be primary.
Use a transaction whenAvoid it when
Two or more documents must change together (money transfer, inventory plus order)The data can live in one document (atomic for free)
An invariant spans collections and cannot be redesigned awayYou are using it to emulate relational joins everywhere
Operations are short and touch few documentsThe work is a long batch job; chunk it into idempotent steps instead
Change streams

A change stream lets an application subscribe to inserts, updates, deletes and DDL events on a collection, database or whole deployment without tailing the oplog by hand. Events come in order, each carries a resume token (the event _id), and a stream can be resumed after a disconnect as long as the token is still inside the oplog window. They need a replica set or sharded cluster.

mongosh - watch a collection and react (command shown, not run)
const cs = db.orders.watch( [ { $match: { operationType: { $in: ["insert", "update"] }, "fullDocument.status": "paid" } } ], { fullDocument: "updateLookup" } ) while (!cs.isClosed()) { if (cs.tryNext()) { /* handle event; persist event._id as the resume token */ } }
  • Store the resume token after processing so a restarted consumer continues where it stopped.
  • fullDocument: "updateLookup" fetches the current document at lookup time, which may differ from the document at the time of the change; for the before/after image enable pre- and post-images (changeStreamPreAndPostImages) on the collection.
  • If the oplog rolls past your token you lose the ability to resume. Size the oplog for your worst consumer outage.
  • Typical use: cache invalidation, search indexing, event publishing. For reliable publication to a broker, change streams plus a persisted token play the role of an outbox reader (see Messaging page 07).
Schema validation with $jsonSchema
mongosh - validation rules on a collection (error text from a real run)
db.createCollection("people", { validator: { $jsonSchema: { bsonType: "object", required: ["name"], properties: { name: { bsonType: "string" }, age: { bsonType: "int", minimum: 0 } } } }, validationLevel: "strict", // "moderate": only validate inserts and updates to already-valid documents validationAction: "error" // "warn": allow but log }) db.people.insertOne({ age: -1 }) // MongoServerError: Document failed validation

Validation runs on writes, so it protects new data but does not check what is already stored. Add rules in warn mode first, fix the log noise, then switch to error. Existing collections can be changed with collMod. Validation is your defence against the "schemaless" trap from chapter 1.

11
Chapter 11 - Spring and Practice

Spring Data MongoDB, Docker Hands-On and Operations

Spring Data MongoDB maps Java classes to documents and offers two styles: the repository interface (MongoRepository) for CRUD and derived queries, and MongoTemplate for everything else (updates with operators, aggregations, bulk operations). Both sit on the MongoDB Java driver. The code on this page was compiled with javac against spring-boot-starter-data-mongodb 3.5.0, which brings Spring Data MongoDB 4.5.0 and the MongoDB Java driver 5.4.0. Spring Boot 4.0.0 brings Spring Data BOM 2025.1.0 and driver 5.6.1 (Spring Data MongoDB 5.x). Check the BOM for the Boot version you use.

Annotation / typePurpose
@Document(collection = "orders")Marks a class as a MongoDB document and names the collection
@IdThe _id field. A String id maps to an ObjectId when it is a valid hex string; the field can also be ObjectId, Long, UUID
@Indexed, @CompoundIndexDeclare indexes. Auto index creation is off by default since Spring Data MongoDB 3.0; set spring.data.mongodb.auto-index-creation=true in development, and create indexes via migrations in production
@Field("total")Rename the stored field
@VersionOptimistic locking: an update with a stale version fails with OptimisticLockingFailureException
MongoRepository<T, ID>CRUD, paging, derived queries (findByCustomerIdAndStatus...), @Query with JSON filters
MongoTemplateFluent Query/Criteria/Update API and Aggregation
MongoTransactionManagerEnables @Transactional against MongoDB; you must declare the bean and the server must be a replica set or sharded cluster
Java - entity with annotations (compiled against Spring Data MongoDB 4.5.0)
package demo; import java.math.BigDecimal; import java.time.Instant; import java.util.List; import org.springframework.data.annotation.Id; import org.springframework.data.annotation.Version; import org.springframework.data.mongodb.core.index.CompoundIndex; import org.springframework.data.mongodb.core.index.Indexed; import org.springframework.data.mongodb.core.mapping.Document; import org.springframework.data.mongodb.core.mapping.Field; @Document(collection = "orders") @CompoundIndex(name = "cust_status_created", def = "{'customerId': 1, 'status': 1, 'createdAt': -1}") public class Order { @Id private String id; // mapped to _id (ObjectId stored when the String is a valid hex id) @Indexed private String customerId; private String status; @Field("total") private BigDecimal total; // stored as Decimal128 by default in Spring Data MongoDB 4.x private Instant createdAt; private List<Line> lines; // embedded documents, no join @Version private Long version; // optimistic locking public record Line(String sku, int qty) { } public String getId() { return id; } public String getCustomerId() { return customerId; } public String getStatus() { return status; } public BigDecimal getTotal() { return total; } public Instant getCreatedAt() { return createdAt; } public List<Line> getLines() { return lines; } }
Java - repository with a derived query and a @Query filter
package demo; import java.util.List; import org.springframework.data.domain.Pageable; import org.springframework.data.mongodb.repository.MongoRepository; import org.springframework.data.mongodb.repository.Query; public interface OrderRepository extends MongoRepository<Order, String> { List<Order> findByCustomerIdAndStatusOrderByCreatedAtDesc(String customerId, String status, Pageable page); @Query("{ 'status': ?0, 'total': { $gte: ?1 } }") List<Order> findBigOrders(String status, java.math.BigDecimal min); }
Java - MongoTemplate aggregation and update, plus MongoTransactionManager
package demo; import java.util.List; import org.springframework.data.mongodb.MongoTransactionManager; import org.springframework.data.mongodb.MongoDatabaseFactory; import org.springframework.data.mongodb.core.MongoTemplate; import org.springframework.data.mongodb.core.aggregation.Aggregation; import org.springframework.data.mongodb.core.aggregation.AggregationResults; import org.springframework.data.mongodb.core.query.Criteria; import org.springframework.data.mongodb.core.query.Query; import org.springframework.data.mongodb.core.query.Update; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.stereotype.Service; import org.springframework.transaction.annotation.Transactional; @Configuration class MongoTxConfig { // Needed for @Transactional: Spring Boot does not register this bean for you. @Bean MongoTransactionManager transactionManager(MongoDatabaseFactory dbFactory) { return new MongoTransactionManager(dbFactory); } } @Service public class OrderService { public record Revenue(String id, int orders, double revenue) { } private final MongoTemplate mongo; public OrderService(MongoTemplate mongo) { this.mongo = mongo; } public List<Revenue> revenueByCustomer(String status) { Aggregation agg = Aggregation.newAggregation( Aggregation.match(Criteria.where("status").is(status)), Aggregation.group("customerId").count().as("orders").sum("total").as("revenue"), Aggregation.sort(org.springframework.data.domain.Sort.Direction.DESC, "revenue"), Aggregation.limit(10)); AggregationResults<Revenue> out = mongo.aggregate(agg, Order.class, Revenue.class); return out.getMappedResults(); } public long markShipped(String customerId) { Query q = Query.query(Criteria.where("customerId").is(customerId).and("status").is("paid")); return mongo.updateMulti(q, new Update().set("status", "shipped"), Order.class).getModifiedCount(); } @Transactional // runs through MongoTransactionManager: needs a replica set or sharded cluster public void transfer(String from, String to, int amount) { mongo.updateFirst(Query.query(Criteria.where("_id").is(from)), new Update().inc("bal", -amount), "acct"); mongo.updateFirst(Query.query(Criteria.where("_id").is(to)), new Update().inc("bal", amount), "acct"); } }
application.properties - connection (Spring Boot 3.5)
spring.data.mongodb.uri=mongodb://localhost:27017/shop?replicaSet=rs0&w=majority&retryWrites=true spring.data.mongodb.auto-index-creation=true
  • The Decimal128/BigDecimal mapping means money round-trips exactly, but the aggregation $sum over decimals is a decimal sum, so use BigDecimal fields for money, not double.
  • Derived queries become real queries on your indexes: findByCustomerIdAndStatusOrderByCreatedAtDesc is the ESR query from chapter 5, so the matching @CompoundIndex matters.
  • Use projections (interface or record projections) in repositories to avoid loading large documents when you need two fields.
  • @Transactional methods get a session bound to the thread. The transaction manager uses the default write concern and read concern of the client unless you configure options.
⚠ @Transactional that silently does nothing
  • If no MongoTransactionManager bean exists, @Transactional is not backed by a MongoDB transaction and the writes are independent. Also, against a standalone server (no replica set) starting a transaction fails. Run a single-node replica set even in development (see the Docker lab below).
Hands-on and operations

The image tag mongo:8 exists on Docker Hub (pulled on 2026-10-31). Important note about the lab. On the machine used to write this page (Linux kernel 7.0), the mongo:8 and mongo:8.0 containers refused to start. The log says: MongoDB cannot start: Linux kernel versions 6.19 and newer has a known incompatibility with this version of MongoDB, referencing SERVER-121912. Version 7.0.45 started fine, so every real output on this page comes from **mongo:7.0 (server 7.0.45)**. On a host with an older kernel mongo:8 is what you want; the commands are identical. Output structure of explain is the same in 8.x, but exact numbers will differ on your machine.

Shell - a single-node replica set (needed for transactions and change streams)
$ docker run -d --name mongo -p 27017:27017 mongo:8 --replSet rs0 $ docker exec mongo mongosh --quiet --eval 'rs.initiate({_id:"rs0",members:[{_id:0,host:"localhost:27017"}]})' $ docker exec -it mongo mongosh rs0 [direct: primary]> db.version() rs0 [direct: primary]> db.adminCommand({ getDefaultRWConcern: 1 }).defaultWriteConcern { w: 'majority', wtimeout: 0 }

The last line is real output from the 7.0.45 run and confirms the majority default. A host connecting from outside Docker must use a replica set host name it can resolve; the localhost:27017 member address above works for tools running inside the container.

Lab script outline
  • Load 100,000 orders with insertMany in batches (customerId = i mod 1000, status cycling over four values, createdAt in one-minute steps).
  • Run the query of chapter 6 with explain("executionStats") and print the helper summary after every createIndex. Expect a collection scan, then an index scan with an in-memory sort, then the ESR index without a sort stage, then a covered query.
  • Insert a duplicate into a unique index and read error code 11000; run a two-document transaction; create a validated collection and violate it. All of these were run for this page.
Backups, qualitatively
MethodIdeaCaveat
mongodump / mongorestoreLogical export of BSON; simple and portableSlow on large data; restoring rebuilds indexes; point-in-time needs the oplog option
Filesystem or volume snapshotSnapshot the data volume (journal on the same volume for consistency)Fast; for sharded clusters you need coordinated snapshots, which is hard by hand
Managed backup (Atlas, Ops Manager, cloud tools)Continuous or scheduled backups with point-in-time recoveryPaid features; know your recovery time and point
Delayed secondaryA member intentionally behind by hoursHelps against operator error; not a backup by itself

A replica set is availability, not backup: a bad deleteMany replicates to every member. Test restores, and know how far back you can go.

Common mistakes, collected
MistakeWhat you seeFix
Unbounded arrays in a documentUpdates slow down, documents near 16 MiB, writes failSeparate collection or bucket pattern (chapters 3 and 4)
Missing indexCOLLSCAN, high docsExamined, CPU spikes under loadexplain, then an ESR compound index (chapters 5 and 6)
Wrong or monotonic shard keyOne hot shard, scatter-gather queries, jumbo chunksHashed or compound key matching queries (chapter 9)
"Schemaless" with no validationMixed types for one field, defensive code everywhere$jsonSchema validation and a schemaVersion field
Giant $in listsHuge query documents, slow planning, scattered index lookupsModel the relationship so the filter is one key or a join collection; batch the ids
w: 1 for important dataAcknowledged writes vanish after failoverw: "majority" and retryable writes
Reading from secondaries for read-your-writesA user does not see their own changeRead from the primary or use causal consistency
Treating a replica set as a backupA deletion replicates everywhereReal backups and tested restores
Transactions everywhereHigh latency, write conflicts, aborted long transactionsEmbed so one document is atomic (chapter 3)
12
Chapter 12 - Interview

Questions You Should Be Able to Answer

Crisp answers for the questions that come up in interviews on MongoDB.

  • Q: When would you choose MongoDB over a relational database?
    When the data is naturally document-shaped and read as a unit, the shape varies or evolves, and you need to scale horizontally. Choose relational when you need many ad hoc joins, strong cross-entity constraints and mature reporting tools. MongoDB also supports multi-document transactions, so "no transactions" is no longer a valid reason to reject it.
  • Q: Embed or reference?
    Embed data that is read together, bounded and belongs to one parent. Reference data that is shared, large, unbounded or updated independently. Embedding gives one read and atomic updates; referencing avoids duplication and unbounded documents.
  • Q: What is the maximum document size and why does it matter?
    16 MiB (BSON document size limit). It prevents one document from using excessive RAM or bandwidth, and it makes unbounded embedded arrays a design error. Larger files go to GridFS.
  • Q: What is in an ObjectId?
    12 bytes: a 4-byte timestamp, a 5-byte per-process random value and a 3-byte counter. It is generated by the client, roughly time ordered, and unique without coordination. It is a poor shard key on its own because it increases monotonically.
  • Q: Explain the ESR rule.
    Order compound index keys as Equality, Sort, Range. Equality fields first keep the rest ordered, the sort field next lets the index deliver the order, and range fields last so they do not break the sort.
  • Q: How do you find out why a query is slow?
    Run explain("executionStats"). Compare nReturned to totalKeysExamined and totalDocsExamined, look for COLLSCAN and in-memory SORT, and add or reorder an index. Use the slow query log or profiler to find the queries.
  • Q: What is a covered query?
    One answered entirely from an index: all filter and returned fields are in the index and _id is excluded unless indexed. Explain shows zero documents examined.
  • Q: What does the default write concern guarantee?
    Since 5.0 the implicit default is w: "majority" (with an arbiter-related exception): the write is on a majority of data-bearing voting members and journaled before acknowledgement, so it survives a primary failure.
  • Q: What can still be lost with w: 1?
    An acknowledged write that has not yet replicated is rolled back if the primary fails and a secondary without it is elected.
  • Q: Describe a replica set election.
    When secondaries have not heard from the primary within electionTimeoutMillis (10 s by default), one calls an election and becomes primary with votes from a majority of voting members. Use an odd number of voters. Retryable writes let drivers ride through it.
  • Q: Read concern levels?
    local (default) may be rolled back, available is for sharded low-latency reads, majority returns majority-committed data, linearizable is the strictest single-document read, snapshot is used in transactions.
  • Q: How does sharding work and how do you choose a shard key?
    Data is split into ranges (chunks) by shard key across shards; mongos routes using config server metadata; the balancer moves chunks. A good key has high cardinality, even write distribution and matches the main query filter. Hashed keys even out writes; ranged keys support range queries; zones pin ranges to shards.
  • Q: What is a jumbo chunk?
    A chunk bigger than the range size that cannot be split, usually because it contains one shard key value. It cannot be balanced; fix with a higher-cardinality shard key.
  • Q: Can you change a shard key?
    Yes, since 5.0 by resharding to a new key, and you can refine a key by adding suffix fields. Resharding needs spare storage and careful planning.
  • Q: Does MongoDB support ACID transactions?
    Yes, multi-document ACID transactions on replica sets (4.0) and sharded clusters (4.2) with snapshot isolation. They cost more than single-document writes, have a 60-second default lifetime and should not replace good schema design.
  • Q: Name some schema design patterns.
    Bucket, subset, extended reference, computed, outlier, approximation, archive, schema versioning. Each solves a specific problem such as huge series, oversized working sets, repeated joins or hot aggregates.
  • Q: What is the oplog?
    A capped collection on each replica set member recording idempotent write operations. Secondaries tail it to replicate, and change streams are built on it. Its size sets how long a secondary can lag or be offline without a full resync.
  • Q: What are change streams used for?
    Real-time notification of data changes: cache invalidation, search index sync, event publishing. They are ordered, resumable via the resume token, and need a replica set or sharded cluster.
  • Q: How does WiredTiger handle concurrency and durability?
    Document-level concurrency, an internal cache plus the OS cache, compression, and a write-ahead journal with periodic checkpoints. j: true or w: majority waits for the journal.
  • Q: Why can $match placement matter in a pipeline?
    A leading $match can use an index and reduces documents for every later stage. The optimiser moves some filters earlier, but you should write it first anyway.
  • Q: How do you make Spring @Transactional work with MongoDB?
    Declare a MongoTransactionManager bean, run against a replica set or sharded cluster, and keep transactions short.
  • Q: Is a replica set a backup?
    No. It protects against node failure; a bad delete replicates. Take real backups (dump, snapshots or a managed service) and test restores.
13
Chapter 13 - Summary

Cheat Sheet and Recap

Cheat sheet
ConceptRememberValue / default (MongoDB 8.x manual)
Document sizeHard limit, so no unbounded arrays16 MiB (BSON document size)
ObjectIdClient-generated, roughly time ordered12 bytes = 4 timestamp + 5 random + 3 counter
Index limitsPer collection / per compound index64 indexes, 32 fields
ESRCompound key orderEquality, Sort, Range
Healthy explainWork close to resultskeysExamined and docsExamined near nReturned
Default write concernMajority of data-bearing voting membersw: "majority" (arbiter exception)
Default read concern / preferenceReads may be rolled back at locallocal / primary
Election timeoutFailover starts after silence10 s (electionTimeoutMillis)
Oplog default sizeDefines catch-up window5% of free disk, min 990 MB, max 50 GB (WiredTiger)
WiredTiger cachePlus OS file cacheLarger of 50% of (RAM - 1 GB) or 256 MB
Chunk / range sizeUnit of balancing128 MB
Transaction lifetimeKeep short60 s (transactionLifetimeLimitSeconds)
Pipeline memory per stageSpills to disk by default100 MB, allowDiskUseByDefault since 6.0
Spring Boot 3.5Starter versionsSpring Data MongoDB 4.5.0, driver 5.4.0
∑ MongoDB - what to remember
  • Model for your queries. Embed what is read together and bounded; reference what is shared or unbounded. Patterns (bucket, subset, extended reference, computed, outlier) are named fixes for specific problems.
  • Indexes decide performance. Use ESR for compound keys and prove every index with explain("executionStats"): keys and documents examined should be near documents returned.
  • Aggregation: $match and $limit early, index the $lookup foreign field, precompute what you read constantly.
  • Replication gives availability, not backup. w: "majority" is the default and protects acknowledged writes; local reads can be rolled back; use an odd number of voters.
  • Sharding is a shard key decision. High cardinality, even writes, matches the main filter. Avoid monotonic keys; resharding exists but is expensive.
  • Transactions work but cost. Prefer single-document atomicity through design; keep transactions short.
  • Validate and version your schema. MongoDB is not schemaless in practice.
  • Spring: declare MongoTransactionManager, create indexes deliberately, use MongoTemplate for updates with operators and aggregations.
Version note

Checked on 2026-10-31 against the MongoDB manual (current pages for limits, BSON types, write and read concern, read preference, elections, oplog, WiredTiger, journaling, indexes, aggregation, sharding, resharding, transactions, change streams and schema design patterns) and Maven Central for Spring artefacts. The manual now also lists 8.3 and 9.0 release notes; this page targets the 8.x line and none of the facts above were found to differ on the pages read. Real command output comes from MongoDB 7.0.45 in Docker because the 8.x images refuse to start on the author's kernel; plan structure and ratios are the same in 8.x but timings will differ. Sharding commands, change streams and the Spring code against a live server were not run; the Spring classes were compiled but not executed. Index intersection is described qualitatively and was not verified against the manual.

Next in this section