mongodb
diff --git a/‎skills/mongodb-schema-design/SKILL.md‎
Lines changed: 141 additions & 0 deletions b/‎skills/mongodb-schema-design/SKILL.md‎
Lines changed: 141 additions & 0 deletions
diff --git a/‎skills/mongodb-schema-design/references/antipattern-excessive-lookups.md‎
Lines changed: 81 additions & 0 deletions b/‎skills/mongodb-schema-design/references/antipattern-excessive-lookups.md‎
Lines changed: 81 additions & 0 deletions
diff --git a/‎skills/mongodb-schema-design/references/antipattern-unnecessary-collections.md‎
Lines changed: 96 additions & 0 deletions b/‎skills/mongodb-schema-design/references/antipattern-unnecessary-collections.md‎
Lines changed: 96 additions & 0 deletions
@@ -0,0 +1,141 @@
+---
+name: mongodb-schema-design
+description: MongoDB schema design patterns and anti-patterns. Use when designing data models, reviewing schemas, migrating from SQL, or troubleshooting performance issues caused by schema problems. Triggers on "design schema", "embed vs reference", "MongoDB data model", "schema review", "unbounded arrays", "one-to-many", "tree structure", "16MB limit", "schema validation", "JSON Schema", "time series", "schema migration", "polymorphic", "TTL", "data lifecycle", "archive", "index explosion", "unnecessary indexes", "approximation pattern", "document versioning".
+license: Apache-2.0
+---
+
+# MongoDB Schema Design
+
+Data modeling patterns and anti-patterns for MongoDB, maintained by MongoDB. Bad schema is the root cause of most MongoDB performance and cost issues—queries and indexes cannot fix a fundamentally wrong model.
+
+## When to Apply
+
+Reference these guidelines when:
+- Designing a new MongoDB schema from scratch
+- Migrating from SQL/relational databases to MongoDB
+- Reviewing existing data models for performance issues
+- Troubleshooting slow queries or growing document sizes
+- Deciding between embedding and referencing
+- Modeling relationships (one-to-one, one-to-many, many-to-many)
+- Implementing tree/hierarchical structures
+- Seeing Atlas Schema Suggestions or Performance Advisor warnings
+- Hitting the 16MB document limit
+- Adding schema validation to existing collections
+
+## Quick Reference
+
+### 1. Schema Anti-Patterns - 3 rules
+
+- [antipattern-unnecessary-collections](references/antipattern-unnecessary-collections.md) - Splitting homogeneous data into multiple collections is often an anti-pattern; consult this reference to validate whether this is the case.
+- [antipattern-excessive-lookups](references/antipattern-excessive-lookups.md) - When encountering overly normalized collections that reference each other or frequent and possibly slow $lookup operations, consult this reference to validate whether this is problematic and how to fix it.
+- [antipattern-unnecessary-indexes](references/antipattern-unnecessary-indexes.md) - Consult this reference when indexes overlap or are not used by queries, to identify and remove unnecessary indexes that add overhead without benefit.
+
+### 2. Schema Fundamentals - 4 rules
+
+- [fundamental-embed-vs-reference](references/fundamental-embed-vs-reference.md) - Consult this reference for approaches to modeling different types of relationships (1:1, 1:few, 1:many, many:many, tree/hierarchical data) and how to decide between embedding and referencing based on access patterns.
+- [fundamental-document-model](references/fundamental-document-model.md) - Fundamentals of the document model. Consult this reference when migrating from SQL or other normalized data to a document database like MongoDB.
+- [fundamental-schema-validation](references/fundamental-schema-validation.md) - Consult this reference when creating new collections, or adding validation to existing collections, for example in response to finding inconsistent document structures or data quality issues.
+- [fundamental-document-size](references/fundamental-document-size.md) - Consult this reference when documents hit the hard 16MB limit, or when accesses are slower than expected as a result of large documents.
+
+### 3. Design Patterns - 11 rules
+
+- [pattern-approximation](references/pattern-approximation.md) - Use approximate values for high-frequency counters
+- [pattern-archive](references/pattern-archive.md) - Move historical data to separate/cold storage for performance
+- [pattern-attribute](references/pattern-attribute.md) - Collapse many optional fields into key-value attributes
+- [pattern-bucket](references/pattern-bucket.md) - Group time-series or IoT data into buckets
+- [pattern-computed](references/pattern-computed.md) - Pre-calculate expensive aggregations
+- [pattern-document-versioning](references/pattern-document-versioning.md) - Track document changes to enable historical queries and audit trails
+- [pattern-extended-reference](references/pattern-extended-reference.md) - Cache frequently-accessed data from related entities
+- [pattern-outlier](references/pattern-outlier.md) - Handle collections in which a small subset of documents are much larger than the rest, to prevent outliers from dominating memory and index costs
+- [pattern-polymorphic](references/pattern-polymorphic.md) - Store different types of entities in the same collection, often when they are different types of the same base entity (e.g. different types of users or different types of products)
+- [pattern-schema-versioning](references/pattern-schema-versioning.md) - Schema evolution, preventing drift, and safe online migrations. Consult when encountering inconsistent document structures, or when planning a schema change that cannot be applied atomically.
+- [pattern-time-series-collections](references/pattern-time-series-collections.md) - Use native time series collections for high-frequency time series data
+
+## Key Principle
+
+> **"Data that is accessed together should be stored together."**
+
+This is MongoDB's core philosophy. Embedding related data eliminates joins, reduces round trips, and enables atomic updates. Reference only when you must.
+
+A core way to implement this philosophy is the fact that MongoDB exposes **flexible schemas**. This means you can have different fields in different documents, and even different structures. This allows you to model data in the way that best fits your access patterns, without being constrained by a rigid schema. For example, if different documents have different sets of fields, that is perfectly fine as long as it serves your application's needs. You can also use schema validation to enforce certain rules while still allowing for flexibility.
+
+Another implication of the key principle is that information about the expected read and write workload becomes very relevant to schema design. If pieces of information from different entities are often queried or updated together, that means that prioritizing co-location of that data in the same document can lead to significant performance benefits. On the other hand, if certain pieces of information are rarely accessed together, it may make sense to store them separately to avoid loading more data than necessary.
+
+#### Schema Fundamentals Summary
+
+- **Embed vs Reference**: Choose embedding or referencing based on access patterns: embed when data is always accessed together (1:1, 1:few, bounded arrays, atomic updates needed); reference when data is accessed independently, relationships are many-to-many, or arrays can grow without bound.
+- **Data accessed together stored together**: MongoDB's core principle: design schemas around queries, not entities. Embed related data to eliminate cross-collection joins and reduce round trips. Identify your API endpoints/pages, list the data each returns, then shape documents to match those queries.
+- **Embrace the document model**: Don't recreate SQL tables 1:1 as MongoDB collections. Instead, denormalize joined tables into rich documents for single-query reads and atomic updates. When migrating from SQL, identify tables that are always joined together and merge them into single documents.
+- **Schema validation**: Use MongoDB's built-in `$jsonSchema` validator to catch invalid data at the database level (type checks, required fields, enum constraints, array size limits). Start with `validationLevel: "moderate"` and `validationAction: "warn"` on existing collections, then tighten to `strict`/`error`.
+- **16MB document limit**: MongoDB documents cannot exceed 16MB—this is a hard limit, not a guideline. Common causes: unbounded arrays, large embedded binaries, deeply nested objects. Mitigate by moving unbounded data to separate collections and monitoring document sizes with `$bsonSize`.
+
+## Embed/Reference Decision Framework
+
+| Relationship | Cardinality | Access Pattern | Recommendation |
+|-------------|-------------|----------------|----------------|
+| One-to-One | 1:1 | Always together | Embed |
+| One-to-Few | 1:N (N < 100) | Usually together | Embed array |
+| One-to-Many | 1:N (N > 100) | Often separate | Reference |
+| Many-to-Many | M:N | Varies | Two-way reference |
+
+This is a **rough** guideline, and whether to embed or reference depends on your specific access patterns, data size, and read/write frequencies. Always verify with your actual workload.
+
+## How to Use
+
+Each reference file listed above contains detailed explanations and code examples. Use the descriptions in the Quick Reference to identify which files are relevant to your current task.
+
+Each reference file contains:
+- Brief explanation of why it matters
+- Incorrect code example with explanation
+- Correct code example with explanation
+- "When NOT to use" exceptions
+- Performance impact and metrics
+- Verification diagnostics
+
+---
+
+## How These Rules Work
+
+### MongoDB MCP Integration
+
+For automatic verification, connect the [MongoDB MCP Server](https://github.com/mongodb-js/mongodb-mcp-server).
+
+If the MCP server is running and connected, I can automatically run verification commands to check your actual schema, document sizes, array lengths, index usage, and more. This allows me to provide tailored recommendations based on your real data, not just code patterns.
+
+**⚠️ Security**: Use `--readOnly` for safety. Remove only if you need write operations.
+
+When connected, I can automatically:
+- Infer schema via `mcp__mongodb__collection-schema`
+- Measure document/array sizes via `mcp__mongodb__aggregate`
+- Check collection statistics via `mcp__mongodb__db-stats`
+
+### ⚠️ Action Policy
+
+**I will NEVER execute write operations without your explicit approval.**
+
+Before any write or destructive operation via MCP, I will: (1) summarize the exact operation (collection, index/validator, estimated number of docs affected), and (2) ask for explicit confirmation (yes/no). I will not proceed on partial or ambiguous approvals.
+
+| Operation Type | MCP Tools | Action |
+|---------------|-----------|--------|
+| **Read (Safe)** | `find`, `aggregate`, `collection-schema`, `db-stats`, `count` | I may run automatically to verify |
+| **Write (Requires Approval)** | `update-many`, `insert-many`, `create-collection` | I will show the command and wait for your "yes" |
+| **Destructive (Requires Approval)** | `delete-many`, `drop-collection`, `drop-database` | I will warn you and require explicit confirmation |
+
+When I recommend schema changes or data modifications:
+1. I'll explain **what** I want to do and **why**
+2. I'll show you the **exact command**
+3. I'll **wait for your approval** before executing
+4. If you say "go ahead" or "yes", only then will I run it
+
+**Your database, your decision.** I'm here to advise, not to act unilaterally.
+
+### Working Together
+
+If you're not sure about a recommendation:
+1. Run the verification commands I provide
+2. Share the output with me
+3. I'll adjust my recommendation based on your actual data
+
+We're a team—let's get this right together.
+
+
@@ -0,0 +1,81 @@
+---
+title: Reduce Excessive $lookup Usage
+impact: CRITICAL
+impactDescription: "Can reduce query cost on hot paths by avoiding repeated cross-collection joins"
+tags: schema, lookup, anti-pattern, joins, denormalization, atlas-suggestion
+---
+
+## Reduce Excessive $lookup Usage
+
+**Frequent $lookup operations on hot paths can indicate over-normalization.** `$lookup` is useful, but repeated joins can be slower and more resource-intensive than querying a single collection, especially when supporting indexes or match selectivity are weak. If the same related fields are read together often, consider embedding or extended references.
+
+**Incorrect (constant $lookup for common operations):**
+
+```javascript
+// Every product page requires repeated joins across collections
+db.products.aggregate([
+  { $match: { _id: productId } },
+  { $lookup: {
+      from: "categories",          // Collection scan #2
+      localField: "categoryId",
+      foreignField: "_id",
+      as: "category"
+  }},
+  { $lookup: {
+      from: "brands",              // Collection scan #3
+      localField: "brandId",
+      foreignField: "_id",
+      as: "brand"
+  }},
+  { $unwind: "$category" },
+  { $unwind: "$brand" }
+])
+// Multiple join stages add planning/execution overhead on hot paths
+```
+
+Join cost depends on cardinality, stage order, index support, and result size. Measure before deciding to embed.
+
+**Correct (denormalize frequently-joined data):**
+
+Embed data that is always displayed alongside the product directly in the product document: include category fields (`_id`, `name`, `path`) and brand fields (`_id`, `name`, `logo`) as subdocuments. A single indexed query returns complete product data without `$lookup`. Listing queries (e.g. by category) also run against a single collection.
+
+**Managing denormalized data updates:**
+
+When category data changes (a rare event), use `updateMany` to update all products matching that category’s `_id` with the new field values. For frequently-changing data, keep both a reference ID (`brandId`) and a cache subdocument (`brandCache`) with a `cachedAt` timestamp; refresh the cache when it exceeds a staleness threshold.
+
+**When NOT to use this pattern:**
+
+- **Data changes frequently and independently**: If brand logos change daily, denormalization creates update overhead.
+- **Rarely-accessed data**: Don't embed review details if only a small fraction of product views load reviews.
+- **Many-to-many with high cardinality**: Avoid embedding large or fast-growing relationship sets.
+- **Analytics queries**: Batch jobs can afford $lookup latency; real-time queries cannot.
+
+## Verify with
+
+```javascript
+// Find pipelines with multiple $lookup stages
+db.setProfilingLevel(1, { slowms: 50 }) // Disable afterwards
+db.system.profile.find({
+  "command.aggregate": { $exists: true },
+  "command.pipeline.$lookup": {
+    $exists: true
+  }
+}).sort({ millis: -1 })
+
+// Check if $lookup foreign fields are indexed
+db.reviews.aggregate([
+  { $indexStats: {} }
+])
+// Look for index supporting the query in result
+
+// Measure $lookup impact
+db.products.aggregate([
+  { $match: { category: "electronics" } },
+  { $lookup: { from: "brands", localField: "brandId", foreignField: "_id", as: "brand" } }
+]).explain("executionStats")
+// Check totalDocsExamined in $lookup stage
+```
+
+Atlas Schema Suggestions flags: "Reduce $lookup operations"
+
+Reference: [Reduce Lookup Operations](https://mongodb.com/docs/manual/data-modeling/design-antipatterns/reduce-lookup-operations/)
@@ -0,0 +1,96 @@
+---
+title: Reduce Unnecessary Collections
+impact: CRITICAL
+impactDescription: "Reduces avoidable joins when related data is repeatedly queried together"
+tags: schema, collections, anti-pattern, embedding, normalization, atlas-suggestion
+---
+
+## Reduce Unnecessary Collections
+
+**Collection count alone is not the anti-pattern.** The anti-pattern is using collections as a substitute for indexes — creating one collection per category, time period, or partition key instead of indexing a single collection. Every collection carries a default `_id` index that consumes storage and strains the replica set, and cross-collection queries require `$lookup` or `$unionWith`, adding complexity and overhead.
+
+**Incorrect (one collection per day as partitioning strategy):**
+
+Creating one collection per time period (e.g. `temperatures_2024_05_10`, `temperatures_2024_05_11`, …) means each collection carries its own default `_id` index (365 collections/year = 365 extra indexes), cross-day queries require `$unionWith` across many collections, schema validation / indexes / TTL must be duplicated on every collection, and application code must dynamically resolve the collection name for each query.
+
+**Correct (single collection with an index):**
+
+```javascript
+// All readings in one collection — the index does the partitioning work
+{ _id: ObjectId(), timestamp: ISODate("2024-05-10T10:00:00Z"), temperature: 60 }
+{ _id: ObjectId(), timestamp: ISODate("2024-05-10T11:00:00Z"), temperature: 61 }
+{ _id: ObjectId(), timestamp: ISODate("2024-05-11T10:00:00Z"), temperature: 68 }
+
+db.temperatures.createIndex({ timestamp: 1 })
+
+// Efficient range query — one collection, one index
+db.temperatures.find({
+  timestamp: { $gte: ISODate("2024-05-10"), $lt: ISODate("2024-05-11") }
+})
+
+// Optional TTL for automatic expiry (e.g. 90 days)
+db.temperatures.createIndex({ timestamp: 1 }, { expireAfterSeconds: 7776000 })
+```
+
+**Even better (bucket pattern or time series collection):**
+
+For high-volume time-stamped data, group readings into buckets or use a native time series collection, which is optimized for this workload:
+
+```javascript
+// Bucket pattern — one document per day
+{
+  _id: ISODate("2024-05-10T00:00:00Z"),
+  readings: [
+    { timestamp: ISODate("2024-05-10T10:00:00Z"), temperature: 60 },
+    { timestamp: ISODate("2024-05-10T11:00:00Z"), temperature: 61 },
+    { timestamp: ISODate("2024-05-10T12:00:00Z"), temperature: 64 }
+  ]
+}
+
+// In this particular case, a native time series collection
+// is also a good option to consider
+db.createCollection("temperatures", {
+  timeseries: { timeField: "timestamp", granularity: "hours" }
+})
+```
+
+**When to use separate collections:**
+
+| Scenario | Separate Collection | Why |
+|----------|--------------------|----|
+| Data accessed independently | Yes | Different query patterns |
+| Unbounded relationships | Yes | Prevents document growth |
+| Many-to-many | Yes | Students ↔ Courses |
+| 1:1 always together | No (embed) | User and profile |
+
+**When NOT to use this pattern:**
+
+- **Data is genuinely independent**: Products exist separately from orders; don't embed full product catalog in every order.
+- **Frequent independent updates**: If customer email changes shouldn't update all historical orders (it shouldn't).
+- **Data is accessed in different contexts**: Same address entity used for shipping, billing, user profile—keep it separate.
+- **Regulatory requirements**: Some industries require normalized data for audit trails.
+
+## Verify with
+
+```javascript
+// Count your collections
+for (const d of db.adminCommand({ listDatabases: 1 }).databases) {
+  const colls = db.getSiblingDB(d.name).getCollectionNames().length
+  print(`${d.name}: ${colls} collections`)
+}
+// Count alone is not sufficient: combine with access and index/storage evidence
+
+// Check if collections are always accessed together
+// If orders always needs customer, items, addresses
+// → they should be embedded
+db.system.profile.aggregate([
+  { $match: { op: "query" } },
+  { $group: { _id: "$ns", count: { $sum: 1 } } },
+  { $sort: { count: -1 } }
+])
+// Collections with similar access patterns should be combined
+```
+
+Atlas Schema Suggestions flags: "Reduce number of collections"
+
+Reference: [Reduce the Number of Collections](https://mongodb.com/docs/manual/data-modeling/design-antipatterns/reduce-collections/)