diff --git a/pages/deployment/benchmarking-memgraph.mdx b/pages/deployment/benchmarking-memgraph.mdx index fbee1cb77..015b7f809 100644 --- a/pages/deployment/benchmarking-memgraph.mdx +++ b/pages/deployment/benchmarking-memgraph.mdx @@ -62,7 +62,7 @@ While `mgbench` gives you full control over benchmarking and produces raw result analyzing performance metrics at scale can be time-consuming. That’s why we created [**Benchgraph**](https://github.com/memgraph/benchgraph)—a companion visualization tool for `mgbench`. -![](/pages/memgraph-in-production/benchmarking-memgraph/benchgraph-snippet.png) +![](/pages/deployment/benchmarking-memgraph/benchgraph-snippet.png) Benchgraph is not required to use `mgbench`, but it offers a **convenient way to visualize and explore** benchmark results across a variety of run conditions: diff --git a/pages/deployment/workloads/_meta.ts b/pages/deployment/workloads/_meta.ts index d091f6364..0c82c3992 100644 --- a/pages/deployment/workloads/_meta.ts +++ b/pages/deployment/workloads/_meta.ts @@ -2,4 +2,5 @@ export default { "memgraph-in-cybersecurity": "Memgraph in cybersecurity", "memgraph-in-graphrag": "Memgraph in GraphRAG use cases", "memgraph-in-high-throughput-workloads": "Memgraph in high-throughput workloads", + "memgraph-in-mission-critical-workloads": "Memgraph in mission critical workloads", } diff --git a/pages/deployment/workloads/memgraph-in-cybersecurity.mdx b/pages/deployment/workloads/memgraph-in-cybersecurity.mdx index 88a591148..8028f4afe 100644 --- a/pages/deployment/workloads/memgraph-in-cybersecurity.mdx +++ b/pages/deployment/workloads/memgraph-in-cybersecurity.mdx @@ -50,7 +50,7 @@ Here's why Memgraph is a great fit for cybersecurity use cases: While many graph databases **max out around 1,000 events per second**, Memgraph can handle **up to 50x more** (see image below), making it ideal for **high-velocity security event processing**. - ![](/pages/memgraph-in-production/benchmarking-memgraph/realistic-workload.png) + ![](/pages/deployment/benchmarking-memgraph/realistic-workload.png) - **Non-blocking reads and writes with MVCC**: Built on multi-version concurrency control (MVCC), Memgraph ensures that **security event ingestion doesn't block threat analysis** and **analysis doesn't block ingestion**, diff --git a/pages/deployment/workloads/memgraph-in-graphrag.mdx b/pages/deployment/workloads/memgraph-in-graphrag.mdx index 086ec6fe6..b09a74c05 100644 --- a/pages/deployment/workloads/memgraph-in-graphrag.mdx +++ b/pages/deployment/workloads/memgraph-in-graphrag.mdx @@ -43,7 +43,7 @@ Here's what makes Memgraph a perfect fit for GraphRAG: ## What is covered? The suggestions for GraphRAG use cases **complement** several key sections in the -[general suggestions guide](/memgraph-in-production/general-suggestions). These sections offer important context and +[general suggestions guide](/deployment/best-practices). These sections offer important context and additional best practices tailored for performance, stability, and scalability in GraphRAG systems: - [Hardware sizing](#hardware-sizing) diff --git a/pages/deployment/workloads/memgraph-in-high-throughput-workloads.mdx b/pages/deployment/workloads/memgraph-in-high-throughput-workloads.mdx index ee289d0f0..44bc35927 100644 --- a/pages/deployment/workloads/memgraph-in-high-throughput-workloads.mdx +++ b/pages/deployment/workloads/memgraph-in-high-throughput-workloads.mdx @@ -45,7 +45,7 @@ Here's why Memgraph is a great fit for high-throughput use cases: While many graph databases **max out around 1,000 writes per second**, Memgraph can handle **up to 50x more** (see image below), making it ideal for **high-velocity, write-intensive workloads**. - ![](/pages/memgraph-in-production/benchmarking-memgraph/realistic-workload.png) + ![](/pages/deployment/benchmarking-memgraph/realistic-workload.png) - **Non-blocking reads and writes with MVCC**: Built on multi-version concurrency control (MVCC), Memgraph ensures that **writes don’t block reads** and **reads don’t block writes**, allowing each to scale independently. @@ -71,7 +71,7 @@ Here's why Memgraph is a great fit for high-throughput use cases: ## What is covered? The suggestions for high-throughput workloads **complement** several key sections in the -[general suggestions guide](/memgraph-in-production/general-suggestions). These sections offer important context and +[best practices guide](/deployment/best-practices). These sections offer important context and additional best practices tailored for performance, stability, and scalability in high-throughput systems: - [Choosing the right Memgraph flag set](#choosing-the-right-memgraph-flag-set)
diff --git a/pages/deployment/workloads/memgraph-in-mission-critical-workloads.mdx b/pages/deployment/workloads/memgraph-in-mission-critical-workloads.mdx new file mode 100644 index 000000000..490a40224 --- /dev/null +++ b/pages/deployment/workloads/memgraph-in-mission-critical-workloads.mdx @@ -0,0 +1,164 @@ +--- +title: Memgraph in mission-critical workloads +description: Suggestions on how to bring your Memgraph to production in mission-critical and high-availability workloads. +--- + +import { Callout } from 'nextra/components' +import { CommunityLinks } from '/components/social-card/CommunityLinks' + +# Memgraph in mission-critical workloads + + +👉 **Start here first** +Before diving into this guide, we recommend starting with the [**Best practices**](/deployment/best-practices) +page. It provides **foundational, use-case-agnostic advice** for deploying Memgraph in production. + +This guide builds on that foundation, offering **additional recommendations tailored to critical and high-availability workloads**. +In cases where guidance overlaps, consider the information here as **complementary or overriding**, depending +on the unique needs of your use case. + + +## Is this guide for you? + +This guide is for you if you're building **mission-critical systems** where uptime, data consistency, and fault tolerance are essential. +You’ll benefit from this content if: + +- 🛡️ You require **high availability** and **automatic failover** for your application. +- 🔒 You need **strong consistency guarantees** even under heavy loads. +- 🔁 You must **recover gracefully** from unexpected failures without data loss. +- 🏢 You need to **support multi-tenant environments** securely across multiple projects or customers. + +If this matches your needs, this guide will help you configure and operate Memgraph to meet the demands of **always-on production environments**. + +## Why choose Memgraph for mission-critical use cases? + +When stability, consistency, and resilience matter most, Memgraph is built to deliver. Here's why Memgraph is a great fit for mission-critical workloads: + +- **In-memory storage engine with persistence** + Memgraph keeps the working set fully in memory for fast access, while ensuring **durability** through **periodic snapshots** and **write-ahead logging (WALs)** in transactional mode. + Keeping the data in memory ensures lightning speed in times when you expect everything to function seamlessly and without issues during peak times of your critical service. + +- **High availability with automatic failover** + Memgraph supports full [**high availability clustering**](/clustering/high-availability), allowing you to deploy **multiple instances** with automatic **leader election** and **failover** when needed. + Deploying Memgraph with high availability will ensure Memgraph is up and running at all times, without compromising uptime of your services. + +- **Multi-version concurrency control (MVCC)** + Built on **MVCC**, Memgraph allows **non-blocking reads and writes**, ensuring that your system remains **responsive** even under concurrent access. Writes are not blocking reads, and vice versa. + +- **Snapshot isolation by default** + Memgraph uses **snapshot isolation** instead of **read-committed** isolation, preventing dirty reads and guaranteeing a **consistent view** of the graph at all times. + +- **Replication for read scaling and redundancy** + Memgraph supports **asynchronous replication**, enabling you to **scale read workloads** independently while ensuring **failover readiness**. For a more consistent view of the data, it + also supports **synchronous replication** which prioritizes consistency over scalability. + +- **Fine-grained access control and security** + Secure your system with [**role-based access control**](/database-management/authentication-and-authorization/role-based-access-control) and + [**label-based access control**](/database-management/authentication-and-authorization/role-based-access-control#label-based-access-control) to ensure only the right users see and manipulate data. + +## What is covered? + +The suggestions for mission-critical workloads **complement** several key sections in the +[general suggestions guide](/deployment/best-practices), with additional best practices to ensure uptime and data protection: + +- [Choosing the right Memgraph flag set](#choosing-the-right-memgraph-flag-set) + Memgraph offers flags to enhance recovery, snapshot management, and failover capabilities. + +- [Choosing the right Memgraph storage mode](#choosing-the-right-memgraph-storage-mode) + Guidance on selecting the **safest** and **most consistent** storage configurations. + +- [Enterprise features you might require](#enterprise-features-you-might-require) + Overview of **replication**, **multi-tenancy**, and **automatic failover** tools that are critical in production. + +- [Backup and recovery mechanisms](#backup-and-recovery-mechanisms) + Best practices to protect your data through snapshots, WALs, and external backup strategies. + +- [Queries that best suit your workload](#queries-that-best-suit-your-workload) + Designing queries that maintain consistent, safe, and predictable behavior in high-availability systems. + +## Choosing the right Memgraph flag set + +For mission-critical setups, you should configure Memgraph to optimize for **durability, fast recovery**, and **stability**. Some important flags include: + +- `--storage-snapshot-interval-sec=x` + Set how often snapshots are created. In mission-critical systems, you may want **frequent snapshots** to minimize recovery time. + +- `--storage-wal-enabled=true` + Ensure **WALs (write-ahead logs)** are enabled to protect all transactions between snapshots. + +- `--storage-parallel-schema-recovery=true` and `--storage-recovery-thread-count=x` + Enable **parallel recovery** to speed up startup time after a crash by using multiple cores. + +- `--query-execution-timeout-sec=x` + Set reasonable query timeouts to **avoid stuck queries** and prevent resource exhaustion.` + +## Choosing the right Memgraph storage mode + +For mission-critical deployments: + +- Always use `IN_MEMORY_TRANSACTIONAL` mode. +- This mode provides **full ACID guarantees**, **WAL support**, and **snapshot consistency**. + + +`IN_MEMORY_ANALYTICAL` is optimized for high-speed ingestion but does **not provide transactional durability**. It is not recommended for mission-critical workloads. + + +## Importing mechanisms +Importing mechanisms are best described in the [guide for high-throughput workloads](/deployment/workloads/memgraph-in-high-throughput-workloads). +The rule of thumb is to always setup the drivers to perform retries if you're doing heavy amount of writes, in order to avoid read conflicts. +The high throguhput guide also outlines the need for idempotent queries, to ensure data consistency if writes fail for any reason. + +## Enterprise features you might require + +For robust production environments, consider enabling: + +- **High availability clustering** ([learn more](/clustering/high-availability)) + Deploy multiple Memgraph instances with automatic leader election and failover. + +- **Replication for resilience** + Distribute replicas geographically or across availability zones to minimize the risk of localized outages. + +- **Role-based and label-based access control** ([learn more](/database-management/authentication-and-authorization/role-based-access-control)) + Protect sensitive graph data and ensure only authorized operations are performed. + +- **Multi-tenancy** ([learn more](/database-management/multi-tenancy)) + Securely isolate data and permissions between different teams, projects, or customers. + +## Backup and recovery mechanisms + +Data durability is critical in mission-critical environments. Memgraph supports: + +- **Snapshots** + Automatically or manually triggered full-database snapshots. + +- **Write-ahead logging (WALs)** + Transaction logs that enable you to **replay changes** made after the last snapshot. + +- **Manual backup and offloading** + Use external tools (like [`rclone`](https://rclone.org/)) to back up snapshots and WALs to **cloud storage** or **remote servers** for additional redundancy. + + +Memgraph currently does not automate backing up data to 3rd party locations, so integrating a backup process into your +system is highly recommended. + + + +Learn more about backup and restore [on our backup and restore documentation page](/database-management/backup-and-restore). + + +## Queries that best suit your workload + +In mission-critical workloads: + +- Prefer **idempotent writes** (`MERGE`) to avoid inconsistent state during retries. +- Optimize long-running queries and **profile** them regularly ([learn more](/querying/clauses/profile)). +- Avoid complex, unpredictable queries inside critical transactional paths. +- Use **schema constraints** and **indexes** wisely to enforce data integrity without hurting performance. + +Example of safe, idempotent data ingestion: + +```cypher +MERGE (n:Customer {id: $id}) +SET n += $props; +``` + diff --git a/public/pages/memgraph-in-production/benchmarking-memgraph/benchgraph-snippet.png b/public/pages/deployment/benchmarking-memgraph/benchgraph-snippet.png similarity index 100% rename from public/pages/memgraph-in-production/benchmarking-memgraph/benchgraph-snippet.png rename to public/pages/deployment/benchmarking-memgraph/benchgraph-snippet.png diff --git a/public/pages/memgraph-in-production/benchmarking-memgraph/realistic-workload.png b/public/pages/deployment/benchmarking-memgraph/realistic-workload.png similarity index 100% rename from public/pages/memgraph-in-production/benchmarking-memgraph/realistic-workload.png rename to public/pages/deployment/benchmarking-memgraph/realistic-workload.png