-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathmessage-queues-tutorial.html
More file actions
245 lines (206 loc) · 13.1 KB
/
Copy pathmessage-queues-tutorial.html
File metadata and controls
245 lines (206 loc) · 13.1 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Message Queues Masterclass 📨</title>
<!-- Google Fonts -->
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Plus+Jakarta+Sans:wght@300;400;500;600;700;800&family=JetBrains+Mono:wght@400;500&display=swap" rel="stylesheet">
<!-- Highlight.js for Code Formatting -->
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.8.0/styles/github-dark.min.css">
<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.8.0/highlight.min.js"></script>
<script>hljs.highlightAll();</script>
<style>
:root {
--bg-color: #0b1121; /* Deep Blue background */
--surface-color: #161e31;
--primary-color: #3b82f6; /* Blue for Queues */
--primary-glow: rgba(59, 130, 246, 0.4);
--text-main: #f8fafc;
--text-muted: #94a3b8;
--border-color: rgba(255, 255, 255, 0.08);
--card-border: rgba(255, 255, 255, 0.1);
--accent-color: #60a5fa;
}
* { margin: 0; padding: 0; box-sizing: border-box; }
html { scroll-behavior: smooth; }
body {
font-family: 'Plus Jakarta Sans', sans-serif;
background-color: var(--bg-color); color: var(--text-main); line-height: 1.6;
background-image:
radial-gradient(at 0% 0%, rgba(59, 130, 246, 0.15) 0px, transparent 50%),
radial-gradient(at 100% 100%, rgba(37, 99, 235, 0.05) 0px, transparent 50%);
background-attachment: fixed; min-height: 100vh;
}
.layout { display: flex; gap: 2.5rem; max-width: 1400px; margin: 0 auto; padding: 3rem 2rem; }
/* Sidebar Styles (Reused) */
.sidebar {
width: 280px; flex-shrink: 0; position: sticky; top: 3rem; max-height: calc(100vh - 6rem); overflow-y: auto;
background: rgba(22, 30, 49, 0.8); border: 1px solid var(--card-border); border-radius: 16px; padding: 1.5rem;
backdrop-filter: blur(12px);
}
.sidebar h3 {
margin-bottom: 1rem; font-size: 1.1rem; text-transform: uppercase; letter-spacing: 1.5px;
color: var(--primary-color); border-bottom: 1px solid var(--card-border); padding-bottom: 1rem;
}
.sidebar ul { list-style: none; }
.sidebar li { margin-bottom: 0.3rem; }
.nav-section {
font-size: 0.85rem; text-transform: uppercase; letter-spacing: 1px; color: var(--text-muted);
margin: 1.5rem 0 0.5rem 0; padding-left: 0.5rem; font-weight: 700;
}
.sidebar a {
text-decoration: none; color: var(--text-muted); display: block; padding: 0.5rem 1rem;
border-radius: 8px; transition: all 0.3s ease; font-size: 0.95rem; font-weight: 500;
}
.sidebar a.active {
background: rgba(59, 130, 246, 0.2); color: #fff; border-left: 3px solid var(--primary-color); border-radius: 4px 8px 8px 4px;
}
.sidebar a:hover:not(.active) { background: rgba(59, 130, 246, 0.15); color: var(--primary-color); transform: translateX(5px); }
.main-content { flex: 1; min-width: 0; }
header { text-align: center; margin-bottom: 3rem; }
h1 {
font-size: 3rem; font-weight: 800; margin-bottom: 1rem;
background: linear-gradient(135deg, var(--primary-color) 0%, #93c5fd 100%);
-webkit-background-clip: text; -webkit-text-fill-color: transparent; text-shadow: 0 0 40px var(--primary-glow);
}
.subtitle { font-size: 1.2rem; color: var(--text-muted); }
/* Content Blocks */
.content-card {
background: rgba(22, 30, 49, 0.6); border: 1px solid var(--card-border);
border-radius: 16px; padding: 2.5rem; margin-bottom: 2rem; backdrop-filter: blur(10px);
box-shadow: 0 20px 40px -20px rgba(0, 0, 0, 0.5);
}
h2 { font-size: 1.8rem; margin-bottom: 1.5rem; display: flex; align-items: center; gap: 0.5rem; color: #fff; border-bottom: 1px solid var(--border-color); padding-bottom: 1rem; }
h3 { font-size: 1.3rem; margin: 1.5rem 0 1rem; color: #e2e8f0; }
p { margin-bottom: 1rem; font-size: 1.1rem; color: #cbd5e1; }
ul { margin-left: 1.5rem; margin-bottom: 1rem; color: #cbd5e1; }
li { margin-bottom: 0.5rem; }
.explanation { background: rgba(59, 130, 246, 0.1); border-left: 4px solid var(--primary-color); padding: 1rem 1.5rem; border-radius: 0 8px 8px 0; margin: 1.5rem 0; }
.explanation p { margin-bottom: 0; color: #e2e8f0; }
pre { background: #010409 !important; border: 1px solid var(--border-color); border-radius: 12px; padding: 1.5rem !important; margin: 1.5rem 0; overflow-x: auto; box-shadow: 0 10px 30px rgba(0,0,0,0.5); }
code { font-family: 'JetBrains Mono', monospace !important; font-size: 0.95rem; line-height: 1.5; }
.diagram { font-family: 'JetBrains Mono', monospace; background: #050505; color: #60a5fa; padding: 1.5rem; border-radius: 12px; border: 1px solid var(--primary-color); box-shadow: 0 0 20px rgba(59, 130, 246, 0.1); overflow-x: auto; white-space: pre; line-height: 1.4; margin: 1.5rem 0; font-size: 0.9rem; }
.cta-banner { background: linear-gradient(135deg, var(--primary-color) 0%, #1e3a8a 100%); padding: 3rem 2rem; border-radius: 16px; text-align: center; margin-top: 3rem; }
.cta-banner h2 { border: none; justify-content: center; }
@media (max-width: 900px) {
.layout { flex-direction: column; }
.sidebar { width: 100%; position: relative; top: 0; max-height: none; margin-bottom: 2rem; }
}
</style>
</head>
<body>
<div class="layout">
<!-- Sidebar Navigation -->
<aside class="sidebar">
<h3>🗂️ Tutorials Link</h3>
<nav>
<ul>
<li><a href="index.html">🏠 Home Hub</a></li>
<li class="nav-section">Databases / Lang</li>
<li><a href="mongodb-conditions-tutorial.html">🍃 MongoDB Class</a></li>
<li><a href="redis-masterclass-tutorial.html">🔥 Redis</a></li>
<li><a href="typescript-oops-tutorial.html">📘 TS OOPs</a></li>
<li class="nav-section">Architecture</li>
<li><a href="message-queues-tutorial.html" class="active">📨 Message Queues</a></li>
<li><a href="notification-system-tutorial.html">🔔 Notifications</a></li>
<li><a href="worker-threads-tutorial.html">⚙️ Node Workers</a></li>
<li><a href="system-design-tutorial.html">🏗️ Sys Design (Part 1)</a></li>
<li><a href="system-design-part2.html">🚀 Sys Design (Part 2)</a></li>
<li class="nav-section">Security & DevOps</li>
<li><a href="github-masterclass-tutorial.html">🐙 Git & GitHub</a></li>
<li><a href="rbac-masterclass-tutorial.html">🔐 RBAC & Auth</a></li>
<li><a href="abac-masterclass-tutorial.html">🛡️ ABAC Policies</a></li>
</ul>
</nav>
</aside>
<!-- Main Content -->
<main class="main-content">
<header>
<h1>Message Queues Masterclass 📨</h1>
<p class="subtitle">Distributed Background Jobs, BullMQ, Dead Letters aur RabbitMQ / Kafka ka concept.</p>
</header>
<div class="content-card" id="intro">
<h2>1. Producer-Consumer (Why Queues?)</h2>
<p>Maan lijiye Food Delivery app me aapne Order place kiya. Backend ko abhi 5 kaam krne hain:</p>
<ul>
<li>Payment Database update karna hai.</li>
<li>Restaurant ko notification bhejna hai.</li>
<li>Aapko Email Receipt bhejna hai.</li>
<li>Delivery Boy ko assign krne ki service jagani hia.</li>
</ul>
<div class="explanation">
<p>👉 Agar ye sab Synchronously (line by line) hoga toh aapki screen par 10 sec tak "Loading..." ghumta rahega. Isilye backend sirf API ko <strong>"Message Queue"</strong> mein daal kar Client ko instantly <code>200 OK</code> (Success) de deta hai. Baki kaam piche background me aaram se hota rahega.</p>
</div>
<div class="diagram">
[ User HTTP Request ] ---> (Node API / PRODUCER) ---> [ RETURN SUCCESS FAST! ]
|
v (Add to Queue)
[ MESSAGE QUEUE SERVER ]
(Stores tasks safely in memory/disk)
|
+--------+--------+
| |
v (Read task) v (Read task)
[ CONSUMER SERVER 1 ] [ CONSUMER SERVER 2 ]
(Sends Email in bg) (Updates DB in bg)
</div>
</div>
<div class="content-card" id="tools">
<h2>2. Popular Queue Softwares</h2>
<p>Market me Message queues chalane ke liye 3 main software widely use hote hain:</p>
<h3>A. RabbitMQ (Traditional & Reliable)</h3>
<p>Pure messaging system hai. Advanced Routing keys use hoti hn. e.g. "Sirf Orders wali email ye server sunega".</p>
<h3>B. Kafka (Big Data Streaming)</h3>
<p>Jab 1 million logs/sec ka sensor data araha ho, tab Kafka use krte hn. Ye actually ek massive distributed log system hai.</p>
<h3>C. BullMQ + Redis (Best for Node.js)</h3>
<p>BullMQ ek npm library hai jo piche <strong>Redis</strong> ko as a message queue use krti hai. Chhoti-Medium Node.js teams isee boht passand karti hain q k isko setup karna seconds ka kaam hai.</p>
</div>
<div class="content-card" id="code">
<h2>3. Implementing BullMQ (NodeJS)</h2>
<p>BullMQ implementation me 2 parts hotay hain. Pehla <strong>Producer</strong> jo queue me message dump krta hai, doosra <strong>Worker (Consumer)</strong> jo Queue pr nazar rakhta hai aur ek-ek krkay tasks execute krta hai.</p>
<h3>A. Add Job to Queue (Producer)</h3>
<pre><code class="language-javascript">const { Queue } = require('bullmq');
// Redis ke zariye connection
const emailQueue = new Queue('EmailDelivery Queue', { connection: { host: 'localhost', port: 6379 } });
app.post('/signup', async (req, res) => {
const userEmail = req.body.email;
// DB me user create kar diya...
// Ab bhari kaam Queue me daaldo:
await emailQueue.add('welcome_email', { email: userEmail, name: req.body.name });
res.json({ message: "Account Created!" }); // Instant Response!
});
</code></pre>
<h3>B. Process Job in Background (Consumer / Worker)</h3>
<p>Ye file ek alag Server (EC2 Instance) par run kar rhi hoti hai. Ye chupchap messages ka wait krti hai:</p>
<pre><code class="language-javascript">const { Worker } = require('bullmq');
// Consumer jo usi Queue ko sun raha hai
const worker = new Worker('EmailDelivery Queue', async (job) => {
console.log(`Sending Email to: ${job.data.email}... ⏳`);
// Simulate AWS SES or SendGrid API call (Takes 3 seconds)
await sendRealEmailApi(job.data.email, job.data.name);
console.log("Email Sent Succesfully! ✅");
}, { connection: { host: 'localhost', port: 6379 } });
</code></pre>
</div>
<div class="content-card" id="dlq">
<h2>4. What if Job Fails? (DLQ)</h2>
<p>Maan lo Mail API server down hai aur <code>sendRealEmailApi</code> fail ho gaya. Kya invoice gum (lost) ho jayega? Nahi! Queue Systems bohot smart hain:</p>
<ul>
<li>✨ <strong>Auto-Retries:</strong> Hum set kar saktay hn (e.g. <code>attempts: 5</code>). BullMQ automatically us kaam ko thori dair bad wapas try krega.</li>
<li>✨ <strong>Dead Letter Queue (DLQ):</strong> Agar 5 baar retry ke baad bhi error aaye, toh Queue us message ko hamesha ke liye Delete nahi krega, balke ek special dustbin queue me daal dega jisy "Dead Letter Queue" kehte hain.</li>
<li>Aap baad mein aakar check kar saktay hn ki kin logo ko emails nahi gai aur unhe dobara manually retry karva saktay hn!</li>
</ul>
</div>
<div class="cta-banner">
<h2 style="color: white">Architecture Secured! 🛡️</h2>
<p style="color: rgba(255,255,255,0.8); margin-top: 10px; font-weight: bold;">System crashes and blocking operations ki tensions ab khatam!<br><br>
<strong>Keep building resilient systems! 🚀</strong>
</p>
</div>
</main>
</div>
</body>
</html>