Skip to content

Commit 418a3cb

Browse files
committed
docs: Drop things marked as unnecessary in presentation, add static presentation
1 parent 6c5c1be commit 418a3cb

File tree

2 files changed

+62
-93
lines changed

2 files changed

+62
-93
lines changed

docs/presentation.md

Lines changed: 62 additions & 93 deletions
Original file line numberDiff line numberDiff line change
@@ -26,12 +26,20 @@ code-block-font-size: \scriptsize
2626

2727
# Efficient Synchronization of Linux Memory Regions over a Network (Presentation Notes)
2828

29+
```plaintext
2930
- Introduction
3031
- Title slide
3132
- ToC
33+
- Introduction
34+
- Methods
35+
- Optimizations
36+
- Results and Discussion
37+
- Implemented Use Cases
38+
- Future Use Cases
39+
- Conclusion
3240
- About me
3341
- Abstract/introduction
34-
- **Technological Landscape**
42+
- **Technological Landscape Today**
3543
- Methods for accessing remote resources:
3644
- Databases
3745
- Custom APIs
@@ -57,7 +65,7 @@ code-block-font-size: \scriptsize
5765
- Act as barriers for adoption
5866
- **Proposal**:
5967
- Instead of application-specific protocols, manage processes by directly operating on the memory region.
60-
- **Thesis Exploration**
68+
- **What I did in my thesis**
6169
- Examines alternative strategies for universal remote memory management.
6270
- **Review of Current Technologies**:
6371
- State of related technology
@@ -175,8 +183,7 @@ code-block-font-size: \scriptsize
175183
- Handles faults in user space in an idiomatic way.
176184
- **userfaultfd Backends**:
177185
- Useful for post-copy migration.
178-
- Backend is a simple pull-only reader interface:
179-
- ReadAt(p []byte, off int64)(n int, err error).
186+
- Backend is a simple pull-only reader interface
180187
- Any io.ReaderAt can provide chunks to a userfaultfd-registered memory region.
181188
- Access to reader:
182189
- Guaranteed to be aligned to system's page size (typically 4 KB).
@@ -303,55 +310,55 @@ code-block-font-size: \scriptsize
303310
- I/O- and CPU-intensive
304311
- To compute hash, entire file must be read
305312
- Context: Only option in file-based synchronization
306-
- **Speeding up Hashing Process**
307-
- Instead of entire file, hash individual file chunks
308-
- Implements delta synchronization
309-
- Method:
310-
- Open file multiple times
311-
- Hash individual offsets with each opened file
312-
- Aggregate changed chunks
313-
- Picking Algorithms:
314-
- Consider:
315-
- Throughput for hash calculation
316-
- Prevalence of hash collisions
317-
- Issue: Different inputs producing same hashes
318-
- Advantages:
319-
- If algorithm is CPU-bound (not I/O):
320-
- Multiple open files can boost concurrent processing
321-
- Reduces time in hashing iteration during polling
322-
- Dividing file into smaller chunks with their hashes:
323-
- Reduces network traffic for synchronization
324-
- Smaller file change leads to smaller chunk transfer
325-
- **Delta Synchronization Protocol**
326-
- Similar to rsync, but simplified
327-
- Supports synchronizing multiple files simultaneously
328-
- Uses file names as IDs
329-
- Supports central forwarding hub
330-
- No need for peer-to-peer connectivity between all hosts
331-
- Reduces network traffic
332-
- Central hub forwards one stream to all peers instead of multiple sends
333-
- Defines three actors:
334-
- Multiplexer
335-
- File advertiser
336-
- File receiver.
337-
- **Multiplexer Hub**
338-
- Accepts mTLS connections from peers
339-
- Upon connection:
340-
- Parses client certificate for common name
341-
- Uses common name as synchronization ID
342-
- Spawns Goroutine
343-
- Allows for more peer connections
344-
- Reads peer type
345-
- **src-control**
346-
- Reads file name from connection
347-
- Registers connection as provider of file
348-
- Broadcasts availability of the file
349-
- **dst-control**
350-
- Listens to file broadcasts from src-control peers
351-
- Relays:
352-
- Newly advertised files
353-
- Previously registered file names
354-
- Enables dst-control peers to start receiving them.
313+
- **Speeding up Hashing Process**
314+
- Instead of entire file, hash individual file chunks
315+
- Implements delta synchronization
316+
- Method:
317+
- Open file multiple times
318+
- Hash individual offsets with each opened file
319+
- Aggregate changed chunks
320+
- Picking Algorithms:
321+
- Consider:
322+
- Throughput for hash calculation
323+
- Prevalence of hash collisions
324+
- Issue: Different inputs producing same hashes
325+
- Advantages:
326+
- If algorithm is CPU-bound (not I/O):
327+
- Multiple open files can boost concurrent processing
328+
- Reduces time in hashing iteration during polling
329+
- Dividing file into smaller chunks with their hashes:
330+
- Reduces network traffic for synchronization
331+
- Smaller file change leads to smaller chunk transfer
332+
- **Delta Synchronization Protocol**
333+
- Similar to rsync, but simplified
334+
- Supports synchronizing multiple files simultaneously
335+
- Uses file names as IDs
336+
- Supports central forwarding hub
337+
- No need for peer-to-peer connectivity between all hosts
338+
- Reduces network traffic
339+
- Central hub forwards one stream to all peers instead of multiple sends
340+
- Defines three actors:
341+
- Multiplexer
342+
- File advertiser
343+
- File receiver.
344+
- **Multiplexer Hub**
345+
- Accepts mTLS connections from peers
346+
- Upon connection:
347+
- Parses client certificate for common name
348+
- Uses common name as synchronization ID
349+
- Spawns Goroutine
350+
- Allows for more peer connections
351+
- Reads peer type
352+
- **src-control**
353+
- Reads file name from connection
354+
- Registers connection as provider of file
355+
- Broadcasts availability of the file
356+
- **dst-control**
357+
- Listens to file broadcasts from src-control peers
358+
- Relays:
359+
- Newly advertised files
360+
- Previously registered file names
361+
- Enables dst-control peers to start receiving them.
355362
- Discussion
356363
- **Limitations**
357364
- Similar to userfaultfd but with different constraints.
@@ -683,9 +690,6 @@ code-block-font-size: \scriptsize
683690
- Similar interfaces as direct mount API.
684691
- Lifecycle of synchronization important.
685692
- Hook system for action registration.
686-
- **WAN Optimization**
687-
- Hybrid pre- and post-copy system.
688-
- Optimizations for WAN scenarios.
689693
- Pull-Based Synchronization with Migrations/Live migration
690694
- Technology section: Pre- and post-copy VM migration, workload analysis
691695
- Definition:
@@ -966,7 +970,7 @@ code-block-font-size: \scriptsize
966970
- Uses the same code generation framework as gRPC
967971
- Easy transition by re-generating code from DSL
968972
- fRPC adapter functions similarly to gRPC adapter.
969-
- Discussion and Results
973+
- Results and Discussion
970974
- Testing Environment
971975
- **Test Machine Specifications**
972976
- Device Model: Dell XPS 9320
@@ -1328,42 +1332,6 @@ code-block-font-size: \scriptsize
13281332
- Architectural constraints
13291333
- High likelihood of chunks being read outside the managed mounts background pull system
13301334
- Compared to Cassandra with lower throughput.
1331-
- General limitations of the r3map library (deadlocks etc.)
1332-
- **Mount APIs Issues and Workarounds**
1333-
- Implemented in Go
1334-
- Performance and usability concerns
1335-
- Go's challenges:
1336-
- Garbage collected language
1337-
- Activity in garbage collector can halt Goroutines
1338-
- Issues when:
1339-
- Using mmap API for managed or direct mounts
1340-
- Garbage collector manages an object referencing the exposed slice
1341-
- Releasing memory while copying data from the NBD device
1342-
- Accessing the slice can halt the NBD server Goroutine, causing deadlock
1343-
- Solutions:
1344-
- Lock mmaped region in memory
1345-
- Fetches all chunks from the remote, increasing Open() latency
1346-
- Recommended:
1347-
- Start NBD server in separate process
1348-
- Prevents garbage collector from halting NBD server and slice access simultaneously
1349-
- Alternative:
1350-
- Use non-garbage collected language (e.g., Rust)
1351-
- Prevents deadlock from occurring
1352-
- **NBD and Performance**
1353-
- NBD (Network Block Device)
1354-
- Underlying tech and protocol for mount API
1355-
- Comparatively performant
1356-
- Not as efficient as raw memory access
1357-
- **ublk[65] as a Potential Solution**
1358-
- Can improve concurrent access speeds
1359-
- Architecture:
1360-
- Similar to NBD
1361-
- User space server provides block device backend
1362-
- Kernel ublk driver creates /dev/ublkb\* block devices
1363-
- Analogous to /dev/nbd\* by NBD
1364-
- Current status:
1365-
- Documentation on this emerging kernel tech is limited
1366-
- NBD remains the standard for creating block devices in user space
13671335
- Implemented Use Cases
13681336
- Using mounts for remote swap with `ram-dl`
13691337
- **Using Mounts for Remote Swap with ram-dl**
@@ -1622,3 +1590,4 @@ code-block-font-size: \scriptsize
16221590
- Configurations for both LAN and WAN.
16231591
- Enables new application architectures and lifecycles.
16241592
- Thanks
1593+
```

docs/static/presentation.pdf

2.95 MB
Binary file not shown.

0 commit comments

Comments
 (0)