Performance during read

[michalpelka]

Environment

• Windows version: 10
• Processor architecture: x64
• Dokany version: 1.1.0
• Library type (Dokany/FUSE): Dokany

Check List

• I checked my issue doesn't exist yet
• My issue is valid with mirror default sample and not specific to my user-mode driver implementation
• I can always reproduce the issue with the provided description below.
• I have updated Dokany to the latest version and have reboot my computer after.
• I tested one of the last snapshot from appveyor CI

Description

Hello,

We are using Dokany in one of our product, but we experience 70% drop of performance using Dokany in comparison to same product on Linux and FUSE. Our product uses Dokan API of Dokany (not FUSE). We experience quite bottleneck in read. Theoretically system is able to provide up to 750 MBPS read rate (and that number we observed using Linux FUSE implementation or "RAW" read from network to memory in C on MS Windows - without filesystem) . Using Dokany cuts rates of read about 70% - up to 250 MBPS - keeping write rates acceptable.

I made some test to check performance of Dokany. And that behaviour is not absent using standard mirror. Therefore I decide to open issue, to hear what drop of performance should be observed taking into acount overhead of Dokany filesystem.

Test was performed using AJA System test (1920x1080 HD, 16BIT RGBA)
https://www.aja.com/products/aja-system-test

Dokany 1.1.0

Solution	Write MBPS	Read MBPS
SoftPerfect RAMDisk	1400	2184
SoftPerfect RAMDisk Mirrored with Dokany	536	449
NTFS SSD	242	247
NTFS SSD Mirrored with Dokany	237	186
Our Product with Dokany	539	246
Our Product - RAW Read	N/A	750

Dokany 2.0 Beta

Solution	Write MBPS	Read MBPS
SoftPerfect RAMDisk Mirrored with Dokany	428	552
NTFS SSD Mirrored with Dokany	235	181

[Liryna]

Hi @michalpelka ,

Thank you for the feedback and the numbers 👍
Dokan is a FS Windows Driver. FUSE is a module include in the kernel Linux. I would say FUSE has a better chance to be faster at this level 😄

Your tests are correct and reflect what is expected. Request on Dokan for read and write will be slower for a couple of reasons that are more explained here #307
TL;DR:
-Dokan Mirroring a device will take obviously more time than the device since it is on top of it (Mirror Time + Device Time).
-Dokan is using a single channel communication between Kernel and Library User. Means that multi query will need to wait one by one to be sent to the user land. There is a solution proposed in the discussion linked previously that need to be implemented.
-Dokan 1.x.x use fixed thread numbers when Dokan 2.x.x has Async thread that allows dynamic thread allocation for a number of queries (no query limit compared to version 1.x.x)
-Most important here for your test: Dokan is not using cache. It means that for each request dokan library and your user FS (mirror) will be called.

Therefore there is two features possible here to bring dokan to high performance: Improving the release on #307 and Implement cache on the kernel side (and even #291)

In general, this performance is not needed by most FS users (mirror) because the service used by them is often slower than the whole dokan/kernel query time.

I hope this answer helps you 🍹