why Linux RAM Drive is slow when its not mapped?

[UzairJawaid]

sudo rapiddisk -l
rapiddisk 9.1.0
Copyright 2011 - 2023 Petros Koutoupis

List of RapidDisk device(s):

RapidDisk Device 1: rd1 Size (KB): 1048576 Usage (KB): 1048576 Status: Unlocked
RapidDisk Device 2: rd0 Size (KB): 2097152 Usage (KB): 1048576 Status: Unlocked

List of RapidDisk-Cache mapping(s):

None

sudo dd if=/dev/urandom of=/dev/rd1 bs=1M
count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 4.54824 s, 236 MB/s

Why this write is slow? if its a ram the writes should be fast

[pkoutoupis]

@UzairJawaid Hello. I do apologize for the delayed response. In your situation, /dev/urandom is the bottleneck because it is taking processing and time to generate a 1M payload for each write operation. Also, it doesn't help that dd is a single threaded and synchronous process. So in your benchmark, you are:

1. Filling a single 1M buffer with random data
2. Sending that one write to the drive
3. Waiting for it to return and reading its status
4. Filling the next single 1M buffer with random data
5. Sending the one write to the drive
6. and so on....

Also, urandom is significantly slower than random because it attempt to generate a truer random payload.

If you are testing single threaded synchronous I/O, then I would just defer to /dev/zero BUT if you want to benchmark a more accurate speed, I'd advise you use fio instead and run multithreaded asynchronous I/O at a larger queue depth.

Personally, and using fio, I have benchmarked as high as 22 GB/s with 1M transfers on a local system.

[UzairJawaid]

Hi, thanks for your reply.

Yes i moved to fio for testing. I used 8 cpus and get the result of 32 GBps. I will try to increase iodepth.
sudo fio --ioengine=libaio --size=1028m --filename=/media/ramdrive/test1.dat --direct=1 --loops=5 --name=test --bs=1M --rw=read --numjobs=8 --runtime=6000

I have two more question asking on the same thread.

Q1) You acheive 22GB on how many CPUs? How many CPUs should we use for best performance? i tried with different number, 8 was giving best result. Increasing more CPUS result was almost same

Q2) if i have 8 channel rams, can i make 8 ramdrive on each channel? will the speed go to 25 * 8 GB/s.?

Thanks @pkoutoupis

[pkoutoupis]

Oh, geez. It was quite a bit ago and I don't have access to that server anymore. I think it may have been a 64-core machine with 1TB of memory. I am sure that the performance plateaued at some point and didn't need all 64 cores but I didn't test for that. I was using the machine (which belonged to another company) to debug a few issues.

As for question 2, you cannot tie a ramdrive to a specific channel. It is allocated from the entire pool that the kernel sees and pages are allocated on-the-fly and upon request from this same pool.