[Jazzy] Does the composable rosbag2 Player/Recorder support intra-process / zero-copy communication?

[munenori-torikawa-j]

In ROS2 Jazzy, rosbag2 Recorder/Player can be used as composable nodes, and composable nodes can enable more efficient intra-process / zero-copy communication (with some exceptions and limitations).

On the other hand, rosbag2 appears to use type-erased serialized pub/sub (GenericSubscription/GenericPublisher). The rclcpp API reference for GenericSubscription states: “It does not support intra-process handling”:
https://docs.ros.org/en/jazzy/p/rclcpp/generated/classrclcpp_1_1GenericSubscription.html

My understanding is that for example, even if a publisher and rosbag2_transport::Recorder are composed into the same process, they do not communicate via intra-process / zero-copy, because the rosbag2 Recorder/Player uses GenericSubscription/GenericPublisher, which is not IPC-capable. Is this correct?

[MichaelOrlov]

@roncapat, since you were the author of the composable Player/Recorder in Rosbag2, could you please help answer this question?

[MichaelOrlov]

@munenori-torikawa-j For me to answer your question with details need to spend some substantial time digging into the ROS 2 rclcpp, rcl, rmw and executors layers. I am sorry, but I don't have the capacity for that currently.
However, I can refer you to others who are more familiar with this matter in practice.
Please refer to the #1419 (comment) post as a reference.

[mcp-ats]

@munenori-torikawa-j I agree with you and I am considering making an IPC node which performs serialization for IPC topics since data needs to be serialized for recording...I do not see this capability out of the existing interfaces.
Would anyone suggest a different way to tackle this? I cannot afford the mixed intra-inter process comms, I need full intra-process.

[roncapat]

Hi everybody, thank you for bringing this in. For sure, at the time I didn't get up to the point to fully understand how subscriptions worked inside the Recorder, as IIRC I didn't have the chance to touch that code. Of course, it would take me some time to check - and currently I'm facing a huge workload. That said, I'm interested in clarifying this aspect, understand why nevertheless performances at least on my systems (but I spoke with other people with the same experience) were improved by the past mentioned changes, devise if possible a plan to get to true intra-process (if not already fully achieved, and if it is possible), for example by extending the recorder with custom subscribers and possibly type adapers(?) but this will involve a deep dive of the data serialization concepts involved in rosbag (like, my reminescence is that data is usually stored in "cdr" serialization format - there may be or not flexibility to use other type of serializations per topic maybe?).

So... Plenty of stuff to dig. In the upcoming months it is planned on my side to do some test campaigns involving robag2 as recorder, so that could be the time I look into that.

Linking this one also: #1957