Relay with All Simulcast Tracks

[sweetbasil-bill]

Your environment.

• Version: latest master
• Environement: mac local
• Are you using a TURN server? No

What did you do?

Try to relay tracks from SFU-a to SFU-b
The tracks are simulcast tracks (contains RID), expect to see all the simulcast tracks on SFU-b instead of the best quality one.
Only be able to see just one track from those simulcast tracks, and missing RID information as well.

What did you expect?

If set PublisherTrack.clientRelay to true, should be able to see all the simulcast tracks.

What happened?

Per my understanding to the code, relay between SFU doesn't support simulcast now in ion-sfu. From the comment of PublisherTrack.clientRelay, seems like we plan to support it however not finished yet.
I tried to force all the simulcast tracks set the flag to true so they can go through, however, seems like DownTrack doesn't maintain RID information and those three tracks (h, f, q) are treated as duplicate tracks (track with same TrackID and StreamID and null RID) and discard by SFU-b.

I also tried to adjust the relay logic myself inside relay.go, create a new method (AddTracksFromReceiver) similar to AddTrack (https://github.com/pion/ion-sfu/blob/master/pkg/relay/relay.go#L438), and sending necessary EncodingParameters as well as using one RTPSender for those 3 simulcast tracks. However, on SFU-b receive side, it fails handling it on the created RTPReceiver: https://github.com/pion/ion-sfu/blob/master/pkg/relay/relay.go#L421. There is a nil pointer exception: https://github.com/pion/webrtc/blob/master/rtpreceiver_go.go#L26

When trace it, seems to me the failure happens because of following:

1. pion/sfu relay isn't building up webRTC peerConnection. There is an Offer / Answer transfer but no real peerconnection object built up. Instead, it uses signal object throw a datachannel to let SFU-b (the relayee) directly create RTPReceiver and set the parameters. So inside the relay.go, it receives the signal and handleRequest, create the RTPReceiver and call receiver.Receiver(params) for setting it up.
2. However, pion/webrtc doesn't expose API in RTPReceiver for handling receiveForRID, and it skips handle RID track parameters in startReceive (https://github.com/pion/webrtc/blob/master/rtpreceiver.go#L163), the only path invoking receiveForRID would happens inside peerconnection.go which indicates the expectation of peerConnection (and I suppose the simulcast tracks setup will be then setup when re-negotiation happens in peerConnection)

Then I have no idea how to make relay simulcast tracks work under current provided APIs in SFU.....does that mean relaying all the simulcast tracks isn't supported yet, or I mis-understand / mis-used the code / features / functions ?

Please guide me super thanks!!