Unknown Error when Extract Video Features with S3D and R(2+1)D

[santiagosilas]

Hi. Firstly, thanks for implementing and sharing this library. It is very functional and helpful. Also, the code is very clear to understand :)

I observed an unexpected/unknown error when trying to extract the features S3D and R(2+1)D from 16-frame sequences of large videos. No error feedback is printed in this case.

Is there any reason for this behavior? A temporary workaround I found was to break the video into small subvideos and then submit it to the extractor.

This behavior only happens for S3D and  R(2+1)D. But the same large videos work for I3D and CLIP feature extractors.

Thanks in advance.

[v-iashin]

Hi.

i think the machine just goes out of ram. Check your process monitor to confirm that that’s the case.

I think it’s expected if you are using feature extractors that are backed by torchvision.io.read_video which reads the whole video in memory at once.

At the same time, if it is based on the cv2 reader, then it should be ok because it reads it frame by frame.

It is a long standing issue. To fix it, i need to check if the quality of these features regress when the model was trained with the torchvision reader but during inference we use another feature extractor.

It is also a bit disappointing that we don’t have a good video reader as a community but that’s another story :/

[santiagosilas]

Ok, thanks for the reply.