Skip to content

Conversation

dvrogozh
Copy link
Contributor

@dvrogozh dvrogozh commented Aug 22, 2025

Changes:

  • Defined stand alone class for the filter graph (based on definition in CpuDeviceInterface)
  • Generalized filter graph to:
    • Cover HW backends (by accepting HW frames context)
    • Support configurable filter pipelines (by accepting string pipeline description). For example, it's possible to pass VAAPI or CUDA specific filters:
FiltersContext.filters = "scale=480:270:sws_flags=bilinear"; # for CPU (current usage)
FiltersContext.filters = "scale_vaapi=480:270:format=rgba";  # for VAAPI

These changes will allow:

  • Easily define ffmpeg driven decoding + video processing pipeline for any backend supported by ffmpeg (for multi-gpu support)
  • Add ffmpeg media filters support to torchcodec (any filter, not only scale or color conversion)

See the following draft PR with the example of using the defined class for ffmpeg-vaapi backend to enable Intel GPU support:

I believe it can be used as a close reference to implement similar support for ffmpeg-cuda filters (scale_cuda, scale_npp).

CC: @scotts @NicolasHug @eromomon

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Meta Open Source bot. label Aug 22, 2025
@dvrogozh
Copy link
Contributor Author

Pushed a fix for the extra ; typo which was failing some ci checks.

@dvrogozh
Copy link
Contributor Author

Latest CI errors don't seem to be related with the PR. 2 errors:

Error: Unable to download artifact(s): Artifact not found for name: pytorch_torchcodec__3.9_cu126_x86_64

and

/Users/ec2-user/runner/_work/torchcodec/torchcodec/pytorch/torchcodec/src/torchcodec/_core/custom_ops.cpp:87:47: error: must specify at least one argument for '...' parameter of variadic macro [-Werror,-Wgnu-zero-variadic-macro-arguments]
  TORCH_INTERNAL_ASSERT(tensor.is_contiguous());
                                              ^
/Users/ec2-user/runner/_work/_temp/conda_environment_17164967222/lib/python3.10/site-packages/torch/include/c10/util/Exception.h:425:9: note: macro 'TORCH_INTERNAL_ASSERT' defined here
#define TORCH_INTERNAL_ASSERT(cond, ...)                                         \
* Getting build dependencies for wheel...

I believe the second error is addressed already in #847, so I will need to rebase. Not sure if the first one is addressed. @scotts : should I rebase now or you need time to fix 1st issue?

Dmitry Rogozhkin added 2 commits August 27, 2025 17:56
FFmpeg filter graphs allow to cover a lot of use cases including
cpu and gpu usages. This commit moves filter graph support out of
CPU device interface which allows flexibility in usage across
other contexts.

Signed-off-by: Dmitry Rogozhkin <[email protected]>
@dvrogozh
Copy link
Contributor Author

Rebased on top of latest master. I hope this will help ci to pass as some fixes for ci has recently landed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Meta Open Source bot.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant