Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
4db7c96
Update CMakeLists.txt
vasilich-tregub Mar 5, 2025
ff831c3
Update x64inc.asm
vasilich-tregub Mar 5, 2025
7723090
sync-ing .vs, out
vasilich-tregub Mar 5, 2025
af7c737
gitignore out/
vasilich-tregub Mar 5, 2025
7a7e973
out/ ignore
vasilich-tregub Mar 5, 2025
9f8b418
Delete out/build/x64-Debug directory
vasilich-tregub Mar 5, 2025
85579bc
ffmpeg-plugin readme
vasilich-tregub Mar 5, 2025
81bafe7
Merge branch 'main' of https://github.com/vasilich-tregub/SVT-JPEG-XS
vasilich-tregub Mar 5, 2025
97df8d8
ready for MSYS2
vasilich-tregub Mar 6, 2025
2187263
x64inc.asm, asm-related CMakeLists ops tidied up
vasilich-tregub Mar 6, 2025
2a9b727
fixed ..asm_flag_debugs for msvc
vasilich-tregub Mar 13, 2025
6dff337
update root README to describe debug/release builds
vasilich-tregub Mar 14, 2025
20fe1ac
README's MSYS section updated
vasilich-tregub Mar 15, 2025
8ecc0de
build sections of readme finalized, jxs test file added
vasilich-tregub Mar 16, 2025
c40a903
reviewer's edits
vasilich-tregub Mar 18, 2025
d7efb5d
reversed build.sh edit in README, removed test file 200.jxs and edite…
vasilich-tregub Mar 18, 2025
7597705
reverse edit in Build/linux/build.sh
vasilich-tregub Mar 18, 2025
b47186f
revision of vasilich-tregub's fork following reviewer's instructions …
vasilich-tregub Mar 20, 2025
d11a099
MSYS2 and WSL2 sections for FFMPEG plugin
vasilich-tregub Mar 25, 2025
d3744d2
return to draft state
vasilich-tregub Mar 25, 2025
6461ccf
return to draft
vasilich-tregub Mar 25, 2025
429e7f5
README-s, instructions and data for enabling libsvtjpegxs codec in FF…
vasilich-tregub Mar 25, 2025
3ec898b
merged ffmpeg_enable-libsvtjpegxs folder with ffmpeg-plugin folder, r…
vasilich-tregub Mar 28, 2025
6b9422e
MSYS2 UCRT64 particulars
vasilich-tregub Mar 28, 2025
1dc808a
corrected a typo in ffmpeg plugin readme
vasilich-tregub Mar 28, 2025
147e588
added ffmpeg license info
vasilich-tregub Mar 28, 2025
a99f19a
CMakeSettings.json deleted
vasilich-tregub Apr 1, 2025
a67043f
to enable svt-av1, ffmpeg of version less than 7.1 must be patched
vasilich-tregub Apr 1, 2025
cd0fd66
tidied up MSYS2build.md, wsl2msys2.md. Use asm_flags_debug 'gcv8' for…
vasilich-tregub Apr 4, 2025
0af2a1d
cmake_minimum_required version value changed; typos in *.md file
vasilich-tregub Apr 5, 2025
45be6ac
added 0001-commit-to-enable-libsvtjpegxs.patch and instructions for g…
vasilich-tregub Apr 6, 2025
4b89cc0
vmaf can be enabled, too
vasilich-tregub Apr 6, 2025
bc278d3
minor retweaks to wsl2msys2.md
vasilich-tregub Apr 8, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,11 @@ Build/linux/
Build/windows/
Build/eclipse/
Build/coverage_linux
out/
*.yuv
*.bin
*.patch
*.raw
*.log
.vs
SVT-JPEG-XS/.vs
25 changes: 16 additions & 9 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -94,20 +94,22 @@ set(CMAKE_CXX_STANDARD 11)

# BUILD_SHARED_LIBS is a standard CMake variable, but we declare it here to
# make it prominent in the GUI.
option(BUILD_SHARED_LIBS "Build shared libraries (DLLs)." ON)
option(BUILD_SHARED_LIBS "Build shared libraries (DLLs)." OFF)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[q] BUILD_SHARED_LIBS is turned OFF, and there is never a way to turn it back ON. Is that an expected behavior?

message(STATUS "BUILD_SHARED_LIBS: ${BUILD_SHARED_LIBS}")

#option(BUILD_TESTING "Build SvtLcevcUnitTests, SvtLcevcApiTests, and SvtLcevcE2ETests unit tests")
option(COVERAGE "Generate coverage report")
option(BUILD_APPS "Build Enc and Dec Apps" ON)

if(WIN32)
set(CMAKE_ASM_NASM_FLAGS "${CMAKE_ASM_NASM_FLAGS} -DWIN64")
if(WIN32 OR MSYS)
set(CMAKE_ASM_NASM_FLAGS "${CMAKE_ASM_NASM_FLAGS} -DWIN64=1 -DUNIX64=0")
set(CMAKE_ASM_NASM_OBJECT_FORMAT win64)
else()
set(CMAKE_ASM_NASM_FLAGS "${CMAKE_ASM_NASM_FLAGS} -DUNIX64")
set(CMAKE_ASM_NASM_FLAGS "${CMAKE_ASM_NASM_FLAGS} -DUNIX64=1 -DWIN64=0")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[minor] Keep the same order for readability

Suggested change
set(CMAKE_ASM_NASM_FLAGS "${CMAKE_ASM_NASM_FLAGS} -DUNIX64=1 -DWIN64=0")
set(CMAKE_ASM_NASM_FLAGS "${CMAKE_ASM_NASM_FLAGS} -DWIN64=0 -DUNIX64=1")

set(CMAKE_ASM_NASM_OBJECT_FORMAT elf64)
endif()

if(UNIX)
if(UNIX AND NOT MSYS)
if(APPLE)
set(CMAKE_MACOSX_RPATH 1)
set(CMAKE_C_ARCHIVE_CREATE "<CMAKE_AR> Scr <TARGET> <LINK_FLAGS> <OBJECTS>")
Expand Down Expand Up @@ -195,7 +197,7 @@ else()
check_both_flags_add(--coverage)
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} --coverage")
endif()
if(MINGW)
if(MINGW OR MSYS)
check_both_flags_add(-mxsave -fno-asynchronous-unwind-tables)
else()
check_both_flags_add(-fstack-protector-strong)
Expand All @@ -206,12 +208,16 @@ if(CMAKE_C_FLAGS MATCHES ".*-march=native.*" OR CMAKE_CXX_FLAGS MATCHES ".*-marc
check_both_flags_add(-mno-avx)
endif()

if(CMAKE_C_FLAGS MATCHES "-O" AND NOT CMAKE_C_FLAGS MATCHES "-O0" AND NOT MINGW)
if(CMAKE_C_FLAGS MATCHES "-O" AND NOT CMAKE_C_FLAGS MATCHES "-O0" AND NOT MINGW AND NOT MSYS)
add_definitions(-D_FORTIFY_SOURCE=2)
endif()

if(CMAKE_ASM_NASM_OBJECT_FORMAT MATCHES "win")
set(CMAKE_ASM_NASM_FLAGS_DEBUG "${CMAKE_ASM_NASM_FLAGS_DEBUG} -gcv8")
if(MSVC)
set(CMAKE_ASM_NASM_FLAGS_DEBUG "${CMAKE_ASM_NASM_FLAGS_DEBUG} -gcv8")
else()
set(CMAKE_ASM_NASM_FLAGS_DEBUG "${CMAKE_ASM_NASM_FLAGS_DEBUG} -gcv8")
endif()
Comment on lines -214 to +220
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[q] Is it if something do x, else do x case? I do not see any difference.
image

elseif(CMAKE_ASM_NASM_COMPILER MATCHES "nasm")
set(CMAKE_ASM_NASM_FLAGS_DEBUG "${CMAKE_ASM_NASM_FLAGS_DEBUG} -gdwarf")
elseif(CMAKE_ASM_NASM_COMPILER MATCHES "yasm")
Expand Down Expand Up @@ -301,8 +307,9 @@ if(SANITIZER)
check_both_flags_add(-fno-omit-frame-pointer -fno-optimize-sibling-calls)
endif()

if(MSVC OR MINGW)
if(MSVC OR MINGW OR MSYS)
#"-Wall" too big restrict for MSVC
check_both_flags_add(-Wno-unused-parameter)
else()
check_both_flags_add(-Wall -Werror)
endif()
Expand Down
278 changes: 278 additions & 0 deletions MSYS2build.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,278 @@
# Building SVT-JPEG-XS with MSYS2

## License

Scalable Video Technology is licensed under the OSI-approved BSD+Patent license. See [LICENSE](LICENSE.md) for details.

We recommend to begin with a fresh installation of MSYS2.

1. Download an installer from the front page of [MSYS2](https://www.msys2.org), run the
installer.
2. When done, click Finish __without__ the `Run MSYS2 now` control selected (there can be
issues when the application is started immediately after installation), or do `$ exit`
from the first run of the terminal, if `Run MSYS2 now` was selected when you finished.
3. Run MSYS2 UCRT64 terminal. All MSYS2 commands of this guide must be run in the
__MSYS2 UCRT64__ terminal only!
4. Update the installation with the command `pacman -Suy`
5. Many packages installed in this guide have long names, like
`mingw-w64-ucrt-x86_64-ninja`. You can avoid writing long package names while using `pacboy`
command, see [Package Naming](https://www.msys2.org/docs/package-naming/):
- Install the pactoys packet group, in which pacboy belongs: `pacman -S pactoys`
6. Install the packet group __mingw-w64-ucrt-x86_64-toolchain__, the packets __yasm__ and __cmake__:
- Install toolchain: `pacboy -S toolchain:u`
If you are not sure about which packets you have to choose from this group, use the
default option `select all`.
- Install yasm (`pacboy -S yasm:u`) and cmake (`pacboy -S cmake:u`).
See all the packages installed: `pacman -Q`.
7. Using pacman, install git (`pacman -S git`). __Notice that this operation uses
pacman, not pacboy command__!
8. Git clone the SVT-JPEG-XS repository to your computer and generate debug/release
configuration of your choise:
```
git clone https://github.com/OpenVisualCloud/SVT-JPEG-XS.git
```
- Change the current working directory:
```
cd ~/SVT-JPEG-XS
```
- Run cmake command, specify a source directory (`builddebug`) to generate a build system for the Debug configuration
```
cmake -S . -B builddebug -DCMAKE_BUILD_TYPE=Debug -DBUILD_SHARED_LIBS=OFF -DCMAKE_INSTALL_PREFIX=$HOME/install-dir
```
- OR specify a source directory (`buildrelease`) for the Release configuration
```
cmake -S . -B buildrelease -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF -DCMAKE_INSTALL_PREFIX=$HOME/install-dir
```
- with the above two cmake commands, you can include any relevant cmake keys
you may find necessary for your project.
- Build Debug or Release configuration
- `cmake --build builddebug -j4 --config Debug --target install`
- OR
- `cmake --build buildrelease -j4 --config Release --target install`

- To test the build, you can run a newly installed program, for example
```
$ ~/install-dir/bin/SvtJpegxsSampleDecoder <path/filename.jxs>
```

## Encoder

### Supported colour formats

#### PLANAR

| format | EncApp: format + input-depth| ffmpeg name |status |
| -- | -- | -- | -- |
| YUV 420 8-bit | yuv420 + 8 | yuv420p | Tested, working properly
| YUV 420 10/12/14-bit |yuv420 + 10/12/14 | yuv420p{10/12/14}le | Tested, working properly
| YUV 422 8-bit | yuv422 + 8 |yuv422p | Tested, working properly
| YUV 422 10/12/14-bit |yuv422 + 10/12/14 | yuv422p{10/12/14}le | Tested, working properly
| YUV 444 8-bit | yuv444 + 8 | yuv444p | Tested, working properly
| YUV 444 10/12/14-bit | yuv444 + 10/12/14 |yuv444p{10/12/14}le | Tested, working properly
| RGB 8-bit | rgb + 8 | gbrp | Tested, working properly
| RGB 10/12/14-bit |rgb + 10/12/14 | gbrp{10/12/14}le | Tested, working properly
| YUV 400 8-bit | yuv400 + 8| - | Unsupported
| YUV 400 10-bit |yuv400 + 10/12/14 | - | Unsupported


#### PACKED
| format | EncApp: format + input-depth| ffmpeg name |status |
| -- | -- | -- | -- |
| RGB 8bit| rgbp + 8 | rgb24/bgr24 | Tested, working properly, **decoder decodes to planar format only**
| RGB 10/12/14bit |rgbp + 10/12/14| - | Tested, working properly, **decoder decodes to planar format only**


### Usage examples

#### Windows

```batch
SvtJpegxsEncApp.exe -i <input_file.yuv> -b <output_bitstream.bin> -w 1920 -h 1080 --input-depth 8 --colour-format yuv422 --bpp 5 --decomp_v 2 --decomp_h 5 --lp 4
```

#### Linux

```shell
./SvtJpegxsEncApp -i <input_file.yuv> -b <output_bitstream.bin> -w 1920 -h 1080 --input-depth 8 --colour-format yuv422 --bpp 5 --decomp_v 2 --decomp_h 5 --lp 4
```

#### Compression ratio

To encode stream with compression rate of 12:1 (assuming 8bit yuv420p), one pixel require 12bits so `--bpp 1 ` should be used:

```shell
./SvtJpegxsEncApp -i <input_file.yuv> -b <output_bitstream.bin> -w 1920 -h 1080 --input-depth 8 --colour-format yuv420 --bpp 1 --decomp_v 2 --decomp_h 5 --lp 4
```

#### Latency measurement

To measure average frame encoding time (latency) `--limit-fps` and high enough `--lp` parameters have to be used, cmd example:

```shell
./SvtJpegxsEncApp -i <input_file.yuv> -b <output_bitstream.bin> -w 1920 -h 1080 --input-depth 8 --colour-format yuv422 --bpp 5 --decomp_v 2 --decomp_h 5 --lp 4 --limit-fps 60
```

The `--limit-fps` must be lower than the maximum achievable number of frames. So that the next frames are not queued (time of waiting frames for processing is included in the latency time).

#### Interlaced video
To encode an interlaced stream, please provide a height that is half of the original.
For example, for a 1080i stream, use ```-w 1920 -h 540``` instead of ```-w 1920 -h 1080```


### Available parameters

Options:

```text
[--help] Show usage options and exit
[--show-bands] Print detailed informations about bands
(enabled:1, disable:0, default:0)
[-v] Verbose Encoder internal level (disabled:0, errors:1,
system info:2, system info full:3, warnings:4, ... all:6,
default: 2)
```

Input Options:

```text
-i Input Filename
-w Frame width
-h Frame height
--colour-format Set encoder colour format (yuv420, yuv422, yuv444, rgb(planar), rgbp(packed))
(Experimental: yuv400)
--input-depth Input depth
--bpp Bits Per Pixel, can be passed as integer or float
(example: 0.5, 3, 3.75, 5 etc.)
[-n] Number of frames to encode
[--limit-fps] Limit number of frames per second
(disabled: 0, enabled [1-240])
```

Output Options:

```text
[-b] Output filename
[--no-progress] Do not print out progress (no print:1, print:0, default:0)
[--packetization-mode] Specify how encoded stream is returned (multiple packets per frame:1, single packet per frame:0, default:0),
for details please refer to Codestream/Slice packetization mode
https://datatracker.ietf.org/doc/html/rfc9134
```

Coding features used during Rate Calculation (quality/speed tradeoff):

```text
[--decomp_v] Vertical decomposition (0, 1, 2, default: 2)
[--decomp_h] Horizontal decomposition have to be greater or equal to
decomp_v (1, 2, 3, 4, 5, default: 5)
[--quantization] Quantization method(deadzone:0, uniform:1, default:0)
[--slice-height] The height of each slice in units of picture luma pixels (default:16, any value that is multiple of 2^(decomp_v))
[--coding-signs] Enable Signs handling strategy (full:2, fast:1, disable:0, default:0)
[--coding-sigf] Enable signification coding (enabled:1, disable:0, default:1)
[--coding-vpred] Enable Vertical Prediction coding (disable:0, zero prediction residuals:1, zero coefficients:2, default: 0)
[--rc] Rate Control mode (
CBR: budget per precinct: 0,
CBR: budget per precinct with padding movement: 1,
CBR: budget per slice: 2,
CBR: budget per slice with nax size RATE: 3,
default 0)
```

Threading, performance:

```text
[--asm] Limit assembly instruction set [0 - 11] or [c, mmx, sse, sse2,
sse3, ssse3, sse4_1, sse4_2, avx, avx2, avx512, max], by default
highest level supported by CPU
[--profile] Threading model type, can be passed as digit (0, 1) or string
(latency, cpu) 0:latency (Low Latency mode), 1:cpu (Low CPU use
mode), default is 0
[--lp] Thread Scaling parameter, the higher the value the more threads
are created and thus lower latency and/or higher FPS can be
achieved (default: 0, which means lowest possible number of threads is created)
```

## Decoder

### Usage examples

#### Windows

```batch
SvtJpegxsDecApp.exe -i <input_bitstream.bin> -o <output_file.yuv> --lp 5
```

#### Linux

```shell
./SvtJpegxsDecApp -i <input_bitstream.bin> -o <output_file.yuv> --lp 5
```

#### Latency measurement

To measure average frame decoding time (latency) `--limit-fps` and high enough `--lp` parameters have to be used, cmd example:

```shell
./SvtJpegxsDecApp -i <input_bitstream.bin> -o <output_file.yuv> --lp 5 --limit-fps 60
```

The `--limit-fps` must be lower than the maximum achievable number of frames. So that the next frames are not queued (time of waiting frames for processing is included in the latency time).

### Available parameters

Options:

```text
[--help] Show usage options and exit
[-v] Verbose decoder internal level (disabled:0, errors:1,
system info:2, system info full:3, warnings:4, ... all:6,
default: 2)
[--force-decode] Force decode first frame
```

Input Options:

```text
-i Input Filename
[-n] Number of frames to decode
[--find-bitstream-header] Find bitstream header
[--limit-fps] Limit number of frames per second
(disabled: 0, enabled [1-240])
[--packetization-mode] Specify how bitstream is passed to decoder
(multiple packets per frame:1, single packet per frame:0, default:0)
```

Output Options:

```text
[-o] Output Filename
```

Threading, performance:

```text
[--asm] Limit assembly instruction set [0 - 11] or [c, mmx, sse, sse2, sse3,
ssse3, sse4_1, sse4_2, avx, avx2, avx512, max], by default highest
level supported by CPU
[--lp] Thread Scaling parameter, the higher the value the more threads are
created and thus lower latency and/or higher FPS can be achieved
(default: 0, which means lowest possible number of threads is created)
```

## Encoder and Decoder design

Please see [Encoder design](documentation/encoder/svt-jpegxs-encoder-design.md)

Please see [Decoder design](documentation/decoder/svt-jpegxs-decoder-design.md)


## API usage

Please see [Encoder snippet](documentation/encoder/EncoderSnippets.md) for encoder structure overview and simplified encoder usage.

Please see [Decoder snippet](documentation/decoder/DecoderSnippets.md) for decoder structure overview and simplified decoder usage.

## Notes

The information in this document was compiled at <mark>v0.9</mark> may not
reflect the latest status of the design. For the most up-to-date
settings and implementation, it's recommended to visit the section of the code.
12 changes: 12 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,18 @@ Supported OS versions:
- __API headers location__
- API headers can be found under `Source/API`

### Windows Services for Linux
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
### Windows Services for Linux
### Windows Subsystem for Linux (WSL)

As of March 2025, ISO/IEC 21122 protocol is in the development phase. WSL2 with
Ubuntu 24.04 distro is a convenient platform for developing experimental
implementations. Use Linux OS (64-bit) build instructions to build SVT-JPEG-XS
in WSL2.
Comment on lines +69 to +72
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[minor] No need for indentation here.

Suggested change
As of March 2025, ISO/IEC 21122 protocol is in the development phase. WSL2 with
Ubuntu 24.04 distro is a convenient platform for developing experimental
implementations. Use Linux OS (64-bit) build instructions to build SVT-JPEG-XS
in WSL2.
As of March 2025, ISO/IEC 21122 protocol is in the development phase. WSL2 with
Ubuntu 24.04 distro is a convenient platform for developing experimental
implementations. Use Linux OS (64-bit) build instructions to build SVT-JPEG-XS
in WSL2.


### MSYS2, Software Distribution and Building Platform for Windows
README.md in the ffmpeg-plugin folder of this repository requires that FFMPEG is
build with [MSYS2](https://www.msys2.org). For a seamless integration you have to
build SVT-JPEG-XS with MSYS2. The guide for building SVT-JPEG-XS with MSYS2 is given
in MSYS2build.md.
Comment on lines +75 to +78
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[minor] No need for indentation here.

  • build -> built
Suggested change
README.md in the ffmpeg-plugin folder of this repository requires that FFMPEG is
build with [MSYS2](https://www.msys2.org). For a seamless integration you have to
build SVT-JPEG-XS with MSYS2. The guide for building SVT-JPEG-XS with MSYS2 is given
in MSYS2build.md.
README.md in the ffmpeg-plugin folder of this repository requires that FFmpeg is
built with [MSYS2](https://www.msys2.org). For a seamless integration you have to
build SVT-JPEG-XS with MSYS2. The guide for building SVT-JPEG-XS with MSYS2 is given
in [`MSYS2build.md`](MSYS2build.md).


## Encoder

### Supported colour formats
Expand Down
2 changes: 1 addition & 1 deletion Source/Lib/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ add_subdirectory(Decoder/ASM_SSE4_1)
add_subdirectory(Decoder/ASM_AVX2)
add_subdirectory(Decoder/ASM_AVX512)

if(UNIX)
if(UNIX AND NOT MSYS)
if(NOT APPLE)
find_library(M_LIB NAMES m)
if(M_LIB)
Expand Down
Loading