HyperHDR is Faster than the TV

[ghfb2020]

I first want to say that I love HyperHDR, and I want to thank everyone who contributes, especially awawa-dev.

I have been having a problem that I don't think I have seen in the discussion. I tried to search, but I am not sure what to search for. First, my rig is the following,

HDMI switch >
EZCOO HDMI Splitter>
MS2109>
Raspberry Pi 4 running HyperHDR 19.0>
ESP 32 running HyperSPI 9.0>
Level Shifter>
SK6812 with 232 LEDs

I have the same set up in another room but instead of the switch it's a AV Receiver and it has 270 LEDs.

My problem is that the lights are appearing slightly before the television. I had this same problem when I was on HyperHDR 18.0, but when I upgraded to 19.0 it was fixed for at least a day or two before the tv fell behind the lights. It was perfectly synced for that day or two, so I feel like this is a problem that can be solved, but I don't really understand how HyperHDR works. I just follow the instructions. The effect that the lights have when they are perfectly synced is far superior to when the TV is lagging behind them, so any help would be greatly appreciated.

Thanks again for your help, and all of the hard work you put into this. I love it.

[awawa-dev]

Thank you :) Well, we usually fight with the opposite problem in the configuration (the LED system lags behind the TV) but in some configurations a situation like yours can also occur. This is usually caused by heavy TV image processing e.g. if we turn on the advanced features of the TV's cinema mode then the delay can be even more than 200ms, which the TV needs to process the signal. For starters, I can suggest the opposite of the standard hints: increase the delay(time) in the smoothing settings (don't reduce the frequency there, it should still be high) and switch smoothing algorithm from "advanced linear" to "linear", in the grabber: set the capture resolution to high , try different codec e.g. MJPEG, do not enable "Quarter of frame mode", use lower framerate or set 'Software frame skipping' higher than 1. Start with the smoothing delay alone, I think it should help.

[awawa-dev]

Looks good, great job! I will take a look later. I'm not 100% sure if capturing a parameter by reference (&) in the lambda that emits the signal is always safe here when the function has already exited? Maybe use the default capture by copy (in terms of performance, it's negligible here)? Since we already have a parameter related to time at the beginning of the smoothing panel maybe we can add a delay at the very end of the scheme (after "continuousOutput") so that the user doesn't confuse them?

[acortelyou]

Thanks!

I added the & to the lambda because the compiler was complaining about not having access to this for _hyperhdr.

C++ is not my native tongue so I'm unsure about the implications here, perhaps it would be better to explicitly pass only the reference to _hyperhdr?

I'm content to put the option wherever, but to me it makes sense logically and aesthetically for updateFrequency and updateDelay to be near each other... Maybe I could just move it to after updateFrequency?

Aside: super excited to have separate LUTs for SDR and HDR. Been using various forks of hyperion for a decade now and automatically calibrated LUTs are by far the biggest step forward I've seen the entire time. Also parallel segments + hyperspi... just... thank you. Great work.

[awawa-dev]

I added the & to the lambda because the compiler was complaining about not having access to this for _hyperhdr.
C++ is not my native tongue so I'm unsure about the implications here, perhaps it would be better to explicitly pass only the reference to _hyperhdr?

Yes I understand and there are some traps here related to both to c++ and QT. Rather the LedColors parameter and not the _hyperhdr is the main problem (although it is used without the smoothing object's context, even though the lambda is very simple, it only emits a signal here, if the lambda is executed without context, when the smoothing object is destroyed it can lead to an exception) when LedColors is captured by reference. I think you can use copy capture which should create a copy of the ledColors structure. You can also use context (to execute it within the smoothing thread) and a more precise timer.
QTimer::singleShot(_updateDelay, Qt::TimerType::PreciseTimer, this, [=]() { emit _hyperhdr->ledDeviceData(ledColors); });
Let me know if it's working for you.

I'm content to put the option wherever, but to me it makes sense logically and aesthetically for updateFrequency and updateDelay to be near each other... Maybe I could just move it to after updateFrequency?

Situations where LEDs are ahead of the TV are rather rare. This is usually due to heavy TV video processing (TV can achieve latency as high as over 200ms in the cinema mode) and very fast video grabber and LED driver: I experienced this situation using Ezcap 320 and Elgato HD60X with my LG in cinema mode. But only for HyperSPI. Using the same setup Philips Hue lamps were slightly behind the TV stream. So I rather want to move it behind the more commonly used option in smoothing. I haven't decided yet, just thinking.

Aside: super excited to have separate LUTs for SDR and HDR. Been using various forks of hyperion for a decade now and automatically calibrated LUTs are by far the biggest step forward I've seen the entire time. Also parallel segments + hyperspi... just... thank you. Great work.

Thank you :)

[acortelyou]

Small update.

QTimer::singleShot(_updateDelay, Qt::TimerType::PreciseTimer, this, [=]() { emit _hyperhdr->ledDeviceData(ledColors); }); does compile and work.

I am able to adjust the delay and on some frames it's near perfect.

However, I'm noticing now though with the highspeed video that the delay does not seem to be constant.

Most transitions are perfect, some are ahead, some are behind.

Not quite sure what's going on at the moment.

Also I'd like to find a way to change the delay with no smoothing, even if it's just on my own fork.

Pi4, ezcap322, 1280x720x60 NV12, Quarter of frame, 25ms time, 25ms delay, 80hz freq, 309 leds.

[awawa-dev]

It may be related to how linear smoothing works. There are two versions in HyperHDR smoothing configuration: standard and advanced. The latter should react faster to changes. Another possible reason is that the task queue is clogged by multiple singleshot calls, and then it should be implemented differently with a single colorlist queue (with set maximum size constraint) and a single timer that will pop them off and send them to the led device.

And if you want to test how it works without a smoothing, you can put these 3 lines into a delayed hardcoded singleshot here:

HyperHDR/sources/base/ImageProcessingUnit.cpp

Line 92 in 7df7fcf

_hyperhdr->updateLedsValues(_priority, colors);

[wiesner-philipp]

Heya, would it be possible for this to be added into an early beta?
I have the same issue as @ghfb2020, my LEDs are like 6-8 frames ahead.

Hyperion has a function to delay the LEDs on a frame by frame basis.

This is especially great for syncing multiple different outputs (SPI, PWM, Network, etc.).

[dmytro-frolov]

Hey @awawa-dev and @acortelyou

Thanks @awawa-dev for the continuous contributions

I was looking at QTimer::singleShot myself, before I stumbled on this post, but unfortunately, I'm not proficient in C++,

Do you have a new version or other suggestions? I saw in the thread that you faced some inconsistency while using your PR.

I have a similar inconvenience, my LEDs are faster than a TV display. In my case, LEDs ignite ~300ms faster than the display's image.
Unfortunately smoothing solution just smoothes slowly the frames change for example from white to black.

Propose

I'm not sure that delay in smoothing is the right place as in HyperionNg, I think it's better to be inside device section, IMHO not all people like smoothing effect.

Add text input called "Delay" or something similar to Led Hardware -> Led Controller. This input field is responsible for delaying frames sent to the LED device, in this case hyperserial-esp32.

Why this is the issue / Background

I have a build in android TV in my TV, it's really fast, since it's android I use hyperion android graber app. The app sends data via Protocol Buffers to HyperionHDR.

[awawa-dev]

Hi
The LED rendering model was recently rewritten to provide rock-stable interval timings. As a side-effect all previous not finished attempts became obsolete. Some work has started previously here #584 and I will continue in the future. I am inclined to not use a delay but a queue, although it will increase the level of difficulty because it will have to be controlled/synchronized (e.g. empty before turning off the device)

[profesaurus]

Hey, I know this is an old issue and it may be worked on in the future, but just wanted to comment that I'm having this issue as well. I'm running v21.0.0.

I'm using the Android Hyperion Grabber on a Shield 2019 and HyperSerialPico is running the leds and HyperHDR is running as a home assistant addon on a Raspberry Pi 4. In the Shield menu everything seems to be synced just fine, but when I play any video, the leds are a fraction of a second quicker than the TV.

I've tried everything in this thread without success.

[ghfb2020]

Have you tried putting your TV in game mode? That solved it for me when nothing else was working.

[profesaurus]

That fixed it, thank you! I wish we could use any mode, but this is a good fix for now.

[Nastyfox]

Hello, I have the same issue using Hyperion Android Grabber and a TCL TV. I can't put the TV in game mode if it's not coming from HDMI. What could I do to avoid the LED being lit before the image displayed on TV ?

Thanks in advance for your answers.

[santievil]

@gghisellini To avoid duplicating threads. And in case you want to take a look.

I’ve been doing some tests on the latest beta and it seems feasible to create a FIFO queue with a size equal to the number of frames to delay.
I added it to the smoothing stage because it’s the last thing that’s done before passing the colors (if I’ve understood the structure correctly…), and because pulling it out as a separate step would be a bit more complicated.

I’ve been testing it and, in my case, with 4–5 frames it works well (smooth colors with a delay, no performance drops).

Anyone up for it?