No image when RustDesk is installed as a service

[asmoday74]

Hi! I have a problem.
RustDesk is installed on approximately 50 identical PCs running Windows 10 Pro. But on one computer the following problem appeared:

• When installing RustDesk as a service, and the “Use hardware codec” option set in the settings, when connecting remotely to a PC, the “Waiting for image” window appears and then nothing happens.
• If you disable hardware acceleration, the picture is there. But for some reason the connection is interrupted every 10-15 seconds.
• If you run the client without installation, under the user, there is also no image.
• If you run the client as an administrator, hardware encoding works.

PC configuration: Intel i5-7400/8 GB RAM/SSD 256 GB/Windows 10 Pro 22H2

In the service logs I found errors in the check-hwcodec-config folder:

[2024-09-02 16:10:12.307020 +03:00] INFO [src\core_main.rs:164] main start args:["--check-hwcodec-config"]
[2024-09-02 16:10:12.406503 +03:00] INFO [C:\Users\runneradmin\.cargo\git\checkouts\hwcodec-74796a7f8f16bbb9\6abd189\src\lib.rs:19] [FFMPEG_VRAM_ENC] FFmpeg vram encoder name: hevc_qsv
[2024-09-02 16:10:12.496940 +03:00] ERROR [C:\Users\runneradmin\.cargo\git\checkouts\hwcodec-74796a7f8f16bbb9\6abd189\src\lib.rs:17] [MFXENC] SyncOperation failed
sts=-17
[2024-09-02 16:10:12.496990 +03:00] ERROR [C:\Users\runneradmin\.cargo\git\checkouts\hwcodec-74796a7f8f16bbb9\6abd189\src\lib.rs:17] [MFXENC] encode failed, sts=-17
[2024-09-02 16:10:12.614065 +03:00] ERROR [C:\Users\runneradmin\.cargo\git\checkouts\hwcodec-74796a7f8f16bbb9\6abd189\src\lib.rs:17] [MFXENC] SyncOperation failed, sts=-17
[2024-09-02 16:10:12.614098 +03:00] ERROR [C:\Users\runneradmin\.cargo\git\checkouts\hwcodec-74796a7f8f16bbb9\6abd189\src\lib.rs:17] [MFXENC] encode failed, sts=-17
[2024-09-02 16:10:12.662082 +03:00] INFO [C:\Users\runneradmin\.cargo\git\checkouts\hwcodec-74796a7f8f16bbb9\6abd189\src\lib.rs:19] [FFMPEG_VRAM_ENC] FFmpeg vram encoder name: h264_qsv
[2024-09-02 16:10:12.904694 +03:00] ERROR [C:\Users\runneradmin\.cargo\git\checkouts\hwcodec-74796a7f8f16bbb9\6abd189\src\lib.rs:17] [MFXDEC] DecodeFrameAsync failed, sts=-17
[2024-09-02 16:10:12.904741 +03:00] ERROR [C:\Users\runneradmin\.cargo\git\checkouts\hwcodec-74796a7f8f16bbb9\6abd189\src\lib.rs:17] [MFXDEC] decode failed, sts=-17
[2024-09-
02 16:10:12.933362 +03:00] ERROR [C:\Users\runneradmin\.cargo\git\checkouts\hwcodec-74796a7f8f16bbb9\6abd189\src\lib.rs:17] [MFXDEC] DecodeFrameAsync failed, sts=- 17
[2024-09-02 16:10:12.933394 +03:00] ERROR [C:\Users\runneradmin\.cargo\git\checkouts\hwcodec-74796a7f8f16bbb9\6abd189\src\lib.rs:17] [MFXDEC] decode failed, sts=-17
[2024-09-
02 16:10:12.958168 +03:00] DEBUG [C:\Users\runneradmin\.cargo\git\checkouts\hwcodec-74796a7f8f16bbb9\6abd189\src\ffmpeg_ram\encode.rs:322] prepare yuv 293.8µs
[2024-09-02 16:10:13.025209 +03:00] DEBUG [libs\scrap\src\common\hwcodec.rs:691] HwCodecConfig { signature: 6474012622650701, ram_encode: [], ram_decode: [CodecInfo { name: "h264",
mc_name: None, format: H264, priority: 0, hwdevice: AV_HWDEVICE_TYPE_D3D11VA }, CodecInfo { name: "hevc", mc_name: None, format: H265, priority: 0, hwdevice: AV_HWDEVICE_TYPE_D3D11VA }, CodecInfo { name: "h264", mc_name: None, format: H264, priority: 3, hwdevice: AV_HWDEVICE_TYPE_NONE }, CodecInfo { name: "hevc", mc_name: None, format: H265,
priority: 3, hwdevice: AV_HWDEVICE_TYPE_NONE }], vram_encode: [FeatureContext { driver: FFMPEG, luid: 56361, api: API_DX11, data_format: H265 }, FeatureContext { driver: FFMPEG, luid: 56361, api: API_DX11, data_format: H264 } ], vram_decode: [DecodeContext { device: None, driver: FFMPEG, luid: 56361, api: API_DX11, data_format: H264 },
DecodeContext { device: None, driver: FFMPEG, luid: 56361, api: API_DX11, data_format: H265 }] }
[2024-09-02 16:10:13.025355 +03:00] INFO [src\ipc.rs:1221] send ok

Any ideas what to do about this? Previously we used TeamViewer/AnyDesk on this PC, no similar problems were observed.

[21pages]

for some reason the connection is interrupted every 10-15 seconds

What does the dialog box display when reconnecting? Could you provide the log C:\Windows\ServiceProfiles\LocalService\AppData\Roaming\RustDesk\log\server\RustDesk_rCURRENT.log? Remember to remove server information.

[asmoday74]

What does the dialog box display when reconnecting?
The dialog box says "Connection error. The connection was reset by the remote node." The requested file is attached.
RDHelp_rCURRENT.log

[asmoday74]

I understand that the problem is most likely on the PC side and is related to a lack of rights. However, I don't quite understand how to fix this. Is the application running as a service, with the "Local Service" account, and so has almost systemic rights. But UAC somehow blocks calls...

[21pages]

[2024-09-03 02:01:37.719270 +03:00] INFO [src\server\connection.rs:3105] #1524 Connection closed: connection manager error: Failed to launch ["--cm"] with session id 1: Запрошенная операция требует повышения. (os error 740)

Yes, the reason of the disconnection is that the cm popup window can't be started. Is rustdesk's session username same as your login user name?

Could you provide another log when hardware codec option is enabled?

[asmoday74]

Yes, the reason of the disconnection is that the cm popup window can't be started. Is rustdesk's session username same as your login user name?

No, the username on the RustDesk server, the username on the remote computer, and the username on my PC are not the same.

Could you provide another log when hardware codec option is enabled?

Of course, the file is attached.
RDHelp_rCURRENT.log

---

Temporary solution: Install the client, disable hardware encoding, and add a new parameter to the registry (REG_SZ) in the following registry branch HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AppCompatFlags\Layers:

Value name: C:\Program Files\RustDesk\rustdesk.exe
Value data: RunAsInvoker

[21pages]

Thanks for the solution.

[asmoday74]

Perhaps not entirely on topic, but for such situations it would be nice to be able to disable hardware encoding via "Strategies".

[21pages]

No, the username on the RustDesk server, the username on the remote computer, and the username on my PC are not the same.

I want to confirm if the rustdesk --server process is in the same session as the computer desktop. That is, the session name in Task Manager is the same as your username when you log into the desktop from the login screen? If not the same, could you make a screenshot?

Strategies

Advanced options in custom client, https://rustdesk.com/docs/en/self-host/client-configuration/advanced-settings/#enable-hwcodec

BTW, could you try https://github.com/21pages/rustdesk/releases/download/nightly/rustdesk-1.3.0-x86_64-waiting-for-image.exe on both side and provide the log? Uninstall the old one, run it without install on the both side, use h265, and provide the log %appdata%/rustdesk/log/RustDesk_rCURRENT.log on the both side.

[asmoday74]

I want to confirm if the rustdesk --server process is in the same session as the computer desktop. That is, the session name in Task Manager is the same as your username when you log into the desktop from the login screen? If not the same, could you make a screenshot?

[21pages]

Thanks, not this problem.

[asmoday74]

BTW, could you try https://github.com/21pages/rustdesk/releases/download/nightly/rustdesk-1.3.0-x86_64-waiting-for-image.exe on both side and provide the log? Uninstall the old one, run it without install on the both side, use h265, and provide the log %appdata%/rustdesk/log/RustDesk_rCURRENT.log on the both side.

Unfortunately, the result is the same. When the hardware codec is enabled, it hangs at the "Waiting for image" status

Remote PC: rustdesk_r2024-09-03_03-59-38.log
My PC: rustdesk_rCURRENT.log

[21pages]

[2024-09-03 03:58:10.773089 +03:00] INFO [src\server\video_service.rs:594] frame encode ok

[2024-09-03 03:58:08.683246 +03:00] ERROR [C:\Users\runneradmin.cargo\git\checkouts\hwcodec-74796a7f8f16bbb9\6abd189\src\lib.rs:17] [FFMPEG_VRAM_DEC] illegal decode parameter

The encode is ok, but the decode is failed. Could you disable hwcodec on your local computer but enable hwcodec on the remote computer and connect again?

https://github.com/rustdesk-org/hwcodec/blob/6abd1898f3a03481ed0c038507b5218d6ea94267/cpp/ffmpeg_vram/ffmpeg_vram_decode.cpp#L174

It's encode failure, because the data length is zero.

[2024-09-02 16:10:12.614065 +03:00] ERROR [C:\Users\runneradmin.cargo\git\checkouts\hwcodec-74796a7f8f16bbb9\6abd189\src\lib.rs:17] [MFXENC] SyncOperation failed, sts=-17

In check hwcodec log, another Intel codec is unsupported.

I'll build a new one for you.

[asmoday74]

Advanced options in custom client, https://rustdesk.com/docs/en/self-host/client-configuration/advanced-settings/#enable-hwcodec

The advanced settings section is good when you have access to a PC. Unfortunately, due to the specific nature of technical support for remote offices, a situation often arises when the program is installed by an unqualified employee who finds it difficult to explain something.
The strategy section in this case would be useful in view of the ability to “remotely” switch the application operating mode.

[rustdesk]

Yes, you are correct.

[asmoday74]

The encode is ok, but the decode is failed. Could you disable hwcodec on your local computer but enable hwcodec on the remote computer and connect again?

Remote PC: rustdesk_rCURRENT.log
My PC:
rustdesk_r2024-09-03_04-20-45.log
rustdesk_rCURRENT.log

In check hwcodec log, another Intel codec is unsupported.
I'll build a new one for you.

Thank you, I'm really looking forward to it! Is this a personal fix or will it appear in the general build?

[21pages]

Thanks for the test, https://github.com/rustdesk-org/hwcodec/blob/6abd1898f3a03481ed0c038507b5218d6ea94267/cpp/ffmpeg_vram/ffmpeg_vram_decode.cpp#L174

This is an encoding failure because the data length is zero. And the decoding doesn't disable the failing codec successfully, I'll give you another program to fix both issues, probably in an hour.

[21pages]

Is this a personal fix or will it appear in the general build?

In the general build
Could you run this one in the controlled side with hardware codec enabled? h264/h265 option may invisible because it's not actually supported, but waiting for image will not happen.

https://github.com/rustdesk/rustdesk/releases/download/nightly/rustdesk-1.3.1-x86_64.exe

[asmoday74]

Could you run this one in the controlled side with hardware codec enabled?

Unfortunately, only in 8 hours. The PC is currently busy. I'll test it in the evening and report back.

[21pages]

Take your time.

[asmoday74]

https://github.com/rustdesk/rustdesk/releases/download/nightly/rustdesk-1.3.1-x86_64.exe

I apologize for the long feedback. I checked the build just now. Works great! Thank you so much for the quick help!