Sorry for the late reply! Your tip was very helpful.
Here is my log.
plugin_mgr_get_object_file_plugins: error opening shared object file 'D:\Programs\Shotcut\lib\ladspa/AUTHORS': "D:\Programs\Shotcut\lib\ladspa\AUTHORS": The specified module could not be found.
plugin_mgr_get_object_file_plugins: error opening shared object file 'D:\Programs\Shotcut\lib\ladspa/COPYING': "D:\Programs\Shotcut\lib\ladspa\COPYING": The specified module could not be found.
plugin_mgr_get_object_file_plugins: error opening shared object file 'D:\Programs\Shotcut\lib\ladspa/readme.txt': "D:\Programs\Shotcut\lib\ladspa\readme.txt":
[h264 @ 00000000054d4f00] Reinit context to 1920x1088, pix_fmt: yuv420p
[h264 @ 00000000054c4e80] Reinit context to 1920x1088, pix_fmt: yuv420p
[h264 @ 00000000054cc6c0] Reinit context to 1920x1088, pix_fmt: yuv420p
[producer avformat] audio: total_streams 2 max_stream 2 total_channels 4 max_channels 2
[AVIOContext @ 000000000540a6c0] Statistics: 2504445 bytes read, 2 seeks
[AVIOContext @ 000000000978ee00] Statistics: 2504445 bytes read, 2 seeks
[AVIOContext @ 0000000004aece80] Statistics: 2504445 bytes read, 2 seeks
[h264 @ 00000000054f8180] Reinit context to 1920x1088, pix_fmt: yuv420p
[h264 @ 00000000054f6ac0] Reinit context to 1920x1088, pix_fmt: yuv420p
[h264 @ 00000000054d80c0] Reinit context to 1920x1088, pix_fmt: yuv420p
[producer avformat] audio: total_streams 2 max_stream 2 total_channels 4 max_channels 2
[AVIOContext @ 00000000054c3c80] Statistics: 2504445 bytes read, 2 seeks
[AVIOContext @ 0000000009ccae00] Statistics: 2504445 bytes read, 2 seeks
[AVIOContext @ 0000000004aeca80] Statistics: 2504445 bytes read, 2 seeks
[h264 @ 00000000054f8640] Reinit context to 1920x1088, pix_fmt: yuv420p
[h264 @ 00000000054f5400] Reinit context to 1920x1088, pix_fmt: yuv420p
[h264 @ 000000000979ac40] Reinit context to 1920x1088, pix_fmt: yuv420p
[producer avformat] audio: total_streams 2 max_stream 2 total_channels 4 max_channels 2
[AVIOContext @ 00000000054c3c80] Statistics: 2504445 bytes read, 2 seeks
[AVIOContext @ 00000000097aeec0] Statistics: 2504445 bytes read, 2 seeks
[AVIOContext @ 0000000004aeca80] Statistics: 2504445 bytes read, 2 seeks
[h264_nvenc @ 00000000054f5d00] Loaded Nvenc version 9.0
[h264_nvenc @ 00000000054f5d00] Nvenc initialized successfully
[h264_nvenc @ 00000000054f5d00] 1 CUDA capable devices found
[h264_nvenc @ 00000000054f5d00] [ GPU #0 - < GeForce RTX 2080 Ti > has Compute SM 7.5 ]
[h264_nvenc @ 00000000054f5d00] OpenEncodeSessionEx failed: out of memory (10)
[h264_nvenc @ 00000000054f5d00] No NVENC capable devices found
[h264_nvenc @ 00000000054f5d00] Nvenc unloaded
[h264 @ 00000000054f61c0] Reinit context to 1920x1088, pix_fmt: yuv420p
[h264 @ 00000000054f7400] Reinit context to 1920x1088, pix_fmt: yuv420p
[h264 @ 0000000005544c40] Reinit context to 1920x1088, pix_fmt: yuv420p
[producer avformat] audio: total_streams 2 max_stream 2 total_channels 4 max_channels 2
[AVIOContext @ 00000000054c3c80] Statistics: 2504445 bytes read, 2 seeks
[mp3 @ 000000000b007040] Skipping 0 bytes of junk at 1003.
[mp3 @ 000000000a0a5980] Skipping 0 bytes of junk at 1003.
[producer avformat] audio: total_streams 1 max_stream 0 total_channels 2 max_channels 2
[AVIOContext @ 00000000093b0500] Statistics: 98304 bytes read, 1 seeks
[filter swresample] 2(stereo) f32le 48000Hz -> 6(5.1) f32le 96000Hz
[producer avformat] D:/Videos/Shotcut/Close_Ending/Apex Legends 2019.mp4
checking VFR: pkt.duration 256
[filter swresample] 2(stereo) f32le 48000Hz -> 6(5.1) f32le 96000Hz
[h264 @ 00000000055470c0] Reinit context to 1920x1088, pix_fmt: yuv420p
[mp4 @ 00000000054c7e40] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.
[mp4 @ 00000000054c7e40] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.
[h264 @ 00000000055470c0] Reinit context to 1920x1088, pix_fmt: yuv420p
[h264 @ 00000000055470c0] Reinit context to 1920x1088, pix_fmt: yuv420p
[h264 @ 00000000055470c0] Reinit context to 1920x1088, pix_fmt: yuv420p
The important lines are the following:
[h264_nvenc @ 00000000054f5d00] Loaded Nvenc version 9.0
[h264_nvenc @ 00000000054f5d00] Nvenc initialized successfully
[h264_nvenc @ 00000000054f5d00] 1 CUDA capable devices found
[h264_nvenc @ 00000000054f5d00] [ GPU #0 - < GeForce RTX 2080 Ti > has Compute SM 7.5 ]
[h264_nvenc @ 00000000054f5d00] OpenEncodeSessionEx failed: out of memory (10)
[h264_nvenc @ 00000000054f5d00] No NVENC capable devices found
[h264_nvenc @ 00000000054f5d00] Nvenc unloaded
And as I found out, for some reason or another, you need to disable NVIDIA Expirience Instant Replay before starting encoding the video. It seems to block access to NVENC device for the Shotcut’s encoding.
With that trick, encoding would work fine, as it did before.