I do not think this is not in our control. It is the way NVENC works. We give it the CQ value, and it does something. Why do you focus on bitrate when you use a quality-oriented variable bitrate mode?
Here is a related bug report on Handbrake
Our version of FFmpeg already includes all of the patches it refers to. So, the workarounds mentioned are not necessary. However, based on one of the last comments Shotcut’s Constrained VBR rate control mode might give different results. I have not yet tried it.
I did say about bitrate but it’s not really important than the fact that cq value higher than 24 did nothing but cq 24 output. Same bitrate and file size for different export settings just can be the evidence for that. And with more testing export with h264_nvenc codec had same issue but up to cq 19.
By the way the source file was recorded by OBS, NVENC HEVC with CQP value of 1. Maybe OBS guys found out the *problems of NVENC with CQ before and didn’t change from CQP to CQ.
But after read the thread you posted I assume that limited buffer size can be the issue and could be solved by constrained VBR setting with controlled buffer size… so yeah. It’s not a bug form Shotcut. That’s for sure. Not sure it’s bug or not after all.
I can test these in future but right now I lose interest with testing cq/cqp values since I find good one for my usage, and that’s not a thing we can dealt with quality-based VBR, I think we can close this topic.
I agree it is confusing and messy. I had read some of these bug reports and issues and did some testing. CQ is preferred over CQP because it gives smaller file sizes for the same quality level (not based on parameter value or Shotcut’s %, one needs to use a video quality metric like PSNR to find comparable qualities).
In case one wants to use CQP with Shotcut 23.12 you can add, for example, to the Other tab (after choosing a preset):
FYI Shotcut’s Constrained VBR mode on NVENC does not use CQ but rather qmin (variable QP via a minimum QP with a maximum bitrate).
I made some tests in Shotcut adding vbufsize= to some high values, in conjunction with CQ VBR, but it does not make a difference. I also tested with something like qmax=1 but it made little difference.
Do users of NVENC prefer having smaller file sizes for a comparable quality - similar to x264 CRF - but only at moderate quality levels? Or is the less confusing and more straightforward CQP preferable?