However, the video codec keeps going back to “h264_amf”, the audio rate control is set to average instead of the constant I had selected, and dual pass is not selected on the video codec area even though I had it selected when I created the preset.
Where are the custom presets stored on my PC? I was thinking a workaround in the short term would be to manually edit the file to have the settings I want.
Unfortunately they show the correct values (well by correct I mean they show the same values as what the preview screen says when you create the custom preset), its just not being loaded correctly in shotcut when selected.
This is because you have hardware encoder checked. That behavior is not going to change.
That is because your preset has ab=384k. The aac codec does differentiate between constant and average: https://trac.ffmpeg.org/wiki/Encode/AAC#NativeFFmpegAACEncoder
Shotcut does not try to make the whole encode parameters UI smart about every combination of codec and parameters. That would be an immense amount of code to write, debug, and maintain. It presents generic rate control UIs, maps those to generic as well as some (popular) codec-specific options. Then, when reading a preset, it reads parameters, some of which are generic like ab here and updates the UI accordingly.
That is because hardware encoder is enabled, and dual pass is not supported with hardware encoding.