Yes that will cause a size increase.
This is something that someone with more knowledge about the nuances of exporting will need to respond to.
For that I suggest using Avidemux. It is an old application. But will do want you want. You can simply add all your videos and save them in copy mode. It just puts everything together and saves it with no quality loss and virtually the same filesize.
This thread might help with filesize and exort issues. Especially @brian’s post.