Also try the w3fdif filter.
This is the one developed and open sourced by the BBC.
@st599 I gave the w3fdif filter a try and the video jumps up and down, probably at 30 frames/second. Frame-by-frame it pops up, then down a line, in a very obvious and disconcerting way! I am suspicious of my camcorder misbehaving on a cold, miserable day since all the other video from that machine is quite good.
-=Ken=-
I finally got rid of the unusual interlacing in some of my Hi8 video. Importing the newly deinterlaced clip synchronized on Shotcut’s track V2 and toggling the view/hide button shows a considerable difference.
Thanks for all the suggestions.
-=Ken=-
After restoring my badly interlaced footage I was able to continue my project. Now, I’m able to see if I can do better using the dnxhd intermediate codec at 720p. I took some of the other, undamaged footage and used FFmpeg to save out a 90 second long clip using the yadif, bwdif and w3fdif deinterlace filters and placed them on a Shotcut 720p Timeline. I also imported the same segment from the original 480p for comparison. I have to say that the 720p clips actually look better than the original. Astonishing! Comparing the deinterlace filters I found the yadif and bwdif filters looked about the same to me. Since the bwdif is the most recently developed, building upon both of the earlier filters I will use it going forward. The w3fdif filter had a slight “bounce” to the picture which was not that bad in normal play but stepping frame-by-frame, the auto-datestamp bounced up and down - kinda weird. There was some definite field gymnastics going on in that filter that the newer bwdif did not show.
Thanks for all the help.
-=Ken=-
Your experience is consistent with my own.
One thing I like about upscaling to 720p is that it converts from non-square to square pixels. Progressive formats with square pixels is the way of the future. I predict that over time device/player support for interlaced and non-square pixels will regress. That is why I choose to deinterlace and upscale all my SD material.
Well, I am convinced the deinterlacing and upscaling to an intermediate codec is superior. So, I added up all my Hi8 video and it is 367 MB. The dnxhd codec writes files that are ~7 times larger which means if I want to archive them in that format I would need 2.6 TB of disk space. Holy smokes!
It doesn’t sound practical.
FFmpeg utilizes all four of my Ryzen 3 cores but a 25 GB tape takes over an hour to convert. I have waited all these years to edit these tapes so the time it takes to convert them is not so difficult. If I can brush up on my Bash scripting I could just let the PC run in a Do loop. It’s just that I presently don’t have THAT much disk space available.
Suggestions?
-=Ken=-
(assuming 23.97 FPS)
How about DNxHD45 ??
1280x720p
4:2:2 Chroma
8 bit
3.5 Minutes per Gigabyte
or if you must have in 10 bit, then:
DNxHD90x . (note the x which denotes 10 bit)
1280x720p
4:2:2 Chroma
1.6 Minutes per Gigabyte
Hi Paul,
Well, I had to look up your claims and I easily found them on a Wikipedia page. I am using 720p/59.94 since the deinterlacing filter writes out a frame for every field. According to the chart my configuration is writing 1.402 Megabytes per minute. It’s 8 bit since this is dealing with amatuer home video. So, your suggestion won’t help. Thanks for trying though.
After thinking about the logistics I have concluded that I will only write out to a 720p intermediate file those sections of my tapes that I will add sequentially to my Timeline. First rough cut in SD and then replace with the HQ version. Once a project is completed the intermediates will have to go away. Such is life!
-=Ken=-
Does more disk space count as a suggestion? an 8tb Mybook(which has a standard 3.5 inch wd helioseal drive in it) is about 150 and would give you more than enough room to work with. https://amzn.to/2rhRQHf
What exactly are your “archive” goals? I prefer to edit with a lossless intra frame codec like dnxhd. But I archive my final videos using H.264.
Your original DV video only has a chroma format of 4:1:1. So if you archive it as upsampled dnxhd, you are storing a lot of redundant information.
My advice would be to edit your files in the highest quality that you and muster, but then export them using H.264 4:2:0 with 60 frame GOP and 2 bframes. For 720p video, 20Mbps would be perceptually lossless given your source quality. And you wouldn’t be losing any chroma resolution because the source was 4:1:1.
@D_S, a good idea. I could bump my secondary 1/2 TB drive to 4 TB for under $100 usd. But grandpa’s ‘toys’ account has run pretty lean in this Christmas season.
@Brian - Archive plan? Of course I have an archive plan. It’s just that it keeps changing 
. The initial strategy was to make a DVD of each grandchild over the last 21 years. I have a paper index of each tape so I can find the segments I need which span many of the 17 tape/captures but it means either transcoding all the 2 hour captures ahead of time, or transcoding the segments as I need them.
I hadn’t decided the final product. Initially, I was going to make flash drives. But I have never seen gift sets of family flash drives. Despite the onslaught of history and technology there is nothing like a DVD or Blu-ray disk in a bookshelf display case. Flash drives tend to get put in a drawer and get lost. A suitable tag on them is several times the size of the flashdrive. Ugh!
I do thank you for revealing your process. I am taking careful notes since I don’t want to reinvent that wheel all over again.
The conventional wisdom seems to be that we archive ALL family footage. Someday, somebody might want to do something with it. It’s obvious though that it can’t be archived in an intermediate form because it is too large. And not THAT valuable…
-=Ken=-
I would never trust a DVD or a thumb drive for data archival, but M-Disc does look promising.
I see. You have two use cases going on here:
- Temporary storage of intermediate files so that you can sort through them all to use individualized clips of each recipient.
- A final export format and storage media to be distributed to the recipients
For the first use case, your goal would be to maximize quality, maintain edit friendliness and be able to fit all the files on the storage device you have available on your editing machine. For this, I could recommend intra-coded H.264. Just do some math to figure out what the highest bitrate you can use without filling up your hard drive. For 720p, I bet you could go as low as 50Mbps and keep perceptibly perfect intermediate files.
For the second use case, I would highly discourage any optical media. Your grandchildren will probably never own optical drives. I would encourage you to look at USB flash drive gift sets:
https://www.etsy.com/listing/619054093/wood-usb-box-with-personalization-usb
https://www.amazon.com/LONMAX-Wooden-Flash-Drive-Memroy/dp/B01N6JHTGC
https://www.amazon.com/Flash-Drive-Display-Memories-Bow-Tied/dp/B076BB11TY
For encoding, you would want a format that will be compatible with as many devices as possible. I would suggest MP4 file with H.264 60 frame GOP at 8Mbps. If you use the x264 encoder in 2 pass mode, the quality of 720p at 8Mbps will be astounding and it will play on any device for the foreseeable future.
Just to add to what @Hudson555x and @brian have said I work in IT(and have worked with archival data before) it’s been my experience that dvd’s lose data rather rapidly thanks to the organic dyes used, we had a large system with about 350 dvd’s and 10 years worth of data but only ~50% of that was accessible since they hadn’t bothered to use any form of disc better than whatever was cheapest. You can get gold foil archival quality dvd’s but I’m not sure I’d trust those any better.
Regarding the flash drives depending on how many, and how often you make them it may be worthwhile to order blanks in bulk and get a CNC laser, making graphics is easy enough thanks to gimp/inkscape(and if you can handle what you’re doing in this thread I bet you can handle that. A small laser capable of engraving wood only costs 180 or so https://amzn.to/2Q8UTAF and at that point if you did just 10 flash drives you’d break even using a bulk pack like this https://amzn.to/2Q6jbeq past that you’re saving a ton.
I too had the cheapest CD\DVD’s backing up data over a decade ago, 2 discs out of 20 readable a few months ago.
One thing I do suggest going down the USB thumb drive route (while I don’t trust them, I never had one fail) is sealing up in a water tight container, which would seal out the moisture and it would still be good if got lost 10 years from now because of the case. Or if they ammunition containers, make sure they put that inside their ammo box for secure storage.
Wow! Do you realize this forum remains one of the best on the Internet?
You folks here are down right amazing!
I missed seeing the flashdrive gift sets. What a novelty! Always something new under the sun. OK, that makes better sense. I’ll be checking that out real soon.
I guess I hadn’t really said what my archive plan was. 17 2-hour Hi8 tapes scanned in as .DV files and 7 HDV files - all archived, for now, to a 2 TB Seagate GoFlex Home network NAS and a 1/2 TB WD Elements USB3 which is headed for the safe deposit box. In addition I have a 1 TB WD Elements USB3 local/working backup drive. So, three backups of the originals .DV files.
This is a hobby. Finishing the “presentation flashdrives” project could take a couple years. That’s why I was thinking of wholesale making all 30+ hours of intermediate files until I knew I was really done. I guess I figured I could archive ISO files and burn new DVDs as I needed to in the future. I think you kind folks have disabused me of that notion.
Bravo - denizens of the Shotcut Forum - you have done it again!
Thank you,
-=Ken=-
@kenj69 Keep in mind spinning discs don’t like bumps so if you’re going to keep one in a safe deposit box look towards the few that are shock mounted
@Hudson555x I actually have had USB flash drives fail, it wasn’t moisture though(as long as they aren’t connected while wet water is a non issue I’ve washed a few) but Heat causes random bit flips over time as I’ve more than tested with a flash drive attached to my car stereo slowly corrupting the mp3 files on hot summer days. If you do plan on using them for long term storage I suggest purchasing them from a T1 or T2 vendor (sandisk, kingston, mushkin) and skipping the generics which are that much easier to cook
I have finally found the proper filter settings for FFmpeg to match those I found in Avidemux. Someone in the Avidemux forum kindly showed me where they are “hidden.” Believe it or not, they are under Video Filters 10.140 named “pp.” They are called postprocessing subfilters. The command that I was looking for will appear like this:
-vf cubicipoldeint|autoq - or in abbreviated form -vf pp=ci|a [that is c, i, vertical brace and a]
http://ffmpeg.org/ffmpeg-filters.html#pp
So, a more complete command line would be:
ffmpeg -i in.dv -c:v dnxhd -vf “pp=ci|a,scale=1280x720,format=yuv422p” -b:v 110M out.mov
For the rest of my footage the bwdif filter works great!
Concerning flash drive gift sets: A local store here sells little wooden boxes of various sizes for arts/crafts and so on. It seems to me that I can buy a premium flash drive and apply some sort of sticker on the box and drive - better quality, lower cost, less hassle. Glad to know about this!
-=Ken=-
@Hudson555x, @D_S it should not alarm you, but I participate in other forums than this one. I just had a discussion in Reddit/r/videography subreddit with someone who seems to have considerable experience with optical media. He makes compelling claims that optical media IS viable for today and into the foreseeable future and IS reliable for archiving family memories!
Now, my intention is not “you and he go fight,” but I really don’t know what to make of these conflicting claims. I used to work in IT as well and tend to trust you guys. Could you guys please check this out? Thanks -=Ken=-