Native DJI Osmo 360 (.OSV) Support: GPU Vulkan Compute Stitching, Zero-Transcode Proxies & Accelerated 360 Reframing

Suggestion
1

Hi everyone,

I’ve been working on native support for DJI Osmo 360 (.OSV) recordings in MLT and Shotcut, eliminating the need to pre-stitch or transcode footage through DJI Studio before editing.

Demo Sample

Attached is an exported sample stitched and reframed entirely within Shotcut using the new pipeline:

• Video: 3840x1920 (4K Equirectangular) @ 29.97 fps, H.264
• Audio: AAC 48kHz Stereo with sample-accurate lip-sync
• Stabilization: Built-in Gyro Horizon Lock using embedded camera IMU telemetry


──────

Architecture & Implementation

The contribution is organized into modular parts:

  1. MLT Framework (modules/osv) — MLT PR Sign in to GitHub · GitHub
    • Dual-stream Demuxing: Demuxes Stream 0 (front lens HEVC), Stream 1 (rear lens HEVC), and Stream 2 (audio).
    • In-tree Telemetry Parser: Zero-dependency parser for camera metadata track (dvtm_oq101) to extract lens centers, focal lengths, distortion parameters, and gyro orientation quaternions.
    • Vulkan Compute Stitcher: Cross-vendor GPU compute pipeline with real-time Horizon Lock and Full Lock stabilization. Embedded SPIR-V bytecode fallback ensures it builds and runs self-contained without
    external shader path configuration.
    • Companion .LRF Proxying: Automatically binds the in-camera 2048x1024 dual-fisheye .LRF companion file during interactive preview for effortless 60fps timeline scrub on modest GPUs.
    • Trim-aware State Tracking: Handles timeline cut changes (I and O trim points) to flush stale frame caches and maintain continuous audio sync.
  2. Shotcut UI Integration — Shotcut PR Sign in to GitHub · GitHub
    • Registers *.osv / *.OSV in file dialogs.
    • Hooks ProxyManager to auto-detect companion .LRF proxy files without transcoding.
  3. Bigsh0t 360 Filter Improvements:
    • Guarded against fov <= 0.0 and NaN parameter resets during trim operations (preventing screen collapse to solid colors).
    • Avoided redundant per-frame map recomputations when camera orientation hasn’t changed.
    • Optimized inner loop coordinate wrapping.
    • (Patch available for Shotcut’s Bigsh0t build recipe).

Feedback and guidance on upstreaming are very welcome!

2

Thanks, I and others have been working on a GoPro MAX(2) workflow that has similar challengees:

https://www.reddit.com/r/ffmpeg/s/4HXerdq5Pc

Good timing! I will try to integrate support for GoPro into your new MLT module.

I think I pinned the bigsh0t version due to a build failure in CI and not desiring anything since that version pinned. Where is your branch?

4

Nice, thanks for that, for sure users of the GoPro will appreciate the effort!

Hopefully we can extract some common code, or share some lessons :slightly_smiling_face:

I’ve created a fork on GH because Bitbucket was not allowing me to, take a look at the first and only PR on fpischedda/bigsh0t