Vuze camera ambisonic audio experimentation

3d87c4 wrote on 9/8/2017, 11:16 PM

I am experimenting with ambisonic audio for my Vuze camera using Vegas Pro 14. This is a four sided 3D 360 view camera with 2 cameras per side (for 3D) and microphones in each corner.

I am stitching the video using AVP/Giga and rendering as a 3840x3840 top/bottom full 3D AVI file.

I wrote an FFMPEG script that extracts each microphones audio into separate, mono, audio files. I use the -c:a copy option so the audio is simply copied into an mp4 file without being re-sampled.

I bought a copy of Reaper and installed the Ambisonics Toolkit For Reaper. With this I convert the file to a B-format ambix acn/sn3d format and render this as a 4 channel wav file.

In Vegas, I load the stitched video and wave files into a 3D project, mute the audio that came with the stitched video, and assign each of the ambisonic audio tracks to a buss, per this tutorial:

http://johnrofrano.com/training/video-tutorials/multichannel-render-template-in-vegas-pro/

At this point, in Vegas, everything is working fine & the audio and video are in sync.

I recently installed X264VFW and used this to render a new 3840x3840 top/bottom full 3D AVI file, using the multichannel option to be sure all 4 ambisonic audio tracks are included. I ran it through the Spatial Media Metadate injector, checking all the boxes, and uploaded it to YouTube. The ambisonic audio worked as well as could be expected with 4 on-axis mics, but the audio and video are out of sync!?!?!?

I have checked to be sure the audio sample rate (48KHz) is the same for the media and project, and tried a mind numbing number of variations over the course of the afternoon with the same result.

Eventually, I took another track...I rendered the video and audio to separate files and merged them using FFMPEG, but still had audio/video sync issues (though not, necessarily the same as when rendered from Vegas). After many permutations, and some googling, I delayed the audio by 2 seconds using the -itsoffset 2 parameter prior to inputting the audio file in my FFMPEG script. I've uploaded the video to YouTube & the ambisonic audio seems to be working and the sync seems reasonable.



Any idea why I can't get this to work rendering directly from Vegas? (I realize you probably need more details---I'm too frazzled to add more now, but will gladly answer questions after I've had a bit of a rest.)

Is there another codec I could be using? The spatial media injector accepts MOV files, but I couldn't find a render option that could deal with the large frame sizes and multi-channel audio output.

I want to combine the video and ambisonic audio within Vegas so any edits---trimming, cross fades, etc. will apply to both.

Is there anything in V15 that will help with this? (At this point I'm thinking of buying V15 edit, just to stay in the game, but not install it 'til the initial-release bugs have been worked out.)


 

Last changed by 3d87c4

Del XPS 17 laptop

Processor    13th Gen Intel(R) Core(TM) i9-13900H   2.60 GHz
Installed RAM    32.0 GB (31.7 GB usable)
System type    64-bit operating system, x64-based processor
Pen and touch    Touch support with 10 touch points

Edition    Windows 11 Pro
Version    22H2
Installed on    ‎6/‎8/‎2023
OS build    22621.1848
Experience    Windows Feature Experience Pack 1000.22642.1000.0

NVIDIA GeForce RTX 4070 Laptop GPU
Driver Version: 31.0.15.2857
8GB memory
 

Comments

3d87c4 wrote on 9/10/2017, 11:26 AM

OK...after a lot of experimentation and googling I've learned that the interaction between x264VFW and the host video editor can cause video/audio sync problems. This thread from the SourceForge x264VFW open discussion section proposes two solutions:

https://sourceforge.net/p/x264vfw/discussion/770224/thread/c1dd92db/

The solution I'm following is to change the Output Mode setting to File, instead of VFW. This causes x264VFW to write the video to a file directly, while Vegas writes the audio to the file specified in the render settings. The two processes don't have to interact, so no sync problem is introduced.

I then combine the audio and video using FFMPEG.

 

Del XPS 17 laptop

Processor    13th Gen Intel(R) Core(TM) i9-13900H   2.60 GHz
Installed RAM    32.0 GB (31.7 GB usable)
System type    64-bit operating system, x64-based processor
Pen and touch    Touch support with 10 touch points

Edition    Windows 11 Pro
Version    22H2
Installed on    ‎6/‎8/‎2023
OS build    22621.1848
Experience    Windows Feature Experience Pack 1000.22642.1000.0

NVIDIA GeForce RTX 4070 Laptop GPU
Driver Version: 31.0.15.2857
8GB memory
 

3d87c4 wrote on 9/10/2017, 6:26 PM

Tried out my editing workflow today with multiple clips. Was pleased to learn I can assign multiple audio tracks to a given audio buss. These clips don't have as much audio "directionality" as I'd hoped, but the edit itself went smoothly:


 

Del XPS 17 laptop

Processor    13th Gen Intel(R) Core(TM) i9-13900H   2.60 GHz
Installed RAM    32.0 GB (31.7 GB usable)
System type    64-bit operating system, x64-based processor
Pen and touch    Touch support with 10 touch points

Edition    Windows 11 Pro
Version    22H2
Installed on    ‎6/‎8/‎2023
OS build    22621.1848
Experience    Windows Feature Experience Pack 1000.22642.1000.0

NVIDIA GeForce RTX 4070 Laptop GPU
Driver Version: 31.0.15.2857
8GB memory