Spatial audio (for 360 video) best workflow?

DDDyson wrote on 8/11/2016, 12:15 PM

I have a Samsung Gear 360 camera and a Zoom H2n audio recorder. The simplest, most straightforward way to record panoramic video with spatial sound, as supported by YouTube on Chrome, and Oculus Video on the Samsung Gear VR.

But the spatial audio format recorded by the Zoom H2n Firmware 2.0 is Ambisonic B-format, 4 channels. Vegas Pro only supports stereo or 5.1 surround. If I import a 4-channel wav, I get 4 grouped mono tracks, on separate tracks.

What's the recommended workaround workflow for editing these?

The best I could think of is:
- set project to 5.1 surround
- import the 4-channel wavs in default channel order, do the edits, syncs and crossfades as needed
- render video with 5.1 audio
- use a tool (like FFMPeg) to strip the 2 extra channels from the video stream

Caveats:
- I don't know what the channel order should be (which 5.1 channel should each mono track be assigned to, so they get rendered in the right order)
- I don't know if I can actually strip/remove channels from a video+audio stream file

Any recommendations? Thanks!

Comments

rraud wrote on 8/11/2016, 12:50 PM

You can ignore the LF FX channel and the center channel. If there's dialog, this is typically in the center channel only). Otherwise two stereo pairs, front and rear.

DDDyson wrote on 8/11/2016, 1:10 PM

The B-format is channel order specific, it's not a front-rear stereo pair.

Channel:
0 = omni (overall amplitude)
1 = horizontal (left/right) dipole
2 = vertical dipole
3 = front-rear dipole

The channels need to be in this specific order, and the video must have only 4 channels of audio, for the result to work in YouTube (Chrome) or in Samsung Gear VR with the rotating virtual surround.

Here's an example video. If you're using Chrome, you should hear the soundscape rotate as you rotate the video (you'll notice it on headphones):

[url=

The above video is an un-edited full take - quite simply because I still lacked the means to edit and combine multiple takes. I used MyFFMpeg and Reaper to combine the Zoom audio recording to the video.

So the question is: what order / what speakers should I assign each of the 4 tracks to, so that they get rendered to this order in the 5.1 video (out of which I can then strip the last 2 channels)?

DDDyson wrote on 8/12/2016, 9:50 AM

OK, I tried rendering a bunch of times out of Vegas, and turns out the correct channel order (so that the imported 4-channel wav ends up in the same order as) is:

L, R, Center, LFE

So, one of the mono tracks needs to be assigned to LFE. Alas, even with the LFE low pass filter disabled, if I assign a track to the LFE channel it gets rendered out at lowered gain, which messes up the spatiality, so in practice I can't render the 4 channels in the right order.

In any case, even if the LFE didn't do that, I'd still have to render the whole 5.1 audio into a wav, then import this to Reaper, render it out as a 4-channel wav with the proper channel order, then use FFMpeg (or such) to replace the audio in the video rendered from Vegas.

I cannot render out a 4-channel wav from Vegas. Vegas says "this template is invalid for the current audio settings, the template has been removed from the list". WTF. Even if I could, Vegas is quite inflexible what comes to assigning channels - I can only render in either Stereo, or 5.1 (with specific channel order).

Re: video rendering: MainConcept MP4 supports the Sansung Gear 360 resolution (3840 x 1920) but only Stereo audio. Sony AVC and XAVC don't support 3840x1920. But I can use FFMPeg to replace the stereo audio in the mp4 with an external file.

Gee whiz, I was hoping that Vegas Pro would be more flexible with formats than the Movie Studio I upgraded from, but seems like it's only ever so slightly. :(

Former user wrote on 8/12/2016, 4:13 PM

This shows you how to render to a multitrack file other than 5.1. You must ENABLE Multichannel and assign channels to render, as well as send a track to a bus.

[url=http://www.sonycreativesoftware.com/webhelp/vegaspro/12/ENU/Content/Rendering_Multichannel_Audio_Files.htm]

DDDyson wrote on 8/14/2016, 2:41 AM

Thank you for the tip, and the link!

With these instructions, I won't be able to render out a video with 4-channel audio directly (at least not at the resolution I need - MXF only goes up to 1920x1080), but I can render the audio track out separately as a 4-channel WAV which I can then replace and multiplex into the video file using (My)FFMpeg.