How to properly reframe 360 equirectangular video?

relaxvideo wrote on 12/9/2023, 4:44 AM

I have Insta X3 cam and recorded a long music event. I like to play with the sphere while editing on Vegas timeline.

However i got pixelated, jagged, blurry export while adding Tiny Planet effect and reframe my 5760x2880 pixel equirectangular video, exported from Insta Studio app. Quality is much-much better when i reframe in Insta app and export here. But i like to have the whole sphere in Vegas and decide here, how to reframe while editing.
I discovered that when my project properties is set to 1920x1080, exported video will be bad quality. Better, when i set project properties to 3840x2160, but still not as sharp as from Insta app. Seems like tiny planet effect rescale source file to project size first and only then apply the fx. Is it true? I cannot set my project properties to full 5760x2880, because i export with Voukoder, which works at project size dimension.

Also: where can i set projection type: fisheye, linear, etc. like in insta app? possible?

Thanks!

Comments

set wrote on 12/9/2023, 5:11 AM

Put the Tiny Planet OFX BEFORE Pan/Crop Chain.

That will ensure you are processing on the source resolution.

 

More explanation and tips from me:

 

https://www.vegascreativesoftware.info/us/forum/post-your-vegas-creations-2--141186/?page=2#ca885312

 

Right now the projection opt you set is just fisheye in VEGAS Pro.

Last changed by set on 12/9/2023, 5:12 AM, changed a total of 1 times.

Setiawan Kartawidjaja
Bandung, West Java, Indonesia (UTC+7 Time Area)

Personal FB | Personal IG | Personal YT Channel
Chungs Video FB | Chungs Video IG | Chungs Video YT Channel
Personal Portfolios YouTube Playlist
Pond5 page: My Stock Footage of Bandung city

 

System 5-2021:
Processor: Intel(R) Core(TM) i7-10700 CPU @ 2.90GHz   2.90 GHz
Video Card1: Intel UHD Graphics 630 (Driver 31.0.101.2127 (Feb 1 2024 Release date))
Video Card2: NVIDIA GeForce RTX 3060 Ti 8GB GDDR6 (Driver Version 551.23 Studio Driver (Jan 24 2024 Release Date))
RAM: 32.0 GB
OS: Windows 10 Pro Version 22H2 OS Build 19045.3693
Drive OS: SSD 240GB
Drive Working: NVMe 1TB
Drive Storage: 4TB+2TB

 

System 2-2018:
ASUS ROG Strix Hero II GL504GM Gaming Laptop
Processor: Intel(R) Core(TM) i7 8750H CPU @2.20GHz 2.21 GHz
Video Card 1: Intel(R) UHD Graphics 630 (Driver 31.0.101.2111)
Video Card 2: NVIDIA GeForce GTX 1060 6GB GDDR5 VRAM (Driver Version 537.58)
RAM: 16GB
OS: Win11 Home 64-bit Version 22H2 OS Build 22621.2428
Storage: M.2 NVMe PCIe 256GB SSD & 2.5" 5400rpm 1TB SSHD

 

* I don't work for VEGAS Creative Software Team. I'm just Voluntary Moderator in this forum.

relaxvideo wrote on 12/9/2023, 6:02 AM

Oh, thanks, i totally forget that!
With lens correction fx i cannot get the same great correction as in Insta.
So this is a feature request :)

Last changed by relaxvideo on 12/9/2023, 6:03 AM, changed a total of 1 times.

#1 Ryzen 5-1600, 16GB DDR4, Nvidia 1660 Super, M2-SSD, Acer freesync monitor

#2 i7-2600, 32GB, Nvidia 1660Ti, SSD for system, M2-SSD for work, 2x4TB hdd, LG 3D monitor +3DTV +3D projectors

Win10 x64, Vegas22 latest

set wrote on 12/9/2023, 3:47 PM

+1 for that projection type requirement choices.

 

However, the more I experiment with it, let's say 'linear', I feel a bit 'uncomfortably not natural feel' due to over-wide result - and the fish eye feels more 'natural'. - but I guess this is personal preferences.

Setiawan Kartawidjaja
Bandung, West Java, Indonesia (UTC+7 Time Area)

Personal FB | Personal IG | Personal YT Channel
Chungs Video FB | Chungs Video IG | Chungs Video YT Channel
Personal Portfolios YouTube Playlist
Pond5 page: My Stock Footage of Bandung city

 

System 5-2021:
Processor: Intel(R) Core(TM) i7-10700 CPU @ 2.90GHz   2.90 GHz
Video Card1: Intel UHD Graphics 630 (Driver 31.0.101.2127 (Feb 1 2024 Release date))
Video Card2: NVIDIA GeForce RTX 3060 Ti 8GB GDDR6 (Driver Version 551.23 Studio Driver (Jan 24 2024 Release Date))
RAM: 32.0 GB
OS: Windows 10 Pro Version 22H2 OS Build 19045.3693
Drive OS: SSD 240GB
Drive Working: NVMe 1TB
Drive Storage: 4TB+2TB

 

System 2-2018:
ASUS ROG Strix Hero II GL504GM Gaming Laptop
Processor: Intel(R) Core(TM) i7 8750H CPU @2.20GHz 2.21 GHz
Video Card 1: Intel(R) UHD Graphics 630 (Driver 31.0.101.2111)
Video Card 2: NVIDIA GeForce GTX 1060 6GB GDDR5 VRAM (Driver Version 537.58)
RAM: 16GB
OS: Win11 Home 64-bit Version 22H2 OS Build 22621.2428
Storage: M.2 NVMe PCIe 256GB SSD & 2.5" 5400rpm 1TB SSHD

 

* I don't work for VEGAS Creative Software Team. I'm just Voluntary Moderator in this forum.

relaxvideo wrote on 12/10/2023, 2:08 PM

Yes, both looks strange because of very wide fov, but for normal fov 5.7k res is just not enough.

With very wide fov i can use in 4k projects too if the lighting was good!
 

#1 Ryzen 5-1600, 16GB DDR4, Nvidia 1660 Super, M2-SSD, Acer freesync monitor

#2 i7-2600, 32GB, Nvidia 1660Ti, SSD for system, M2-SSD for work, 2x4TB hdd, LG 3D monitor +3DTV +3D projectors

Win10 x64, Vegas22 latest

3d87c4 wrote on 12/10/2023, 5:58 PM

From left field: Have you tried using the FFMPEG V360 filter for any of this?

FFMPEG has been a handy tool over the years in my 3D and 3D360 workflow. Lately, I've been using it for VR180 format conversions---SBS fisheye (already 3D aligned in Vegas) to SBS equilateral, 3D360 to VR180, etc.

Last changed by 3d87c4 on 12/10/2023, 5:59 PM, changed a total of 1 times.

Del XPS 17 laptop

Processor    13th Gen Intel(R) Core(TM) i9-13900H   2.60 GHz
Installed RAM    32.0 GB (31.7 GB usable)
System type    64-bit operating system, x64-based processor
Pen and touch    Touch support with 10 touch points

Edition    Windows 11 Pro
Version    22H2
Installed on    ‎6/‎8/‎2023
OS build    22621.1848
Experience    Windows Feature Experience Pack 1000.22642.1000.0

NVIDIA GeForce RTX 4070 Laptop GPU
Driver Version: 31.0.15.2857
8GB memory
 

relaxvideo wrote on 12/11/2023, 2:19 AM

What you mean "left field"? For linear projection? I dont tried ffmpeg with vegas.
Any tutorial?

#1 Ryzen 5-1600, 16GB DDR4, Nvidia 1660 Super, M2-SSD, Acer freesync monitor

#2 i7-2600, 32GB, Nvidia 1660Ti, SSD for system, M2-SSD for work, 2x4TB hdd, LG 3D monitor +3DTV +3D projectors

Win10 x64, Vegas22 latest

3d87c4 wrote on 12/11/2023, 12:13 PM

@relaxvideo: Sorry...I meant an idea that isn't available within Vegas itself and might be considered a distraction from the main thread of discussion.

I don't know of a tutorial for that specifically, but a couple of options come to mind:

First, you could reorient the 360 so the subject of interest is centered, create the cubemap face centered on that location, and crop it to your desired frame size.

Or, you could reorient the image and convert it to a rectilinear (normal) projection, again cropping or specifying the target field of view for the final image.

I haven't tried either of these myself, though I have tinkered a little with cubemaps.

Here is a link to the ffmpeg filters documentation. Search for the v360 filter.

https://ffmpeg.org/ffmpeg-filters.html

If you haven't used it before the learning curve is daunting...I've learned by doing, referencing a book (in storage now) and lots of google searches for specific tasks.

There are a couple of GUI's available: I used myffmpeg for a while many years ago, but it was a bit limited.

 

Last changed by 3d87c4 on 12/11/2023, 12:16 PM, changed a total of 1 times.

Del XPS 17 laptop

Processor    13th Gen Intel(R) Core(TM) i9-13900H   2.60 GHz
Installed RAM    32.0 GB (31.7 GB usable)
System type    64-bit operating system, x64-based processor
Pen and touch    Touch support with 10 touch points

Edition    Windows 11 Pro
Version    22H2
Installed on    ‎6/‎8/‎2023
OS build    22621.1848
Experience    Windows Feature Experience Pack 1000.22642.1000.0

NVIDIA GeForce RTX 4070 Laptop GPU
Driver Version: 31.0.15.2857
8GB memory
 

3d87c4 wrote on 12/11/2023, 7:23 PM

I tinkered with the FFMPEG v360 filter back in July 2021 to get an understanding of it's cubemap functionality. Looking back I'm thinking the results may be applicable to 360 video reframing, though more cropping is likely needed either in the ffmpeg command or in Vegas.

The videos I tinkered with were created while I was playing with a new to me Rokinon/Samyang 8mm fisheye lens on a Nikon D7000 SLR, mounted on a Neewer panorama head on a tripod. I shot a still panorama using 6 shots around with 2 up and 2 down, plus several offset nadir shots to hide the tripod. I also shot 6 short videos around. I stitched the videos using Autopano Video Pro & Giga. I forget my exact workflow, but believe I used the still images in Giga to correct any issues in the apex and nadir of the video either as part of the Autopano workflow or using masking in Vegas.

The video itself was silly...me walking around the tripod with the camera in each position, counting so I spent about the same time walking through the 60 degree field of view (after stitching). I'd hoped it would look like I was walking around a stationary 360 camera, but that didn't work, eh? I made a remix where I'm walking through each view at the 8 time! Here are the two original videos:



To create a 3x2 cube map I used the following FFMPEG input:

ffmpeg -i input -vf v360=e:c3x2:cubic:out_pad=0.01 cubemap.mp4

Note: I didn't explicitly specify the codec so it defaults to libx264 and yields a 3840x1920 video (or 3840x3840 for the individual cubes below).

and the video

The image order in the upper row is Right, Left, Up.
The image order in the bottom row is Down, Foreward, Back.

Here are the FFMPEG inputs, and resulting videos for each of the individual cube faces:

ffmpeg -i input -vf v360=e:sg:d_fov=90:yaw=90:pitch=90 right.mp4 https://youtube.com/shorts/KJvLHCM-wcc

ffmpeg -i input -vf v360=e:sg:d_fov=90:yaw=-90:pitch=90 left.mp4 https://youtube.com/shorts/cNerKgIrwJs

ffmpeg -i input -vf v360=e:sg:d_fov=90:yaw=0:pitch=180 up.mp4 https://youtube.com/shorts/c3cn95uG398

ffmpeg -i input -vf v360=e:sg:d_fov=90:yaw=0 down.mp4 https://youtube.com/shorts/5rj0bJAebXs

ffmpeg -i input -vf v360=e:sg:d_fov=90:yaw=0:pitch=90 foreward.mp4 https://youtube.com/shorts/YcZBZFCczqQ

ffmpeg -i input -vf v360=e:sg:d_fov=90:yaw=180:pitch=90 back.mp4 https://youtube.com/shorts/hX3U48AkL1w

A couple of comments on the input: V360=e:sq:d_fov=90 tells it to convert an equirectangular projection to a Stereographic projection cropped to a 90 degree field of view. Yaw is horizontal rotation, pitch is rotation up and down. There is also a roll parameter but it wasn't used here.

If it isn't cropped the stereographic projection yields the popular "little planet" view. Here the first video formatted as an uncropped little planet:



Finally, an alternative to using a cube face for reformatting a 360 is to use the flat, i.e. regular video, option:

ffmpeg -i input -vf v360=e:flat:yaw=180 flat.mp4
 

 

Del XPS 17 laptop

Processor    13th Gen Intel(R) Core(TM) i9-13900H   2.60 GHz
Installed RAM    32.0 GB (31.7 GB usable)
System type    64-bit operating system, x64-based processor
Pen and touch    Touch support with 10 touch points

Edition    Windows 11 Pro
Version    22H2
Installed on    ‎6/‎8/‎2023
OS build    22621.1848
Experience    Windows Feature Experience Pack 1000.22642.1000.0

NVIDIA GeForce RTX 4070 Laptop GPU
Driver Version: 31.0.15.2857
8GB memory
 

relaxvideo wrote on 12/12/2023, 7:12 AM

cool, thanks for the detailed infos!

relaxvideo wrote on 1/23/2024, 1:59 PM

If i set project properties to 4k, i see much more detailed video on my fullhd preview monitor (best/full) than when i set project properties to full hd. Is it normal? (source is Insta X3 5.7k equirectangular video)

What if i like to render the video in fullhd with voukoder? I cannot use 4k properties in this case.

#1 Ryzen 5-1600, 16GB DDR4, Nvidia 1660 Super, M2-SSD, Acer freesync monitor

#2 i7-2600, 32GB, Nvidia 1660Ti, SSD for system, M2-SSD for work, 2x4TB hdd, LG 3D monitor +3DTV +3D projectors

Win10 x64, Vegas22 latest