NVMe drive is loading video files extremely slow in Vegas...

Bryan-Vahey wrote on 12/23/2019, 11:23 PM

Hello!

I just purchased a new Western Digital WD BLACK SN750 NVMe M.2 (WDS100T3XHC). I bought this specifically because I am using massive recordings of 60-100 gig 1080p mp4 video files with 3 audio tracks and I want to use the NVMe to quickly load them into Vegas.

The only problem is, the read rates of the NVMe are extremely slow; ~17 MB/s slow (it is rated for 3500) in Vegas when I drag-and-drop a video file. The video files are stored on the NVMe and Vegas is installed on the NVMe, too.

 

I ran a benchmark and found the drive to be operating perfectly normal. It has a heatsink and the max temperature it reached was 46C during the benchmark (max recommended is 80; so it's running cold).


Am I misunderstanding how the NVMe should be performing in this specific task? Is there an issue with Magix Vegas?

I am running Vegas Pro 16 build 424.


Thanks!

 

 

 

 

Comments

fred-w wrote on 12/23/2019, 11:30 PM

It may be that the real time through put demands are not that much. Usually the hard drive is not the bottle neck for Vegas (though I'm not working with 2k, 4k, 8k files, but I do do layers in compositing. IOW, the card is only going to load out what the real time request for data is at any given time.

As far as Loading into Vegas, well, when you load the program, you're only loading the file location information and thumbnails, not the files themselves, though of that you may be quite well aware.

Phil-MJr. wrote on 12/23/2019, 11:38 PM

If you are reading from a slow drive, the read rates will not exceed that source drive's speed capability to read. If the source files are already on the NVMe drive, should not be slow.

Bryan-Vahey wrote on 12/24/2019, 12:54 AM

If you are reading from a slow drive, the read rates will not exceed that source drive's speed capability to read. If the source files are already on the NVMe drive, should not be slow.

The source files are already on the NVMe.

fifonik wrote on 12/24/2019, 2:06 AM

Are you sure that the drive speed is bottleneck? When these files were of normal SSD/HDD were they opened fast in VP?

May be the footage you using cannot be decoded fast on your system for some reason? You should play with GPU acceleration decoder options or the "so4 thing". You may think about providing some details about your footage or providing link to small sample.

Camcorder: Panasonic X1500 + Panasonic X920 + GoPro Hero 11 Black

Desktop: MB: MSI B450M MORTAR TITANIUM, CPU: AMD Ryzen 5700X, RAM: G'Skill 16 GB DDR4@3200, Graphics card: MSI RX6600 8GB, SSD: Samsung 970 Evo+ 1TB (NVMe, OS), Samsung 870 Evo, HDD WD 4TB, HDD Toshiba 4TB, OS: Windows 10 Pro 22H2

NLE: Vegas Pro [Edit] 11, 12, 13, 15, 17, 18, 19

Bryan-Vahey wrote on 12/24/2019, 3:21 AM

Are you sure that the drive speed is bottleneck? When these files were of normal SSD/HDD were they opened fast in VP?

May be the footage you using cannot be decoded fast on your system for some reason? You should play with GPU acceleration decoder options or the "so4 thing". You may think about providing some details about your footage or providing link to small sample.

Good idea on supplying a sample. Will do soon. I am dealing with mp4 files that have three audio layers. The drive speed is certainly not bottlenecked as it is barely using any of its potential speed. Hardly any of my pc components are really being utilized when the peaks are being formed etc. it was a very slow process on the HDD/SSD that’s what prompted the NVMe purchase to begin with. I need to do a comparison and time how long it takes from the HDD/SSD versus the nvme. I’m thinking there is something about the file that VEGAS does not like/cannot handle properly making this a software issue. I have read, for example, that wav files are almost instantly loaded versus other formats.

bitman wrote on 12/24/2019, 10:20 AM

I am pretty sure the drive speed is not the bottleneck. I have done tests on my system with reading from HD, NVMe ssd to and from in all combinations, and even created a RAM disk for a while to experiment drive speed. The conclusion in my case with my typical media files (usually from RX10 mark 3 or gopro), it did not make much of difference.

I suppose if my theory is correct, it probably makes more of difference (and see your transfer speed go up) if you use filetypes (for example lossless intermediates) which are not codec compressed or only lightly compressed, those files are typically extremely large in size, but do not require much decoding overhead (CPU and/or GPU workload), but due to their large size benefit more from fast disks and interfaces.

Your disk transfer speed readings are not only capped by it's own HW limits, but the readings will further drop speed if your CPU has not finished decoding the file he is working on...

Last changed by bitman on 12/24/2019, 10:28 AM, changed a total of 4 times.

APPS: VIDEO: VP 365 (21 - build 187), VP 365 20, VP 19 post (latest build -651), (uninstalled VP 12,13,14,15,16 Suite,17, VP18 post), Vegasaur, a lot of NEWBLUE plugins, Mercalli 6.0, Respeedr, Vasco Da Gamma 16 HDpro XXL, Boris Continuum 2024, Davinci Resolve Studio 18, SOUND: RX 10 advanced Audio Editor, Sound Forge Pro 17, Spectral Layers Pro 10, Audacity, FOTO: Zoner, DXO, Luminar, Topaz...

  • OS: Windows 11 Pro 64, version 23H2 (November 2023)
  • CPU: i9-13900K (upgraded my former CPU i9-12900K), Air Cooler: Noctua NH-D15s
  • RAM: DDR5 Corsair 64GB (5600-40 Vengeance)
  • Graphics card: ASUS GeForce RTX 3090 TUF OC GAMING (24GB) 
  • Monitor: LG 38 inch ultra-wide (21x9) - Resolution: 3840x1600
  • C-drive: Corsair MP600 PRO XT NVMe SSD 4TB (PCIe Gen. 4)
  • Video drives: Samsung NVMe SSD 2TB (980 pro and 970 EVO plus) each 2TB
  • Mass Data storage & Backup: WD gold 6TB + WD Yellow 4TB
  • MOBO: Gigabyte Z690 AORUS MASTER
  • PSU: Corsair HX1500i, Case: Fractal Design Define 7 (PCGH edition)
  • Misc.: Logitech G915, Evoluent Vertical Mouse, shuttlePROv2

 

 

fr0sty wrote on 12/24/2019, 10:49 AM

When dragging and dropping, does Vegas do anything in the background like building peaks or building proxies after you drop the new media on the timeline? By default, it will automatically start building peaks.

Last changed by fr0sty on 12/24/2019, 10:50 AM, changed a total of 3 times.

Systems:

Desktop

AMD Ryzen 7 1800x 8 core 16 thread at stock speed

64GB 3000mhz DDR4

Geforce RTX 3090

Windows 10

Laptop:

ASUS Zenbook Pro Duo 32GB (9980HK CPU, RTX 2060 GPU, dual 4K touch screens, main one OLED HDR)

Bryan-Vahey wrote on 12/24/2019, 11:26 AM

When dragging and dropping, does Vegas do anything in the background like building peaks or building proxies after you drop the new media on the timeline? By default, it will automatically start building peaks.

This is what happens with just a 16 gig mp4 file with 3 audio tracks. Notice the read speed on the NVMe drive. Uber slow. I'm thinking this is a software issue and Vegas does not like to load compressed audio.

fr0sty wrote on 12/24/2019, 11:43 AM

Look in the bottom left corner, where it says "building peaks". It is drawing the waveform peaks for the audio waveforms, which it then stores in a separate file. This is why the read speed is so slow, it has already loaded the video, but is doing a separate process on the audio at that point in time. You can actually start editing before that process has finished, but if you need to use your waveforms to sync or anything like that, you'll want to wait for the process to finish. Once the waveforms have been drawn and the .sfk files that contain them have been saved into your project media's folder, you will not have to do that process again. It's only on the first import.

This process won't be sped up by buying faster hard drives, unfortunately.

Last changed by fr0sty on 12/24/2019, 11:45 AM, changed a total of 1 times.

Systems:

Desktop

AMD Ryzen 7 1800x 8 core 16 thread at stock speed

64GB 3000mhz DDR4

Geforce RTX 3090

Windows 10

Laptop:

ASUS Zenbook Pro Duo 32GB (9980HK CPU, RTX 2060 GPU, dual 4K touch screens, main one OLED HDR)

Bryan-Vahey wrote on 12/24/2019, 11:49 AM

Look in the bottom left corner, where it says "building peaks". It is drawing the waveform peaks for the audio waveforms, which it then stores in a separate file. This is why the read speed is so slow, it has already loaded the video, but is doing a separate process on the audio at that point in time. You can actually start editing before that process has finished, but if you need to use your waveforms to sync or anything like that, you'll want to wait for the process to finish. Once the waveforms have been drawn and the .sfk files that contain them have been saved into your project media's folder, you will not have to do that process again. It's only on the first import.

This process won't be sped up by buying faster hard drives, unfortunately.

Yeah. I know once it does it the first time you don't need to do it again. Just hoped it would have hastened the initial load time. Oh well! Thanks for the help.

Chief24 wrote on 12/24/2019, 11:52 AM

+1 Frosty!

OP needs to look "DOWN" to see that all his Audio Peaks are being created. There is no problem with his drives, processor, GPU...it is the AUDIO being DE-COMPRESSED so you can see all those "wonderful & glorious" waveform peaks that help us to ensure our Audio/Video is synchronized.

I have two ThreadRipper systems, one with an AMD 1950x/Radeon VII, the other AMD 2920X/RTX 2070 FE. When I "slap" any of my GoPro's or AX-53 straight to the timeline, just like the OP's video, I Have To Wait! Been like that since using Movie Studio Platinum 12 Suite, 13, 14, 15, and now 16 Suite; did it with Vegas Pro 12 Suite, 13, 14, and 15 Suite (no Vegas Pro installed since rebuilding 1950x system due to PSU & Motherboard issue from leaking AIO). Did that when I was on Intel platforms, with 3930K, 5820, 5930K, 6800K. So it is not inherently limited to only certain "Brands".

@Bryan-Vahey

Just be patient! Nothing wrong with your drives/CPU/GPU. It will be "faster" opening project and media on second and subsequent loads. Remember though, if you add another file of the type you showed the first time to the Media Bin, it will do the same thing.

Self Build: #1 MSI TRX40 Pro Wi-Fi w/3960X (be Quiet! Dark Rock Pro TR4) @ stock; 128GB Team Group 3200 MHz; OS/Apps - WDSN850X PCI-e 4.0x4 4TB, Documents/Extras - WDSN850X PCI-e 4.0x4 4TB; XFX AMD Radeon 7900XTX (24.1.1); Samsung 32 Inch UHD 3840x2160; Windows 11 Pro 64-Bit (23H2 22631.3155); (2) Inland Performance 2TB/(2) PNY 3040 4TB PCI-e on Asus Quad M.2x16; (2) WD RED 4TB; ProGrade USB CFExpress/SD card Reader; LG 16X Blu-Ray Burner; 32 inch Samsung UHD 3840x2160.

VEGAS Pro 20 Edit (411) & HOS (Happy Otter Scripts); DVD Architect 7.0 (100);

Sound Forge Audio Studio 15; ACID Music Studio 11; SonicFire Pro 6.6.9 (with Vegas Pro/Movie Studio Plug-in); DaVinci Resolve (Free) 18.6.5

#2: Gigabyte TRX50 Aero D w/7960x (Noctua NH-U14S TR5-SP6) @ stock; 128GB Kingston Fury Beast RDIMM @4800 MHz; OS/Apps - Seagate Firecuda 540 2TB PCI-e 5.0x4; Documents/Extras/Source/Transcodes - 4TB WDSN850X PCI-e 4.0x4; 4TB Inland Performance PCI-e 3.0x4; 2TB Inland Performance PCI-e 4.0x4; BlackMagic PCI-e Decklink 4K Mini-Recorder; ProGrade USB SD & Micro SD card readers; LG 32 Inch UHD 3840.x2160: PowerColor Hellhound RX Radeon 7900XT (24.1.1); Windows 11 Pro 64-Bit (22631.3155)

VEGAS Pro 20 Edit (411) & HOS; DVD Architect 7.0 (100); Sound Forge Audo Studio 15; Acid Music Studio 11

Canon EOS R6 MkII, Canon EOS R6, Canon EOS R7 (All three set for 4K 24/30/60 Cinema Gamut/CLog3); GoPro Hero 5+ & 6 Black & (2) 7 Black & 9 Black & 10 Black & 11 Black & 12 Black (All set at highest settings - 4K, 5K, & 5.3K mostly at 29.970); Sony FDR AX-53 HandyCam (4K 100Mbps XAVC-S 23.976/29.970)

Chief24 wrote on 12/24/2019, 11:53 AM

Sorry, you both posted while I was still typing. 🙂

Self Build: #1 MSI TRX40 Pro Wi-Fi w/3960X (be Quiet! Dark Rock Pro TR4) @ stock; 128GB Team Group 3200 MHz; OS/Apps - WDSN850X PCI-e 4.0x4 4TB, Documents/Extras - WDSN850X PCI-e 4.0x4 4TB; XFX AMD Radeon 7900XTX (24.1.1); Samsung 32 Inch UHD 3840x2160; Windows 11 Pro 64-Bit (23H2 22631.3155); (2) Inland Performance 2TB/(2) PNY 3040 4TB PCI-e on Asus Quad M.2x16; (2) WD RED 4TB; ProGrade USB CFExpress/SD card Reader; LG 16X Blu-Ray Burner; 32 inch Samsung UHD 3840x2160.

VEGAS Pro 20 Edit (411) & HOS (Happy Otter Scripts); DVD Architect 7.0 (100);

Sound Forge Audio Studio 15; ACID Music Studio 11; SonicFire Pro 6.6.9 (with Vegas Pro/Movie Studio Plug-in); DaVinci Resolve (Free) 18.6.5

#2: Gigabyte TRX50 Aero D w/7960x (Noctua NH-U14S TR5-SP6) @ stock; 128GB Kingston Fury Beast RDIMM @4800 MHz; OS/Apps - Seagate Firecuda 540 2TB PCI-e 5.0x4; Documents/Extras/Source/Transcodes - 4TB WDSN850X PCI-e 4.0x4; 4TB Inland Performance PCI-e 3.0x4; 2TB Inland Performance PCI-e 4.0x4; BlackMagic PCI-e Decklink 4K Mini-Recorder; ProGrade USB SD & Micro SD card readers; LG 32 Inch UHD 3840.x2160: PowerColor Hellhound RX Radeon 7900XT (24.1.1); Windows 11 Pro 64-Bit (22631.3155)

VEGAS Pro 20 Edit (411) & HOS; DVD Architect 7.0 (100); Sound Forge Audo Studio 15; Acid Music Studio 11

Canon EOS R6 MkII, Canon EOS R6, Canon EOS R7 (All three set for 4K 24/30/60 Cinema Gamut/CLog3); GoPro Hero 5+ & 6 Black & (2) 7 Black & 9 Black & 10 Black & 11 Black & 12 Black (All set at highest settings - 4K, 5K, & 5.3K mostly at 29.970); Sony FDR AX-53 HandyCam (4K 100Mbps XAVC-S 23.976/29.970)

Bryan-Vahey wrote on 12/24/2019, 11:55 AM

+1 Frosty!

OP needs to look "DOWN" to see that all his Audio Peaks are being created.

I know that was what was happening... I just didn't know the process could not be sped up with a faster [NVMe] drive.

fr0sty wrote on 12/24/2019, 12:06 PM

I haven't noticed significant improvements with CPU upgrades either, TBH... the audio mostly has remained the same over the years, even though the video codecs have become much more complex, but the amount of time I spend building peaks seems to stay the same for as long as I can remember.

Maybe they could implement a system that gives us a rough waveform that we can only zoom in on so much until it is done building peaks, and as it progresses, it lets us slowly zoom in more and more. I know DAW's do not make you wait to build waveforms, so there has to be some way to either speed this up or eliminate the need for it.

Systems:

Desktop

AMD Ryzen 7 1800x 8 core 16 thread at stock speed

64GB 3000mhz DDR4

Geforce RTX 3090

Windows 10

Laptop:

ASUS Zenbook Pro Duo 32GB (9980HK CPU, RTX 2060 GPU, dual 4K touch screens, main one OLED HDR)

VanSoVan wrote on 3/8/2020, 1:31 PM

I just built a new Ryzen 9 3900x, Geforce 2070 plus, NVME system. It is wicked fast at rendering, but the 'building waveforms' process doesn't seem any faster than with my older system. It works, but it doesn't seem that Magix has coded this module to take advantage of newer systems ... sound like a feature request.

TheRhino wrote on 3/8/2020, 3:39 PM

I have two ThreadRipper systems, one with an AMD 1950x/Radeon VII, the other AMD 2920X/RTX 2070 FE. When I "slap" any of my GoPro's or AX-53 straight to the timeline, just like the OP's video, I Have To Wait!

@Bryan-Vahey, what CPU do you have in your system? Some processes in Vegas, like writing audio peaks, are very linear & typically only push one core to max. Therefore my 8-core 9900K overclocked to 5.0 ghz outperforms 16-core AMD CPUs only running at 4.0 ghz on many Vegas tasks, especially writing audio peaks, but also when apply certain FX & codecs that are not efficient at using multi-cores...

If I pull-up Task Manager during audio peaks ONE core is pushed to 100% while the others do practically nothing... Part of this is due to the fact that Vegas was designed over 20 years ago when single core CPUs were the norm & part is due to the fact that it is VERY difficult to write code that truly takes advantage of 8+ cores without losing efficiency as work is divided between the cores...

For SOURCE video, my 9900K system has (4) 2TB M.2 drives setup as (2) 4TB RAID0 drives both with crazy fast read speeds... However, none of this matters, just that lonely core running at 5.0 ghz...

Workstation C with $600 USD of upgrades in April, 2021
--$360 11700K @ 5.0ghz
--$200 ASRock W480 Creator (onboard 10G net, TB3, etc.)
Borrowed from my 9900K until prices drop:
--32GB of G.Skill DDR4 3200 ($100 on Black Friday...)
Reused from same Tower Case that housed the Xeon:
--Used VEGA 56 GPU ($200 on eBay before mining craze...)
--Noctua Cooler, 750W PSU, OS SSD, LSI RAID Controller, SATAs, etc.

Performs VERY close to my overclocked 9900K (below), but at stock settings with no tweaking...

Workstation D with $1,350 USD of upgrades in April, 2019
--$500 9900K @ 5.0ghz
--$140 Corsair H150i liquid cooling with 360mm radiator (3 fans)
--$200 open box Asus Z390 WS (PLX chip manages 4/5 PCIe slots)
--$160 32GB of G.Skill DDR4 3000 (added another 32GB later...)
--$350 refurbished, but like-new Radeon Vega 64 LQ (liquid cooled)

Renders Vegas11 "Red Car Test" (AMD VCE) in 13s when clocked at 4.9 ghz
(note: BOTH onboard Intel & Vega64 show utilization during QSV & VCE renders...)

Source Video1 = 4TB RAID0--(2) 2TB M.2 on motherboard in RAID0
Source Video2 = 4TB RAID0--(2) 2TB M.2 (1) via U.2 adapter & (1) on separate PCIe card
Target Video1 = 32TB RAID0--(4) 8TB SATA hot-swap drives on PCIe RAID card with backups elsewhere

10G Network using used $30 Mellanox2 Adapters & Qnap QSW-M408-2C 10G Switch
Copy of Work Files, Source & Output Video, OS Images on QNAP 653b NAS with (6) 14TB WD RED
Blackmagic Decklink PCie card for capturing from tape, etc.
(2) internal BR Burners connected via USB 3.0 to SATA adapters
Old Cooler Master CM Stacker ATX case with (13) 5.25" front drive-bays holds & cools everything.

Workstations A & B are the 2 remaining 6-core 4.0ghz Xeon 5660 or I7 980x on Asus P6T6 motherboards.

$999 Walmart Evoo 17 Laptop with I7-9750H 6-core CPU, RTX 2060, (2) M.2 bays & (1) SSD bay...