SSD Storage setup for editing?

Cliff Etzel wrote on 5/28/2018, 12:10 PM

In addition to making the transition to Vegas 15 thanks to the latest update, I'm reassessing my storage setup and wondering what others use and what I'm thinking for the upgrade myself.

My current configuration for storage devices is as follows:

  • Crucial 240GB Boot/Programs drive
  • WD 640GB 7200RPM Black Drive for all Project files and Renders
  • 2x1TB WD 7200RPM Black laptop drives Mini Raid 0 in an Akitio enclosure attached via eSata card
  • Seagate 160GB 7200RPM for all media cache and temp files

My thinking is to move as much as possible to SSD's but not sure still - the read/write for my single SSD is off the chart compared to my spinning HD's according to AJA's system test app. My SSD read/write is basically 500MB/sec, where as my mini RAID0 clocks in at about 1/4 that! My concern is the read/write lifespan of SSD's.

My thinking for the storage medium upgrade is as follows:

  • Crucial 240GB for boot drive (Already using)
  • Crucial 500GB SSD for Project files/final Renders
  • Crucial 2x1TB SSD's in eSata/USB 3.1 mini Raid enclosure for all assets
  • Crucial 120GB SSD for media cache/temp files

I'm thinking to repurpose the spinning drives for archiving completed projects in addition to using a couple of WD 4TB Red drives for weekly offsite rotating backup.

I'm still editing 1080p footage and do plan to move to 4K later this year hence the idea of moving to SSD's.

What are others using and what are your thoughts on this configuration?

Comments

fr0sty wrote on 5/28/2018, 12:33 PM

I'd keep an eye on the NVMe M.2 PCI Express drives. WAAAAAY faster than SATA, and if you can grab a 1TB drive (keep larger magnetic drives for storage, and just use it to hold any projects currently being edited), you will notice a significant boost in performance when editing multicamera 4K video. I'm using one of those Samsung Evo drives, 3000MBps (MB, not mb) read speeds really help. Plus, for my system drive, windows installed in 10 minutes and cold boots in less than 30 seconds.

Systems:

Desktop

AMD Ryzen 7 1800x 8 core 16 thread at stock speed

64GB 3000mhz DDR4

Geforce RTX 3090

Windows 10

Laptop:

ASUS Zenbook Pro Duo 32GB (9980HK CPU, RTX 2060 GPU, dual 4K touch screens, main one OLED HDR)

fr0sty wrote on 5/28/2018, 12:37 PM

https://www.newegg.com/Product/Product.aspx?Item=N82E16820147595

Max Sequential Read

Up to 3200 MBps (a SATA 3 drive will max out at 600 MB/s)

Max Sequential Write

Up to 1900 MBps

Just make sure your motherboard supports the drives or that you have a free PCI-E x4 port to buy an adapter card.

Before long, SATA will no longer be fast enough.. best to stay ahead of the curve.

Last changed by fr0sty on 5/28/2018, 12:39 PM, changed a total of 1 times.

Systems:

Desktop

AMD Ryzen 7 1800x 8 core 16 thread at stock speed

64GB 3000mhz DDR4

Geforce RTX 3090

Windows 10

Laptop:

ASUS Zenbook Pro Duo 32GB (9980HK CPU, RTX 2060 GPU, dual 4K touch screens, main one OLED HDR)

Cliff Etzel wrote on 5/28/2018, 12:39 PM

I'd keep an eye on the NVMe M.2 PCI Express drives. WAAAAAY faster than SATA, and if you can grab a 1TB drive (keep larger magnetic drives for storage, and just use it to hold any projects currently being edited), you will notice a significant boost in performance when editing multicamera 4K video. I'm using one of those Samsung Evo drives, 3000MBps (MB, not mb) read speeds really help. Plus, for my system drive, windows installed in 10 minutes and cold boots in less than 30 seconds.

I've been watching NVMe drives as well but my motherboard (ASUS Z370-H Gaming) loses two internal sata ports when one is installed (I can install two NVMe SSD's but that means I'm left with 4 of the 8 sata ports available and two of those ports are for internal raid setup.

Thoughts?

Cliff Etzel wrote on 5/28/2018, 12:41 PM

https://www.newegg.com/Product/Product.aspx?Item=N82E16820147595

Max Sequential Read

Up to 3200 MBps

Max Sequential Write

Up to 1900 MBps

Just make sure your motherboard supports the drives or that you have a free PCI-E x4 port to buy an adapter card.

Before long, SATA will no longer be fast enough.. best to stay ahead of the curve.

Ahhh - I hadn't thought about using a PCI-e card to mount a pair of NVMe SSD's - something more to research now... Any recommendations on a reliable adapter card?

fr0sty wrote on 5/28/2018, 12:42 PM

I'd get one 1TB NVME M.2 and use the rest SATA. Let the one drive be the one you are actively editing on. It should have more than enough bandwidth to stream all angles at once.

That leaves you 6 of them. Buyer beware!!! There are SATA NVME M.2 drives that DO NOT hit the max 3200MBPS throughput these newer drives are capable of, and instead are limited to SATA's 6gbps interface. Those cards look identical to the kind I posted above, and they might be why you are giving up SATA ports to use them... the real NVME cards you want to use instead utilize PCI-Express to connect to the computer, so it'll take PCIE lanes, but not SATA.

The card linked to above is the real type of NVME. When shopping, pay attention to their max transfer rates. If it is 6gbps, it's a SATA drive, if it's 3200MBps *capital MB, megabytes, not bits), it's what you want. If your motherboard is only compatible with the SATA type of NVME cards, they sell PCIEx4 adapter cards.

Last changed by fr0sty on 5/28/2018, 12:47 PM, changed a total of 2 times.

Systems:

Desktop

AMD Ryzen 7 1800x 8 core 16 thread at stock speed

64GB 3000mhz DDR4

Geforce RTX 3090

Windows 10

Laptop:

ASUS Zenbook Pro Duo 32GB (9980HK CPU, RTX 2060 GPU, dual 4K touch screens, main one OLED HDR)

fr0sty wrote on 5/28/2018, 12:46 PM

I edited that last post not realizing you may not get the notification I added to it and may not see it, be sure to read what I added, it's important to know when buying.

Systems:

Desktop

AMD Ryzen 7 1800x 8 core 16 thread at stock speed

64GB 3000mhz DDR4

Geforce RTX 3090

Windows 10

Laptop:

ASUS Zenbook Pro Duo 32GB (9980HK CPU, RTX 2060 GPU, dual 4K touch screens, main one OLED HDR)

BruceUSA wrote on 5/28/2018, 1:04 PM

Here is my opinion. If you are going to build a high end computer, by all means go all out, go with NVMe.M2 .SSD for OS, Media Import, Rendering. I did exactly that with my 1950X, On a non high end system you are limited by PCIE lanes. With that, you can just go with one SSD or NVMe .m2 SSD for OS and 1 7200rpm for media storage and 1 for rendering. You 'll be just fine with that. On my 4930K I run 480GB Corair SSD and 2 spindle 7200rpm for media storage and rendering. Its serve me fine.

Intel i9 Core Ultra 285K Overclocked all P Cores @5.6, all E-Cores @5ghz               

MSI MEG Z890 ACE Gaming Wifi 7 10G Super Lan, thunderbolt 4                                

48GB DDR5 -8200mhz Overclocked @8800mhz                  

Crucial T705 nvme .M2 2TB Gen 5  OS. 4TB  gen 4 storage                    

RTX 5080 16GB  Overclocked 3.1ghz, Memory Bandwidth increased from 960 GB/s to 1152 GB/s                                                            

Custom built hard tube watercooling.                            

MSI PSU 1250W, Windows 11 Pro

 

Cliff Etzel wrote on 5/28/2018, 1:14 PM

Here is my opinion. If you are going to build a high end computer, by all means go all out, go with NVMe.M2 .SSD for OS, Media Import, Rendering. I did exactly that with my 1950X, On a non high end system you are limited by PCIE lanes. With that, you can just go with one SSD or NVMe .m2 SSD for OS and 1 7200rpm for media storage and 1 for rendering. You 'll be just fine with that. On my 4930K I run 480GB Corair SSD and 2 spindle 7200rpm for media storage and rendering. Its serve me fine.

So my thinking is - keep the current Crucial 240GB Sata SSD with OS, add an NVMe 1TB on a PCI-E 4x adapter card for all current project assets, a 500GB NVMe for projects and renders and a 120GB Sata SSD for media cache/temp files.

What I've read is it's STRONGLY advise storing Project files on their own storage medium in case the assets drive fails. I would use my remaining spinning drives for backup/archiving.

OldSmoke wrote on 5/28/2018, 6:24 PM

@Cliff Etzel you are running exactly into the problem I told you you would. Check how many PCIe lanes your system has, if I am not mistaken you have only 16?

Last changed by OldSmoke on 5/28/2018, 6:55 PM, changed a total of 1 times.

Proud owner of Sony Vegas Pro 7, 8, 9, 10, 11, 12 & 13 and now Magix VP15&16.

System Spec.:
Motherboard: ASUS X299 Prime-A

Ram: G.Skill 4x8GB DDR4 2666 XMP

CPU: i7-9800x @ 4.6GHz (custom water cooling system)
GPU: 1x AMD Vega Pro Frontier Edition (water cooled)
Hard drives: System Samsung 970Pro NVME, AV-Projects 1TB (4x Intel P7600 512GB VROC), 4x 2.5" Hotswap bays, 1x 3.5" Hotswap Bay, 1x LG BluRay Burner

PSU: Corsair 1200W
Monitor: 2x Dell Ultrasharp U2713HM (2560x1440)

john_dennis wrote on 5/28/2018, 6:46 PM

https://www.vegascreativesoftware.info/us/forum/new-desktop-build-thoughts--111272/#ca687521

OldSmoke wrote on 5/28/2018, 6:58 PM

https://www.vegascreativesoftware.info/us/forum/new-desktop-build-thoughts--111272/#ca687521

Exactly. That means adding anything aside from the GPU will cause the GPU to run x8.

Proud owner of Sony Vegas Pro 7, 8, 9, 10, 11, 12 & 13 and now Magix VP15&16.

System Spec.:
Motherboard: ASUS X299 Prime-A

Ram: G.Skill 4x8GB DDR4 2666 XMP

CPU: i7-9800x @ 4.6GHz (custom water cooling system)
GPU: 1x AMD Vega Pro Frontier Edition (water cooled)
Hard drives: System Samsung 970Pro NVME, AV-Projects 1TB (4x Intel P7600 512GB VROC), 4x 2.5" Hotswap bays, 1x 3.5" Hotswap Bay, 1x LG BluRay Burner

PSU: Corsair 1200W
Monitor: 2x Dell Ultrasharp U2713HM (2560x1440)

BruceUSA wrote on 5/28/2018, 7:24 PM

Keep his current sata SSD in OS and run 2 spindle hard drive, media storage and rendering. Run the GPU at 16X . He'll be just fine with this.

Intel i9 Core Ultra 285K Overclocked all P Cores @5.6, all E-Cores @5ghz               

MSI MEG Z890 ACE Gaming Wifi 7 10G Super Lan, thunderbolt 4                                

48GB DDR5 -8200mhz Overclocked @8800mhz                  

Crucial T705 nvme .M2 2TB Gen 5  OS. 4TB  gen 4 storage                    

RTX 5080 16GB  Overclocked 3.1ghz, Memory Bandwidth increased from 960 GB/s to 1152 GB/s                                                            

Custom built hard tube watercooling.                            

MSI PSU 1250W, Windows 11 Pro

 

Cliff Etzel wrote on 5/28/2018, 7:40 PM

I did a random test today... Copied files to my ssd that would not play real time without reducing quality on my current raid. Those UHD files on the timeline played back with no dropped frames. It seems it's not my gpu, but my media storage drive thoughput. That's the bottle neck and thought it was my gpu... Looking at getting an nvme 1tb SSD to replace my esata mini raid 0

fifonik wrote on 5/28/2018, 7:56 PM

Do not use that 160GB Seagate as temp drive. It is old and slow. You'd like to have fast drive for temp files. Keep these temp files on system drive (if you have room there, I prefer this way. KISS) or put them on separate SSD.

Always use SSD as system drive. Preferably NVMe.

Using SSD for renders does not have much sense (if you are not rendering uncompressed/lossless). Your source footage should be on fast drives, not target. On target it would be linear write with not very high speed.

Keeping project files separately from footage for me looks over complicated. I'm keeping projects with footage for simplicity. Set up backups to keep your project files in case of failure.

Last changed by fifonik on 5/28/2018, 7:57 PM, changed a total of 1 times.

Camcorder: Panasonic X1500 + Panasonic X920 + GoPro Hero 11 Black

Desktop: MB: MSI B450M MORTAR TITANIUM, CPU: AMD Ryzen 5700X, RAM: G'Skill 32 GB DDR4@3200, Graphics card: MSI RX6600 8GB, SSD: Samsung 970 Evo+ 1TB (NVMe, OS), HDD WD 4TB, HDD Toshiba 4TB, OS: Windows 10 Pro 22H2

NLE: Vegas Pro [Edit] 11, 12, 13, 15, 17, 18, 19, 22

Author of FFMetrics and FFBitrateViewer

Cliff Etzel wrote on 5/28/2018, 7:59 PM

@OldSmoke I couldn't afford anything more powerful for the motherboard - had to compromise someplace - figure I can use it for awhile and when I'm ready, replace it with something more powerful.

@fifonik - The 7200 RPM 160GB Seagate drive was what I had at the time and my current set up is what I had been using for a long time with Premiere Pro CS6 - times have changed so I'm re-evaluating my setup.

OldSmoke wrote on 5/28/2018, 8:14 PM

@Cliff Etzel I doubt the socket 2066 motherboard would have been so much more expensive and a 6-core 28PCIe lane CPU like the 7800x would have been sufficient and more future proof because you can upgrade the CPU to a 8 or 10 core later if you need more PCIe lanes.

Last changed by OldSmoke on 5/28/2018, 8:30 PM, changed a total of 1 times.

Proud owner of Sony Vegas Pro 7, 8, 9, 10, 11, 12 & 13 and now Magix VP15&16.

System Spec.:
Motherboard: ASUS X299 Prime-A

Ram: G.Skill 4x8GB DDR4 2666 XMP

CPU: i7-9800x @ 4.6GHz (custom water cooling system)
GPU: 1x AMD Vega Pro Frontier Edition (water cooled)
Hard drives: System Samsung 970Pro NVME, AV-Projects 1TB (4x Intel P7600 512GB VROC), 4x 2.5" Hotswap bays, 1x 3.5" Hotswap Bay, 1x LG BluRay Burner

PSU: Corsair 1200W
Monitor: 2x Dell Ultrasharp U2713HM (2560x1440)

wwaag wrote on 5/28/2018, 10:28 PM

Like Cliff, I also have an 8700K. Although there are only 16 direct lanes, there's another 24 lanes on the motherboard sharing a single DMI 3.0 (the equivalent of four PCIe lanes) connection. On my system, I have 2 NVMe's, one for the C drive and the other for my project drive. Here's the crystal disk marks for both drives.

First the c-drive.

Next, my editing drive.

As seen, the sequential read/write are virtually identical. And here's one of my archive drives.

Unless I'm missing something, a couple of NVMe drives work just fine on 8700K and are significantly faster than typical SSD's. My suggestion--go with the NVMe's.

AKA the HappyOtter at https://tools4vegas.com/. System 1: Intel i7-8700k with HD 630 graphics plus an Nvidia RTX4070 graphics card. System 2: Intel i7-3770k with HD 4000 graphics plus an AMD RX550 graphics card. System 3: Laptop. Dell Inspiron Plus 16. Intel i7-11800H, Intel Graphics. Current cameras include Panasonic FZ2500, GoPro Hero11 and Hero8 Black plus a myriad of smartPhone, pocket cameras, video cameras and film cameras going back to the original Nikon S.

OldSmoke wrote on 5/28/2018, 10:51 PM

@wwaag Sorry but such a test says nothing about the overall system performance when the NLE is pulling data from the disc, exchanges data with the GPU and CPU, has to write to another disc, if you are rendering and all the computation that is going on.

I am not saying that a 8700k system is a bad thing but you are limited to what’s onboard as adding a PCIe card along a GPU means you loose performance.

Last changed by OldSmoke on 5/28/2018, 11:25 PM, changed a total of 1 times.

Proud owner of Sony Vegas Pro 7, 8, 9, 10, 11, 12 & 13 and now Magix VP15&16.

System Spec.:
Motherboard: ASUS X299 Prime-A

Ram: G.Skill 4x8GB DDR4 2666 XMP

CPU: i7-9800x @ 4.6GHz (custom water cooling system)
GPU: 1x AMD Vega Pro Frontier Edition (water cooled)
Hard drives: System Samsung 970Pro NVME, AV-Projects 1TB (4x Intel P7600 512GB VROC), 4x 2.5" Hotswap bays, 1x 3.5" Hotswap Bay, 1x LG BluRay Burner

PSU: Corsair 1200W
Monitor: 2x Dell Ultrasharp U2713HM (2560x1440)

john_dennis wrote on 5/29/2018, 2:46 AM

The die was cast when Cliff bought the motherboard. Now, the maximum performance available from the platform comes by using NVMe drives in the two motherboard M.2 sockets. If the total throughput is limited by disk devices sharing lanes assigned only to the motherboard I/O and not lanes dedicated to the PCIe slots, so be it. Avoid PCIe riser cards, as that will rob lanes from the video card.

In this case, don't do what I do...

Edit: After some consideration "don't do what I do" is probably the best advice I've ever given on the forum.

NickHope wrote on 5/29/2018, 9:40 AM

This is what I have on my 3.5-year-old desktop. I back them up pretty regularly. Large projects open pretty quickly. I'm happy.

  • O/S, programs, paging file: Samsung 850 Pro 256Gb
  • Video & audio media: 2 x Western Digital Black 4TB drives in an Intel IRST "fake" RAID 0
  • Documents & other data inc. Vegas project files: Western Digital Black 2TB HDD


The last of those drives was also an IRST RAID 0 at first, but I used 2 oldish Samsung drives, and had months of random corrupt files and Google Drive sync weirdness before finally concluding that it was that drive that was playing up.

Cliff Etzel wrote on 5/29/2018, 10:02 AM

This is what I have on my 3.5-year-old desktop. I back them up pretty regularly. Large projects open pretty quickly. I'm happy.

  • O/S, programs, paging file: Samsung 850 Pro 256Gb
  • Video & audio media: 2 x Western Digital Black 4TB drives in an Intel IRST "fake" RAID 0
  • Documents & other data inc. Vegas project files: Western Digital Black 2TB HDD


The last of those drives was also an IRST RAID 0 at first, but I used 2 oldish Samsung drives, and had months of random corrupt files and Google Drive sync weirdness before finally concluding that it was that drive that was playing up.

@NickHope I like the simplicity of your configuration. I'm basically running the same setup with spinning drives. What's your playback performance off of your 4K footage while editing? I'm experiencing what I thought was a GPU bottle neck but it occurred to me yesterday to copy to my boot SSD a small 4K project I created that was not playing back well even at full Preview mode with some sample footage shot on a GH5 from Neumann Films. Opened it up with all the same effects applied and I saw a marked improvement in playback with no dropped frames - and this coming from mty lowly GTX-660ti. I also tried as you suggested disabling GPU timeline acceleration - that dropped my framerates by over 50% for timeline playback so Vegas apparently is utilizing my GPU for timeline playback.

It seems my frustration has been not with my current GPU, but my eSata Raid0 being constrained by data rate it can transfer. The suggestion of using a single 1TB NVMe SSD internally for current project assets seems to be the solution for what ails me and backing completed projects on a spinning archive HD. Never had a problem in Premiere Pro, but I could render the timeline and it would playback those timeline rendered mpeg files and never thought twice about the RAID being the bottle neck. I'm wondering if using the new USB 3.1 connectors with an external single SSD or 2x1TB SSD raid0 would be the middle ground for data throughput. As has been pointed out, the motherboard has limitations - but it was all I could afford. Not ideal, but the external SSD raid0 in a new USB3.1 (or Thunderbolt with an add on card) external enclosure seems to be the next best option for editing the kinds of projects I shoot.

OldSmoke wrote on 5/29/2018, 10:15 AM

@Cliff Etzel I can understand your budget constrains and as I also mentioned, it's not a bad setup you have. I personally moved away from eSata. I always had issues with it even on my servers as simple external backup drive connection. It should be ok to render to an eSata drive but don't use it for source media in a project; I don't even use USB3.1 drives for source media. Basically, I keep everything internal. Don't forget you have a pretty good amount of Sata 6GB ports too, use it. My other personal preference are NAS or SAS rather than eSata or USB RAID enclosures because I can move them away from my editing station into my server room but that may not be an issue for you.

Edit: I don't see an eSata port on you motherboard?

Last changed by OldSmoke on 5/29/2018, 10:16 AM, changed a total of 1 times.

Proud owner of Sony Vegas Pro 7, 8, 9, 10, 11, 12 & 13 and now Magix VP15&16.

System Spec.:
Motherboard: ASUS X299 Prime-A

Ram: G.Skill 4x8GB DDR4 2666 XMP

CPU: i7-9800x @ 4.6GHz (custom water cooling system)
GPU: 1x AMD Vega Pro Frontier Edition (water cooled)
Hard drives: System Samsung 970Pro NVME, AV-Projects 1TB (4x Intel P7600 512GB VROC), 4x 2.5" Hotswap bays, 1x 3.5" Hotswap Bay, 1x LG BluRay Burner

PSU: Corsair 1200W
Monitor: 2x Dell Ultrasharp U2713HM (2560x1440)

Cliff Etzel wrote on 5/29/2018, 10:27 AM

@Cliff Etzel I can understand your budget constrains and as I also mentioned, it's not a bad setup you have. I personally moved away from eSata. I always had issues with it even on my servers as simple external backup drive connection. It should be ok to render to an eSata drive but don't use it for source media in a project; I don't even use USB3.1 drives for source media. Basically, I keep everything internal. Don't forget you have a pretty good amount of Sata 6GB ports too, use it. My other personal preference are NAS or SAS rather than eSata or USB RAID enclosures because I can move them away from my editing station into my server room but that may not be an issue for you.

Edit: I don't see an eSata port on you motherboard?

I'm using a PCI-e card for eSata I brought over from my previous x58 setup - I got to thinking about internal raid set up as there are 2 ports designated for setting up a 2 drive RAID using a Silicon Image Sil 3132 SATALink Controller - I have also been researching SAS options as well as you stated, the ability to use eSata on the rare occasion I need to do basic editing with Vegas on my Dell M4500's esata port is another reason I opted for it - at the time. Seems alot of hardware upgrades are still in my future. The laptop is really for basic editing (cuts, dissolves, basic color correction as needed) and it has a display port so I can bring with me a little 15" HDTV color calibrated with my Spyder3 as a second monitor if editing on a hotel room or similar.Again, not "Ideal" but it gets the job done as long as I use my scopes in Vegas.

OldSmoke wrote on 5/29/2018, 10:40 AM

I'm using a PCI-e card for eSata I brought over from my previous x58 setup

That card will limit your graphic card to PCIe x8. I would rather use a USB3 to eSata adapter.

Proud owner of Sony Vegas Pro 7, 8, 9, 10, 11, 12 & 13 and now Magix VP15&16.

System Spec.:
Motherboard: ASUS X299 Prime-A

Ram: G.Skill 4x8GB DDR4 2666 XMP

CPU: i7-9800x @ 4.6GHz (custom water cooling system)
GPU: 1x AMD Vega Pro Frontier Edition (water cooled)
Hard drives: System Samsung 970Pro NVME, AV-Projects 1TB (4x Intel P7600 512GB VROC), 4x 2.5" Hotswap bays, 1x 3.5" Hotswap Bay, 1x LG BluRay Burner

PSU: Corsair 1200W
Monitor: 2x Dell Ultrasharp U2713HM (2560x1440)