After Shift+B, how do I Force VP to release RAM?

Comments

Grazie wrote on 2/12/2013, 7:37 AM
Render to a New track creates a finished HARD non-volatile file, safely deposited on a hard drive. The same as Rob: If it's not releasing memory after a render to disk then I'd think that's a bug

Therefore I don't understand why you are saying that Render to New Track remains in cache?

G

Satevis wrote on 2/12/2013, 8:20 AM
Well, Vegas needs to render those frames it writes to that file on the disk. So for each frame in your section, it runs its generate-the-frame-at-time-X process. This process includes things like reading frames from source clips, applying effects, compositing, transformations, you name it. Once all that is done, Vegas has a frame that represents the final composition of your timeline at time X. It then performs at least two additional steps. One is to encode that frame in your selected video format and store it in the file on the disk. The other step is to store the frame in the frame cache. If someone asks for the frame at time X again in the near future (which is generally very likely when editing video), it can skip the whole reading-source-files-applying-effects-compositing-transforming hassle and instead just pull the finished frame from the cache. If no one asks for it, well, fair enough, it had to render that frame anyway, so no harm done.

Of course, this behavour isn't specific to rendering to a new track. It's basically the same whenever Vegas needs to render a frame. If you play the timeline, for instance, Vegas constantly has to render new frames to feed the preview window, and because you might soon play that part of the timeline again, it places all frames it rendered in the cache. If you scrub through the timeline, Vegas will have to render frames here and there to update the preview window and these, too, will be placed in the cache. If you have it create a prerendered file, it will have to render the frames to store in that file, and again, these frames will also be placed in the cache. And if you hit Shift-B, it will start rendering your timeline frame by frame and place each of these frames in the cache.

So, while you use Vegas, the cache will continuously grow containing more and more frames from your timeline that don't need to be rendered but can just be grabbed from the cache. At one point, the cache will reach the limit you set in the preferences and will stop growing. From this point on, Vegas will store any new frames it renders by replacing old ones that haven't been used for some time, so the total cache size remains the same.

The whole thing is there to speed up your editing experience, as just grabbing a finished frame from the cache and throwing it at the screen is faster than having to go through the whole rendering process or even just having to read it from a prerendered file on the disk. So it's really not a bad thing. Still, if the memory consumption scares you, you can always just turn the whole thing off. Vegas will then render every frame from scratch every time it needs it but won't consume any more memory than what is needed to handle your project, compute effects, and generally just go about its daily business.

If your actual concern is that the frames rendered during render to new track might not be used again because you were planning to delete those original tracks and just use the rendered track instead - well, first of all, Vegas can't really guess which frames might be used again and which won't, so it doesn't even start to try telling the future but just caches all frames as they come along. Also, nothing is wasted here: With the exception of Shift-B, Vegas never renders frames just to fill the cache. Here, it had to render these frames to fulfill your request of rendernig to a new track. And since those frames were there anyway, it might as well store them. It's no memory leak either: These frames go into the same cache as all other cached frames and as long as you set a sensible limit in the preferences, that cache will stop growing before it buries the whole system under a pile of frames no one will ever want to see again. Finally, it doesn't prevent Vegas from caching other frames in the future. Should the cache run full, it will simply replace some frames. Again, no harm done.
Grazie wrote on 2/12/2013, 8:36 AM
OK, so a Render to New Track Event will NOT be substituting this cache for the Render to New Track non-volatile file. Is that it?

G

Satevis wrote on 2/12/2013, 10:01 AM
The original frames and those in the rendered track will indeed be cached separately, because while they're basically the same, they're not identical (the frames in the new track passed through the encoder of whatever format you chose). If you leave the new track alone for now and play the original clips, Vegas will reuse the frames it's still got from your recent render to new track operation. If, instead, you mute or delete the original events and play only the newly rendered track, Vegas will eventually replace those original frames with those from the rendered track (unless your cache is large enough to hold both the old and new frames). Either way, your cache will always hold as many of your most recently requested frames as it can, while never growing beyond the size you set in the preferences.
Grazie wrote on 2/12/2013, 10:09 AM
(I've just had a 101 on PageFile Caching from another Forum member . . penny is kinda dropping here . . )

So, what I am looking at here, in Physical Memory, is NOT necessarily sticks of RAM. It's sticks of RAM plus that all important PageFile Cache lurking on one of my drives. And what you're saying, Satevis, is that the PageFile is the Cache that deals with this temporary dealing of Rendering to New Track. Ah hah!

Thank you Satevis for your patience. And thanks to the rest of y'all!

Toodles

Grazie

rmack350 wrote on 2/12/2013, 12:50 PM
I sent you a loong missive privately. I've got to get some work done here so I'll not repost it all to the forum but you're welcome to do so if you think it's useful.

Satevis makes some good points. I agree with 99% of it but the Dynamic RAM Preview (DRP) cache is a bit of a black box. We can shake it and hear it rattle to make guesses about how it works.

The main thing here is that Vegas is filling the RAM Cache while it also does a render to disk. Frames in the RAM Cache are uncompressed, while the frames you write to disk could be in a compressed format, so they aren't the same thing. Also, frames in the cache could be half or quarter size depending on your preview setting, the frames you write to disk are whatever size you chose in the render settings. Again, not the same.

If you're watching that memory usage rise until Vegas phones home then that's not a desirable result and the best course would be to turn the DRP setting way down before starting your render to disk.

I *think* that Vegas didn't used to fill the DRP during disk renders. I know it was something I wondered about several years ago and I thought it wasn't doing that when I tested it. Maybe I'm wrong though. The main thing though is that it's not helpful for Vegas to be filling the DRP during a render. It might be useful sometimes for Vegas to draw from and empty the DRP, but never to fill it up.

Something to consider is that a Dynamic RAM Preview is only useful if it's actually in RAM. It probably follows that the DRP is structured in such a way that it can't be paged to disk (meaning other things will get paged to disk). So the DRP must consume physical RAM which you might need elsewhere during a render. Best to turn it way down during a render.

Flushing it by changing the preview window probably isn't good enough since it's still there to be filled. The only good recourse is to turn it way down before that render to disk. It'd be VERY NICE if Vegas would handle this more intelligently.

Another possiblility might be to set Vegas to *not* show render progress in the Preview window. I've not tried that but logically it ought to stop Vegas from caching frames to RAM. Of course that doesn't help if the DRP starts out full.

Rob
Grazie wrote on 2/12/2013, 12:57 PM
Rob, you have mail . . . .
farss wrote on 2/12/2013, 1:11 PM
I just checked this in the latest build of V11 and it does apparently fill the cache whilst Selectively Rendering To New Track but then releases the RAM as soon as the render completes. That makes sense because what's now in the prerendered file is NOT what I would see during Preview. I had Preview set to Preview / Half.

Bob.
rmack350 wrote on 2/12/2013, 1:20 PM
BTW, I just tried similar tests in Vegas 10 and it also loads up the DRP cache while it renders to disk. I don't see the advantage to doing this, but the behavior goes back at least to VP10.

I also tried turning off the Render Progress in Preview setting. That has no effect on whether Vegas loads up the DRP during a render. For that matter, I tried setting the Preview at a quarter frame to see if that changed the slope of the graph in TaskManager as Vegas filled the cache. It doesn't, so maybe Vegas is actually ignoring the preview setting and caching full resolution frames during the render.

I really don't want to dig up Vegas 5 to test this...

Rob
rmack350 wrote on 2/12/2013, 1:30 PM
"...but then releases the RAM as soon as the render completes."

In my last test with 10 I didn't let it complete because I just was checking to see if it was loading the DRP. So now you've raised a new question. Is your preview window set to something other than Full? Maybe Vegas finishes the render, checks the preview setting, and decides it needs to dump the DRP Cache?

The thing is, it might be more helpful if Vegas just capped the DRP to a sane setting during a render, or to your own setting if it's lower as some people are setting it to zero.

Rob
rmack350 wrote on 2/12/2013, 2:02 PM
Okay, I couldn't resist. In VP10, the Dynamic RAM Preview cache gets filled while you render. If your preview window is set to full then the DRP is retained at the end of the render to disk. If the preview setting is something else, like "Quarter" for example, then the DRP cache gets dumped once the render to disk is finished.

Rob
farss wrote on 2/12/2013, 2:36 PM
I think the Preview RAM is also used as a general purpose frame buffer by Vegas.
Depending on the codec being encoded frames may need to be cached to save a lot of time in the render process.

Also I now understand why dialing in some Motion Blur impacts how many frames can be held for Preview. To compute MB Vegas needs adjoining frames and they are being stored in the Preview RAM.

Bob.
rmack350 wrote on 2/12/2013, 3:44 PM
I think the Preview RAM is also used as a general purpose frame buffer by Vegas.

I think the fact that vegas would quite merrily fill an 8GB RAM cache whether it needs to or not makes the idea that it's doing anything intelligent very suspect.

I got an answer from SCS a long time ago where they said that Vegas fills the cache dynamically, based on need. This could mean that Vegas is caching every frame, or every other frame, or every third frame, or just frames here and there where it needs them.

My guess is that Vegas is just timing the frame renders and whenever it can't get a frame rendered in real time it saves a frame or some frames to the cache. It might have to take two approaches though. In the shift+B case maybe it caches every frame that renders late. In the cache-as-you-play case, it probably saves previous frames whenever a new frame has to be skipped. In this later case that means there must always be some previous frames available to save.

In the case of a motion blur, yeah it needs adjacent frames to work with but isn't it also just a slow enough process that Vegas would end up having to cache every single frame because it'll never render one in time?

To put it another way, a fixed Preview RAM would hold a fixed number of frames no matter what. It just goes twice as far if it holds every second frame in a region.

Rob

PS. Why are you awake, Bob?
farss wrote on 2/12/2013, 6:25 PM
"Why are you awake, Bob?"

Sore knee woke me up.

Bob.
rmack350 wrote on 2/12/2013, 7:16 PM
Sorry to hear that. I've got one of those but it just gets mad in cold weather. It doesn't wake me up.

I filed this with SCS as a product suggestion. Vegas should be managing Preview RAM better during renders to disk. There's no need to cache away 8GB of frames you won't need during a render. My suggestions were to either temporarily reduce the preview RAM to a sane number (or the user's number if it's lower) OR to use the usable frames in the cache as it renders and then flush them and reduce the cache.

I also suggested a dashboard panel with a slider and flush button for preview RAM, and maybe also buttons to enable/disable GPU acceleration. All in one easy to reach spot.

Maybe the dashboard could also have some meters to show how much memory is in use. It'd be a great distraction for the Natural Born Lever Pullers.

I didn't file it with tech support because ... what's the point of that?

Rob