Low memory errors I can't explain

drdoak wrote on 6/21/2008, 11:33 AM

For over a week now I've been trying to render an hour long video that consists of mostly JPEGs from an 8MP camera and video from a Canon TX-1 (motion jpeg AVI if I recall). I'd prefer to render to 1080-30 WMV but have also tried 720-30 WMV, equivalent AVI settings, MPEG2. All of them give me low memory errors after a couple of hours. WMVs all fail after around 1.5GB, give or take a few hundred MB. I have tried increasing the paging file so far as to 12GB. I have tried converting all JPEGs to PNG files. I disable OneCare and the NIC before rendering. I've disabled most all powersaving. I have set the preview RAM to 0 and tried both 2 and 4 threads.

The last failure last night managed to capture a little bit of performance data. It seems to indicate that while I sat here and watched it for an hour and saw RAM usage of 320-550MB of RAM by Vegas, it eventually spiked to 1500MB and died. That's also when the monitoring died.

I'm running Vista Ultimate 32bit w/ SP1. This is a very recent install and has little installed aside from the basic (Office and Creative Suite, Firefox, etc...) stuff. I have an Asus P5K, Intel Q6600 Quad Core 2.4GHz, 4GB DDR2 (Vista reports 3GB for use, 4 installed). Unfortunately, this was my last XP computer, so I don't have an XP machine to go try this on, though I may try it at work one night.

Any helpful thoughts? Last year when I made another hour long video on XP, it still took 5-10 tries before it worked for WMV 1080p, but 720p worked every time.

Comments

Himanshu wrote on 6/21/2008, 11:57 AM

In your message I didn't read exactly what errors are presented to you when Vegas dies, but in your title you say "low memory errors" so I'm going to take a shot here - each process on a 32-bit machine can address 2GB of memory. So if at some point Vegas gets to that limit, the OS will not allow it to go further. In that case increasing the swap space to 12GB doesn't do much for you. Having 4GB of physical RAM also doesn't help because you're still running on a 32-bit OS.

What *may* help, and I have no idea if it will, is to attempt to use the /3GB switch that you can find more information about on Microsoft's web site in this KB article -- it applies to WinXP so I don't know about Vista. It also requires application support and I don't know if Vegas supports it. Might require some research. Give it a try and let us know what happens.

blink3times wrote on 6/21/2008, 12:11 PM

Vegas has a problem with large jpg files. Resize the jpg's so that they are no bigger than 700 900 k. This SHOULD clear the memory problem.

You can also try and convert the jpg's to png's. Vegas seems to prefer png files.

drdoak wrote on 6/21/2008, 12:59 PM

I converted all the JPEG files to PNG already since that seemed to be a common issue in the forums. All that seems to have done is make it always die with a low memory error instead of crashing half the time. I suppose that counts as some progress...

I will look into the 3GB thing. Apparently http://msdn.microsoft.com/en-us/library/aa906211.aspx give info on how this can work in Vista. It does seem though, according to the performance monitor, that it errored out using only 1.5GB of RAM and never made it to 2GB. It'd be nice if it would just let me set how much memory Vegas can use IN Vegas.

rmack350 wrote on 6/21/2008, 2:19 PM

First off, whether it's jpeg or png, at some point Vegas has to deal with the media in an uncompressed state. This is why you might see memory problems using 8MP images that you probably wouldn't see if you resized them down to "just big enough". Hypothetically, PNG vs JPEG compression shouldn't make a difference since they should both consume a similar amount of RAM while uncompressed.

Regarding the /3GB switch, this might help if you also hacked the Vegas executable to make it large address aware. There's a utility that can do this for you pretty painlessly but I don't remember its name. You can probably figure that out by searching this forum. Remember that making Vegas and your OS able to use 3 GB doesn't mean that Vegas will use 3 GB of RAM - it'll be a combination of RAM and page file. Also keep in mind that Vista needs memory too so if you go this route expect to see a lot of disk thrashing.

Short answer is that you should resize the stills to be just big enough for your needs.

Rob

drdoak wrote on 6/22/2008, 9:05 AM

I hesitate to start resizing because I do lots of panning and zooming. It'd be hard to tell how small I can make each one individually. Also, there's like 400 of them in there, so that's something I'd like to consider a last resort. I also did a video like this in XP a year ago that was like 5 minutes shorter with similar picture counts from the same camera. That worked.

I copied the entire project over to my laptop (Dell D630, Core2 Duo at 2.2GHz, reports 1GB RAM). It actually made it a little bit further in the render (twice) than my primary PC with triple the reported RAM and a quad core. If anything I would have though it should error out much sooner.

The fact that it uses a pretty set amount of memory for over an hour when I'm watching it, but then suddenly spikes sometime in the middle of the night sounds like a memory leak or other bug to me. Also the fact that it dies in virtually the same spot on more than one machine with somewhat different RAM and CPU.

Going to look into the 3GB hack for Vegas to see if that does anything for me. Will also try this on an XP machine at work. Has anyone had more or less issues regarding rendering with the various flavors of 8.0? A or B over C or D or something? Is there a good way to estimate file size for uncompressed quicktime or avi files? Looking at 1440x1080 and 30fps or so.

Also, Is there some reason with a single stream on pictures that Vegas would need to keep more than 2 or 3 loaded into memory at any one time?

JJKizak wrote on 6/22/2008, 9:25 AM

Render out smaller sections to AVI (size determined by you) then substitute the AVI's on the timeline then delete the individual jpgs or pngs of that avi. Once you get all of the small sections rendered to avi and substituted on the timeline then the render will be a piece of cake and fast. You must delete all of the individual jpgs or pngs or you gain nothing. I would guess that you would have to make about at least 6 avi's. You should not have to do this with any DV on the timeline, just the pictures.
JJK

rmack350 wrote on 6/22/2008, 10:40 AM

Yes, this is the best advice. Work smarter, lift lighter loads and more of them.

You might play around with just running a prerender of your project to AVI format, then doing the wmv encode. Prerenders are generally made in short segments. (it's also possible to go find the prerender files and drag them onto a new track in the project. That ought to make them permanent even after you do a save-as and then remove all the stills. Hmmm...I wonder if a script could be written to promote prerender files to a track?)

Anecdotally, in seems like VP8 has a little more trouble with memory than V7. Maybe it's a leak, maybe it's a bug-like feature. Maybe you've got a corrupt image file.

I have the impression that Vegas keeps still image data in RAM for quite a while during a render. If you are using some flavor of long GOP media in addition to the stills then I'd think Vegas needs to keep a long stretch of media in memory but I doub't that that explains having a memory error after an hour or so of rendering.

Rob

Darren Powell wrote on 6/22/2008, 5:46 PM

Ground Control to Sony ...

Your software doesn't work.

I repeat ...

Your software doesn't work.

Where's 8.0c?

Ground Control to Sony?

Are you there ... over?

Has a space alien eaten your brain?

I repeat.

Your software doesn't work.

Darren Powell
Sydney Australia

Oops ... just tried to open a little 10 minute project with m2t's.

-----------------------------------------------------------------

Sony Vegas Pro 8.0
Version 8.0b (Build 217)
Exception 0xC0000005 (access violation) READ:0x258ED79C IP:0xD12F221
In Module 'mcmpgdmux.dll' at Address 0xD120000 + 0xF221
Thread: ProgMan ID=0x878 Stack=0x306F000-0x3070000
Registers:
EAX=00000000 CS=001b EIP=0d12f221 EFLGS=00010246
EBX=0000003d SS=0023 ESP=0306f130 EBP=0306f16c
ECX=78134c58 DS=0023 ESI=257f8f48 FS=003b
EDX=016a0608 ES=0023 EDI=258d4f48 GS=0000
Bytes at CS:EIP:
0D12F221: 8B 87 54 88 01 00 50 56 ..T...PV
0D12F229: E8 12 F7 FF FF 83 C4 08 ........
Stack Dump:
0306F130: 25809058 24340000 + 14C9058
0306F134: 257F8F48 24340000 + 14B8F48
0306F138: 0D12B52E 0D120000 + B52E (mcmpgdmux.dll)
0306F13C: 257F8F48 24340000 + 14B8F48
0306F140: 258D4F48 24340000 + 1594F48
0306F144: 00000000
0306F148: 25782F48 24340000 + 1442F48
0306F14C: 3BFFDD90 3A0C0000 + 1F3DD90
0306F150: 2322453A 23200000 + 2453A (mcmpegin.dll)
0306F154: 257F8F48 24340000 + 14B8F48
0306F158: 257971B8 24340000 + 14571B8
0306F15C: 00000000
0306F160: 00000000
0306F164: 7C838F36 7C800000 + 38F36 (kernel32.dll)
0306F168: 7C838F36 7C800000 + 38F36 (kernel32.dll)
0306F16C: 00000000
> 0306F170: 33BBB3DA 33BB0000 + B3DA (mcplug.dll)
> 0306F188: 33BBB520 33BB0000 + B520 (mcplug.dll)
> 0306F194: 33BB8449 33BB0000 + 8449 (mcplug.dll)
> 0306F1AC: 01163691 01150000 + 13691 (vegas80k.dll)
0306F1B0: 05DC1EF8 05AD0000 + 2F1EF8
0306F1B4: 0306F1E8 02F70000 + FF1E8
0306F1B8: 0000A6B4
0306F1BC: 3A47EF3C 3A0C0000 + 3BEF3C
> 0306F1E0: 01163578 01150000 + 13578 (vegas80k.dll)
0306F1E4: 05DC1EF8 05AD0000 + 2F1EF8
0306F1E8: 3BFFDD90 3A0C0000 + 1F3DD90
0306F1EC: 3A47EEF0 3A0C0000 + 3BEEF0
0306F1F0: 00000000
> 0306F1F4: 007425B2 00400000 + 3425B2 (vegas80.exe)
0306F1F8: 01FC6870 01F30000 + 96870
0306F1FC: 3A47EF3C 3A0C0000 + 3BEF3C
0306F200: 3A47EEF0 3A0C0000 + 3BEEF0
0306F204: 00000000
> 0306F21C: 00744E70 00400000 + 344E70 (vegas80.exe)
0306F220: 0306F234 02F70000 + FF234
0306F224: 00000000
0306F228: 0306F7A8 02F70000 + FF7A8
0306F22C: 00000000
> 0306F294: 7C910945 7C900000 + 10945 (ntdll.dll)
> 0306F298: 7C91094E 7C900000 + 1094E (ntdll.dll)
> 0306F2AC: 7C914190 7C900000 + 14190 (ntdll.dll)
> 0306F2B4: 7C901005 7C900000 + 1005 (ntdll.dll)
> 0306F2C4: 7C90EE18 7C900000 + EE18 (ntdll.dll)
> 0306F2C8: 7C910970 7C900000 + 10970 (ntdll.dll)
> 0306F2CC: 7C97E4C0 7C900000 + 7E4C0 (ntdll.dll)
> 0306F2D0: 7C913E6F 7C900000 + 13E6F (ntdll.dll)
> 0306F2D4: 7C913E62 7C900000 + 13E62 (ntdll.dll)
> 0306F308: 006E006C 00400000 + 2E006C (vegas80.exe)
- - -
0306FFF0: 00000000
0306FFF4: 005258F0 00400000 + 1258F0 (vegas80.exe)
0306FFF8: 00ADA678 00400000 + 6DA678 (vegas80.exe)
0306FFFC: 00000000

drdoak wrote on 6/23/2008, 10:51 AM

Along a slightly different vein... If I render a bunch of identical WMV files (settings wise) can I join them without reencoding? I was looking at Boilsoft's video joiner which has a couple of good comments around. Any experiences there?

Joining smaller WMVs would probably work for me and be much easier to manage on my existing free drive space.

Thanks

As an addendum... Some more strange behavior I'm seeing. I'm trying the render on a test XP system at work. It was getting ready to die of low memory right around the same place as usual for my other PCs. On a whim, I started up Media Player and had it start playing some sample music to see if it might do anything interesting with the system RAM. Strangely enough, everytime the songs cycle, the memory available resets to a higher number. The number doesn't seem to drop fast enough to keep leaking in a destructive way. The memory leak seems to have either been stopped or at least confused and the render is now several percent farther along than I've seen it before. When I try the render at home again tonight, I'm going to get me a couple of .5 - 1 minute songs to play in WMP and see what happens! If it works, I'll just shake my head and be happy for what it is I suppose...