Memory leak in motion tracking?

klt wrote on 1/3/2019, 2:27 AM

Happy new year!

I started this week to work on some "talking head" video. I wanted to motion-track the cravat of the talking person, this is an approx. 27 minutes of talk. Started the motion tracking, and after a while noticed, that Vegas took all the memory of my computer and then crashed. About the 60% of the motion tracking was done before the crash. Then restarted Vegas, and it did the rest, but now takes up more then 80% of the memory. I needed to retart Vegas anagain to be able to continue working. Anyone else noticed this? This is with Vegas16 build 352, I don't remember I had this problem with the previous build.

Comments

Musicvid wrote on 1/3/2019, 11:13 PM

No prior reports of this that I recall -- maybe someone else has seen it.

"Memory leaks" are pretty rare starting with Win 7.

 

klt wrote on 1/4/2019, 12:18 AM

This looks to me very much like a memory leak.

Mormally for this purpose I make the motion tracking here I don't need keyframes to be so frequent. One keyframe per second would be more than enough. I set it to every frame because I noticed that placing more keyframes per time makes the leak more noticeable.

Last changed by klt on 1/4/2019, 12:19 AM, changed a total of 1 times.

Camera: JVC GY-HM600

Desktop: AMD Ryzen 5 1600, 16GB RAM (dual channel 2400 MHz) - Videocard: Radeon R9 380 2GB

Laptop: i5 5200u, 8GB RAM (1600MHz single channel) Videocard: integrated HD5500

Musicvid wrote on 1/4/2019, 3:44 AM

I set it to every frame because I noticed that placing more keyframes per time makes the leak more noticeable.

So are you saying putting 30 extra buffers each second in memory shouldn't overload it? Just how much scratch space do you have?

Use fewer Keyframes.

klt wrote on 1/4/2019, 4:22 AM

My bad english again, sorry.

In the real project I use way much fewer keyframes, as I don't need to follow the person that tight. I just need him keep framed, in an 1:1 square on the top-left of the screen, the rest is a logo and a presentation.

I just wanted to show what is going on, and for that I set the keyframe interval so short. Otherwise I should have uploaded a 30 min video 😎

So the phenomenon: motion tracking 20 minutes video, and Vegas takes up 14GB of RAM and chokes...

It simply crashes, and when restarting restore the last autosave, and continue from the last good keyframe, or I do pay attention, and cancel the motion tracking before the crash, save the project, close Vegas, reopen, and continue from the last good keyframe. So the workaround is there, it's just ugly.

The question is that anyone else can confirm, that motion tracking slowly but surely eats up memory? I guess, the majority just uses motion tracking to pin an image to something, or moves an effect along a moving thing, thus uses this motion tracking for a few seconds. Until now I motion tracked few minutes of videos the same way.

Just those were short enough not to notice the missing memory ;)

The RAM is not taken by the keyframes, as those are present if I reopen the project, and then there's no huge RAM usage, just normal. I mean, as I work with Vegas, the RAM usage of it's process grows up to somewhere between 3..5GB and stays there until I finish the project, except motion tracking.

The problem is I think, that during motion tracking some buffers are allocated and never freed.

I went back to build 307, and it's the same in this regard, so it's not the build. I think it's just the length of video, becasue I did only much shorter motion trackings so far - this is why I noticed this for the first time.

 

Last changed by klt on 1/4/2019, 4:23 AM, changed a total of 1 times.

Camera: JVC GY-HM600

Desktop: AMD Ryzen 5 1600, 16GB RAM (dual channel 2400 MHz) - Videocard: Radeon R9 380 2GB

Laptop: i5 5200u, 8GB RAM (1600MHz single channel) Videocard: integrated HD5500

Musicvid wrote on 1/4/2019, 10:07 AM

So the phenomenon: motion tracking 20 minutes video, and Vegas takes up 14GB of RAM and chokes...

So you added 35,964 keyframes and it used up all of your buffer memory?

I am not surprised, Kit.

You are not "leaking" memory, you are using it all up.

Like flooring the gas pedal with the brakes on. Uses a lot of gas but you won't get very far.

Happy Otter has some tracking built in. Have you tried it?

 

 

 

klt wrote on 1/4/2019, 11:57 AM

you added 35,964 keyframes

No. I added approximately 8000 keyframes by mistake, because in the first run forgot to change the keyframe interval from default 5 to 25. Otherwise I would just created approx. 1600.

But I could do this just in 2 steps, because around 60% Vegas crashed the first time. So adding ~5000 keyframes fills up the 16GB memory.

So this small mistake lead to me to possibly find a bug.

Again: the keyframe interval "1' was set here for demo purpose only.

Questions:

What's the problem with 8000 keyframes right after creating them, so they need a dozen of GB?

Why is that amount of "used up" memory not freed, if I delete the keyframes?

Why is enough the fraction of memory for storing them and working with them after they are already present? In fact later I duplicated them because I copied them to another instance of the clip to pan/crop with my script? So here we talk about 16000 keyframes, and Vegas still handles them with zero problems. Memory is used up only while tracking the motion.

When I stop the motion tracking (cancel it) the already occupied memory stays occupied. It's never released until I quit Vegas.

You are not surprised, but I am. How come here brake and gas pedals?

And I still think there's a leak in that process where Vegas creates those keyframes.

 

Musicvid wrote on 1/4/2019, 6:52 PM

I don't write the code, Kit.

Good luck with your experiment.

fr0sty wrote on 1/5/2019, 7:47 PM

Is there any way you can share your project so we can try to reproduce this?

Systems:

Desktop

AMD Ryzen 7 1800x 8 core 16 thread at stock speed

64GB 3000mhz DDR4

Geforce RTX 3090

Windows 10

Laptop:

ASUS Zenbook Pro Duo 32GB (9980HK CPU, RTX 2060 GPU, dual 4K touch screens, main one OLED HDR)

klt wrote on 1/6/2019, 6:57 AM

@fr0sty:

Yes, shure. Thank you.

However, no need for the whole project, I have set up a "zero" project for this test.

I have preset the Bezier mask to the cravat of the person, but left the other settings on default values.

Just start motion tracking on the starting point. Of course there will be moments, where the priest turns too much, or hides the cravat with his hands. Not too often though. Just scrub forward a bit, adjust the mask and continue tracking. Watch memory consumtion of Vegas. Then when you already see a memory consumption noticable even with "naked eyes" - say 6GB for Vegas process alone-, you can cancel tracking. Delete the keyframes. Remove the Bezier from the event. Delete the event, or close the whole project. do whatever you want: the memory consumption stays until you quit Vegas. Isn't this a memory leak?

Can you confirm this phenomenon?

I uploaded the project here:

http://kovacsoltvideo.hu/videos/motiontracktest.zip

I can keep it up until wednesday.

 

Last changed by klt on 1/6/2019, 6:57 AM, changed a total of 1 times.

Camera: JVC GY-HM600

Desktop: AMD Ryzen 5 1600, 16GB RAM (dual channel 2400 MHz) - Videocard: Radeon R9 380 2GB

Laptop: i5 5200u, 8GB RAM (1600MHz single channel) Videocard: integrated HD5500

musko wrote on 1/6/2019, 10:45 AM

I try your project and here is memory consumption:

And still going up.
FYI starting with 500MB RAM consumption.

j-v wrote on 1/6/2019, 11:47 AM

Nothing wrong with me at laptop from signature, no leaking, no unusual use of memory with Vegas playing your file while Bezier Masking is following the movements of the head in the video and OBS is running to make a screenvideo. Something has to be wrong with your hardware or settings I think.

met vriendelijke groet
Marten

Camera : Pan X900, GoPro Hero7 Hero Black, DJI Osmo Pocket, Samsung Galaxy A8
Desktop :MB Gigabyte Z390M, W11 home version 24H2, i7 9700 4.7Ghz,16 DDR4 GB RAM, Gef. GTX 1660 Ti with driver
566.14 Studiodriver and Intel HD graphics 630 with driver 31.0.101.2130
Laptop  :Asus ROG Str G712L, W11 home version 23H2, CPU i7-10875H, 16 GB RAM, NVIDIA GeForce RTX 2070 with Studiodriver 576.02 and Intel UHD Graphics 630 with driver 31.0.101.2130
Vegas software: VP 10 to 22 and VMS(pl) 10,12 to 17.
TV      :LG 4K 55EG960V

My slogan is: BE OR BECOME A STEM CELL DONOR!!! (because it saved my life in 2016)

 

klt wrote on 1/6/2019, 4:56 PM

@musko, @j-v Thank you for testing.

Musko, I have the feeling that you can confirm my suspicion.

J-v, I think you judged too early at 9%.

While I'm doing different things, made screenshot videos how it behaves here.

Here's the first video:

It shows when I start the process, Vegas takes below 1GB.

Now at 40% I cancelled, at this point Vegas took about 9GB RAM. I saved it again, to reuse.

Around 1 minute in the video I delete the keyframes, and show that only little memory is freed, the occupation stays about 9GB. Then remove the Bezier FX, and memory stays filled.

I close the project, and there's a Vegas open completely empty, and still takes around 9GB.

That memory could be cached useful buffer, but when I readd the FX, and start tracking again, the memory consumption starts to grow again. This is ahown around 2:01 in the video. So those allocated buffers are not reused.

So I close Vegas, reopen it, now it consumes very few memory. In this video I show that even working with that huge lot of keyframes and doing some random edits Vegas still uses a sane amount of RAM:

So the memory is eaten up while doing the motion tracking. Vegas can use up a dozen GB, and never frees before quit. At least on my machine. If this is only at me, the pardon. But if this behavior is similar on other machines too, I think it's a flaw in motion tracking.

Anyone can confirm, or contradict?

Thank you all...

 

 

j-v wrote on 1/6/2019, 5:20 PM

I tested your problem first with my own GoPro HEVC FHD 50p files ( no proxies) of 17 minutes, much heavier than yours, making keyframes every 5 seconds and waited till the end.
Starting to look the amount of memory used was about 3,5 GB and after loading the file and played it it was rising till and stayed at 5 GB of my availabble 16 GB.
After adding the mask and setting it to do the tracking it rises to 5,5 and stayed there till the end of the tracking on the 17th minute.
Would you like and see a videograb of 17 minutes to understand it's something with you ( and not with me)?

But I don't further care your problem for which I tried to help, sorry to "mislead" you by saying it's something with your hardware.😔👎

met vriendelijke groet
Marten

Camera : Pan X900, GoPro Hero7 Hero Black, DJI Osmo Pocket, Samsung Galaxy A8
Desktop :MB Gigabyte Z390M, W11 home version 24H2, i7 9700 4.7Ghz,16 DDR4 GB RAM, Gef. GTX 1660 Ti with driver
566.14 Studiodriver and Intel HD graphics 630 with driver 31.0.101.2130
Laptop  :Asus ROG Str G712L, W11 home version 23H2, CPU i7-10875H, 16 GB RAM, NVIDIA GeForce RTX 2070 with Studiodriver 576.02 and Intel UHD Graphics 630 with driver 31.0.101.2130
Vegas software: VP 10 to 22 and VMS(pl) 10,12 to 17.
TV      :LG 4K 55EG960V

My slogan is: BE OR BECOME A STEM CELL DONOR!!! (because it saved my life in 2016)

 

klt wrote on 1/6/2019, 5:48 PM

Thank you again for testing. I just saw your videograb around 9%, and around 10% I got similar memory consumptions I see on your video.

Lacking the other informations you just provided in your recent post, I thought you gave up the testing around that 10%.

Can't say enough thank you for taking your time on this.

j-v wrote on 1/7/2019, 4:05 AM

OK, but I have also a question asking you:

Why should a program as Vegas with its new possibility have to track motion during a very long time of an upsidedown head hanging in the air while that head is only talking. No more or less I could see.
Do you think that tool is designed for such things?

met vriendelijke groet
Marten

Camera : Pan X900, GoPro Hero7 Hero Black, DJI Osmo Pocket, Samsung Galaxy A8
Desktop :MB Gigabyte Z390M, W11 home version 24H2, i7 9700 4.7Ghz,16 DDR4 GB RAM, Gef. GTX 1660 Ti with driver
566.14 Studiodriver and Intel HD graphics 630 with driver 31.0.101.2130
Laptop  :Asus ROG Str G712L, W11 home version 23H2, CPU i7-10875H, 16 GB RAM, NVIDIA GeForce RTX 2070 with Studiodriver 576.02 and Intel UHD Graphics 630 with driver 31.0.101.2130
Vegas software: VP 10 to 22 and VMS(pl) 10,12 to 17.
TV      :LG 4K 55EG960V

My slogan is: BE OR BECOME A STEM CELL DONOR!!! (because it saved my life in 2016)

 

klt wrote on 1/7/2019, 7:52 AM

OK, let me try to explain that with my very limited english. That's also the story of why I was excited about that motion tracking.

I often have to make video on conferences. This means I record the speech, follow the talking person with my camera. Not a very creative work, but pays well.

I also get the presentations to include in the video, either pdf, series of pictures, or the powerpoint file itself.

I never ever record a projection screen on a conference with the camera. That sucks so much, ugly, bad colored, flickers, etc... Instead, I take only the picture of the talking person. Then in Vegas I'll have at least 4 video tracks:

-track1: the talking person full screen

-track2: the presentation

-track3: a thumbnail of the speaking person (this is his/her video cropped to 1:1 or 4:3 and resized to fit beside the presentation)

-track4: some backdrop.

When the speaker just tells his/her stories, so that the content of the presentation is not important at the moment, I have him/her fullscreen. When the speaker tells something closely related to the presentations picture, or any other way it is important to show, I have the PIP screen. A very-very short sample:

I found that the speakers tend to love this kind of setup. 😉

When I do the shooting, I try to "be there", and when detect that the speaker is very closely on topic, I get in my "tight mode", I compose him/her strictly in the middle of picture as possible, so that he/she preferably fits in an 1:1 crop, but at least in the 4:3 marker. That makes the creation of the "thumbnail" much easier. But recording that is not too easy if the speaker walks here and there quickly on the stage. So when I know I won't need the PIP screen, I loosen the tight follow, and keep just framed in 16:9.

Well, sometimes I make this "loosen" by mistake or when the content is extremely boring to me my brain stops working really, and I find myself forgetting to get in the "tight" mode.

Whenever that happens, I have to make the thumbnail closely follow the speaker by placing dozens of pan/crop keyframes. That's not a fun job, even if Vegas is a great partner in doing that.

Now please imagine, If I can motiontrack the speaker, even for couple hours long, I can completely forget the "tight" mode. It gets simply totally unnecessary. You can now guess why I was so excited about this new feature?

Now about the story of the "head hanging in the air".

Once I met the pastor, we got in a good friendship. His idea was to record preaches, and he already had some attempts. We agreed that I can and will help. The task is basically the same as with a conference. This pastor and his church is in a distance from me, so they do there everything, I just edit what they record. I do this them for free.

Later the camera was disturbing standing on the floor, so someone made a "stilling" mounted on the ceiling of that church. That stilling holds a small tripod head, which carries the small camera they use. But then the camera was way too high, I told it should come down at least a half meter. But making another longer stilling would be too much work and cost. So I said, try to mount the tripod head below the arm of that stilling (of course that also means upside down), so the cam gets 30 cm lower. It is still a bit high, but acceptable. So they didn't need to make a new stilling, while turning back the upside down picture means zero additional work to me. Because they can't learn how to compose the picture, I have to crop it anyway.

So that sample you tested looked very silly and stupid. But it was just for testing if the problem I found is in my device, or is it a more general problem. I assumed it is a more general thing, because I try to keep everything in my device as perfect as possible - probably wrong assumption.

At the end that silly thing looks like this (very short sample):

I beg your pardon again, if I did upset you. And please believe, I appreciate your valuable time you spent on my silly thing.

j-v wrote on 1/7/2019, 9:18 AM

I know how to do that kind of things and many years back in my working days I did a lot of such presentations, so nothing new and strange to me, also not your way of working.
But the tracking part of such a task is never for more than a short period of time with the same view, otherwise it is becoming very dull.
So what I normally do is using parts to follow the head in this case, therefore you can split the video and masks only a particular part, otherwise you get way too much keyframes.
That works this way: as long as the program is running with a task ( Bezier follow track motion) the keyframes will not be stored somewhere but are as long as the task runs holding in memory, untill you end this task and carry on to use those keyframes ( (for instance for letting following it by a picture or text).
When that tasks ends by storing the result by saving, the memory will be cleared.
You are able to regulate the number of made keyframes in more than on way:

1. by using splitted parts of a video
2. by decreasing the number of keyframes
3 by not selecting preciser tracking of the mask.

met vriendelijke groet
Marten

Camera : Pan X900, GoPro Hero7 Hero Black, DJI Osmo Pocket, Samsung Galaxy A8
Desktop :MB Gigabyte Z390M, W11 home version 24H2, i7 9700 4.7Ghz,16 DDR4 GB RAM, Gef. GTX 1660 Ti with driver
566.14 Studiodriver and Intel HD graphics 630 with driver 31.0.101.2130
Laptop  :Asus ROG Str G712L, W11 home version 23H2, CPU i7-10875H, 16 GB RAM, NVIDIA GeForce RTX 2070 with Studiodriver 576.02 and Intel UHD Graphics 630 with driver 31.0.101.2130
Vegas software: VP 10 to 22 and VMS(pl) 10,12 to 17.
TV      :LG 4K 55EG960V

My slogan is: BE OR BECOME A STEM CELL DONOR!!! (because it saved my life in 2016)