I have a green screen with 2 pro 1200 watt lights and two backdrop lights and I still get the green glow around my human subjects.
Can this be fixed with a Video effect in Vegas?
I currently use secondary color correction for green screen and then Chroma key video effects.
If it's spill you have from the screen on your subjects, you probably won't have good results removing them. Having the subject too close to the screen is a common problem. Light reflects off the screen and lands on the subject with a green cast, and that's nearly impossible to fix even with Commotion or other high end tools. Vegas is a great key tool, but you're asking Vegas to "unbake a cake" so to speak, by pulling out an ingredient that's not going to pull out easily.
Move your subject away from the screen for at least the distance of their height. 5'5" subject? Pull them 5' from the screen. If they are lit with the same light as the screen is, you're gonna really be hating the result, you need to separate them with light, not merge them with light.
A good backlight or rimlight on the subject help wonders in pulling them away from the backdrop. Plain old white light will work but a soft magenta or pink gel on the backlight will cancel out the green even more. Spot's right about distance too. Get them as far away as possible to minimize spill.
It sounds like WAY too much wattage is on the screen. Try lighting the subject first with the minimal amount necessary for the camera not to be struggling to see them, then slowly add just enough to the screen to make it a deep rich green.
I had a horrible time using the Green Screen features of Vegas 4 no matter what I tried. I did not have time/money to try better lighting options. Because my green screen had some dark grey spots due to poor lighting, I could not get rid of those spots without removing some of the greys in my objects.
HOWEVER, using Premiere 6.5, I was able to remove the green screen [including the grey looking areas] without affecting my objects [people-moving/talking] Once I removed the background in Premiere and saved to an AVI, I edited the entire project in Vegas [I hate editing in Premiere] Premiere left NO haze around my objects. It seemed to do a better job of isolating the background colors.
I might have not tried hard enough with Vegas, but I have less experience with Premiere & was able to figure it all out in 15 minutes.
I almost feel bad about this post but yeah...Vegas doesn't really have a good chroma key feature. And yes, I have to agree with rhino that Premiere is better at keying...that and the ability to now use both After Effects Plug-ins AND Photoshop plug in filters.
Vegas is the best NLE going but it falls a little short with anything to do with good FX work. And believe me I've tried !