I've created a relinking media nightmare

erikd wrote on 8/5/2008, 5:20 AM
I am still new to Vegas and have run into a pretty solid brick wall. Stay with me as this does get complicated.

I have hundreds of mxf files that were imported into Vegas using XDCAM explorer import. I edit away for a few weeks and realize that my library of clips are becoming a jumbled mess inside of Windows explorer. I want to reorganize them in Windows explorer so that as the library is built over time and used for many different jobs, the clips will be organized in a manner that will be logical for me to easily find clips and work with them.

We shoot XDCAM HD 50gb discs. Each disk for each shoot is numbered sequently 01 though whatever. When you get back to edit each mxf clip file on each disc is numbered automatically by the camera sequentially 001.mxf through whatever as well. Before I import them I rename the Disc itself to be specific to the shoot date of the project but I do not rename any of the individual clips.

Lets say I want to import Disc 04-08-01. In my jargon this would be April, 2008 Disc 01. I import the clips into a folder thats named Sony Vegas XDCAM Library and a sub-folder named 04-08. Now, on this particular shoot (there are many other shoots already imported into the library as well) there were seven discs so when I'm done importing there are 7 unique folders inside the 04-08 folder but there is no way to tell after importing which folder goes to which disc when navigating back through windows explorer. I thought I would rectify this problem by simply creating sub-folders named disc 01 through 07 and then putting the correct the unique folder into the correct disc folder and then simply relinking back inside Vegas. BIG MISTAKE!!!

It seems that Vegas and Veggie Toolkit for that matter can't see what I can see as plain as day. That is how to find the uniquely named folder in the correct sub-folder and then correctly relink the missing media for the timeline and project.

Again, an example. Let's say I want to relink Disc 05 Clip 004. Clip 004 from Disc 05 is in the unique folder named B9110500533705C10045080046020224 but what Vegas and Veggie Toolkit both do is to be perfectly content with just finding any clip named 004 in ANY folder within the 04-08 folder. With the example folder B911... Vegas dumbly offers clip 004 in the folder named F822700453705C10045080046020224 for a relink. WHAT??? Why can't it clearly see that the unique folder is the defining element for relinking offline media, not the clip name? Clip names in XDCam are basically useless what matters is the unique folder! Understand that I tell Vegas to look in the correct drive/folder including sub-folders and it can't figure out how to automatically match a unique folder name when relinking offline media? It needs me to point it to each correct individual file?

I know that Vegas gives you the option to try another search for a particular file if you don't like what it offers you but this very clumsy for a couple of reasons. One, the dialogue box for relinking is so small (and can't be resized) that the full path of the clip it shows is truncated and you can't see the important part of the path which of course is the unique folder name. Secondly, after you manually navigate to the correct clip and say yes, it then dumbly says that it has also found 005.mxf in the same folder would you like to relink? One would think now that you've shown it what the correct unique folder is that it would start to catch on at this point and only offer relinks for clips that are from that folder but it simply offers to link 005 (randomly it seems) incorrectly from another unique folder which is not the one that I have nagivated to.

I hope someone will tell me how to make Vegas look for the part of the path that matters when relinking, the unique folder name. The one that is a gazillion letters long to make sure it is unique. Not the goofy clip names created in XDCAM.

Keep in mind that there are hundreds of clips and I'm trying to avoid going in manually one clip at a time and telling Vegas, "look here stupid".

Erik

Erik

Comments

farss wrote on 8/5/2008, 6:00 AM
Clearly Vegas assumes clip file names are unique and they should be if your files were captured from tape. All shouldn't be lost, your problem should be solveable by someone writing a script that implements the logic that you need or have you tried contacting Peachrock, he might be able to knock up something quick and dirty to get you going again.

If a script cannot access what it needs to solve your problem exporting an EDL, haveing code massage that and then importing it would be another stratergy.

One other point, I think if you'd use Vegas's own explorer to re-order your files you wouldn't be in this mess.

Bob.
JohnnyRoy wrote on 8/5/2008, 6:52 AM
> Secondly, after you manually navigate to the correct clip and say yes, it then dumbly says that it has also found 005.mxf in the same folder would you like to relink?

You're right this is kind of hard to follow but there may be a quick solution if your naming is consistent. Help me to understand with an example.

What was the old fully qualified original names of Clip 004 and Clip 005?

What is the new fully qualified names of Clip 004 and Clip 005?

If the difference is consistent with the other names, it may be possible to write a script to correct all of the names in your project.

~jr
erikd wrote on 8/5/2008, 7:55 AM
Johnny,
I believe that I am very consistent with my file naming. Here is an example of what would happen when trying to relink.

First, the clip that Vegas is looking for in the timeline.
E:\Import with Sony Vegas Proxy Library\04-08 Israel\Disc 05\2D072100513705C10045080046020224\Sub\C0004S01.MXF

Then the clipt that Vegas recommends for relinking.
E:\Import with Sony Vegas Proxy Library\04-08 Israel\Disc 01\2F811700453705C10045080046020224\Sub\C0004S01.MXF


Another example doesn't yield anything new other than showing the consistency of the naming.

What Vegas wants:
E:\Import with Sony Vegas Proxy Library\04-08 Israel\Disc 03\6C521900483705C10045080046020224\Sub\C0005S01.MXF

What it offers as a solution:
E:\Import with Sony Vegas Proxy Library\04-08 Israel\Disc 07\9DC91D00533705C10045080046020224\Sub\C0005S01.MXF


I hope this answers the question in a way that is what you need.
Thank you very much and I must say that I've been watching your posts and comments closely for many months now. I greatly respect your work.

Erik
erikd wrote on 8/5/2008, 8:11 AM
One other point, I think if you'd use Vegas's own explorer to re-order your files you wouldn't be in this mess.

Fars,

Is this for sure? If Ihad created the new sub folders in Vegas explorer and then moved files around in Vegas explored, there wouldn't have been any relink issues?

Erik
JohnnyRoy wrote on 8/5/2008, 8:25 AM
> I hope this answers the question in a way that is what you need.

No it does not. :( I need to know the correct file name to substitute. Let me ask it a different way.

Given the original file name:

E:\Import with Sony Vegas Proxy Library\04-08 Israel\Disc 05\2D072100513705C10045080046020224\Sub\C0004S01.MXF

What is the CORRECT new file name that should be linked?

~jr

JohnnyRoy wrote on 8/5/2008, 8:29 AM
> Is this for sure? If Ihad created the new sub folders in Vegas explorer and then moved files around in Vegas explored, there wouldn't have been any relink issues?

No. this only works for renaming files. You actually moved folders and that would not be possible because Vegas has the file open so the file system won't allow it. Try it and see. It doesn't work.

~jr
erikd wrote on 8/5/2008, 8:41 AM
Oops, made a mistake. This is how the post should have read.

First, the clip that Vegas is looking for in the timeline.
E:\Import with Sony Vegas Proxy Library\04-08 Israel\2D072100513705C10045080046020224\Sub\C0004S01.MXF

Then the clipt that Vegas recommends for relinking.
E:\Import with Sony Vegas Proxy Library\04-08 Israel\Disc 01\2F811700453705C10045080046020224\Sub\C0004S01.MXF

What the correct link is:
E:\Import with Sony Vegas Proxy Library\04-08 Israel\Disc 05\2D072100513705C10045080046020224\Sub\C0004S01.MXF

Erik
JohnnyRoy wrote on 8/5/2008, 3:43 PM
I don't think you could expect Vegas or any program to understand that the MXF folder name is what you want to match on. That's asking for a lot of intelligence that just can't be assumed.

Here is a script that will fix your particular problem:

/**
* Program: ResolveMXFMedia.cs
*
* This script will fix a folder renaming problem when using XDCAM discs.
* It does this by finding the correct disc folders and updating the project
* media pool.
*
* Author: John Rofrano
* Copyright: (c) 2008 VASST, All Rights Reserved
* Version: 1.1
* Revision: Aug. 6, 2008
**/

using System;
using System.IO;
using System.Collections;
using System.Windows.Forms;
using Sony.Vegas;

class EntryPoint
{
Hashtable MXFFolderCache = new Hashtable();

//
// Change this to be the base path to search (path must end with "\")
//
string basePath = @"E:\Import with Sony Vegas Proxy Library\";

public void FromVegas(Vegas vegas)
{
int processCount = 0;
int totalOfflineFiles = 0;
ArrayList replacedMedia = new ArrayList();

//
// Find all offline MXF media that are in the base path
//
foreach (Media media in vegas.Project.MediaPool)
{
string originalFilename = media.FilePath;

if (media.IsOffline() && originalFilename.StartsWith(basePath) && originalFilename.ToUpper().EndsWith(".MXF"))
{
totalOfflineFiles++;

// break up the path into pieces we can use
string mxfPath = GetRightPath(originalFilename, 3);
string mxfFolder = mxfPath.Substring(0, mxfPath.IndexOf('\\'));
string mxfFile = mxfPath.Substring(mxfPath.IndexOf('\\'));

// try and find the MXF folder
string newFolder = FindFolder(basePath, mxfFolder);
if (newFolder != null)
{
string replacementFilename = newFolder + mxfFile;
Media newHDMedia = new Media(replacementFilename);
if (newHDMedia.IsValid())
{
try
{
media.ReplaceWith(newHDMedia);
processCount++;
replacedMedia.Add(originalFilename);
}
catch (Exception e)
{
MessageBox.Show(String.Format("Error: Could not replace [{0}] Reason: {1}", replacementFilename, e.Message));
}

}
}
}
}

// Remove the replaced files from the pool
foreach (string replacedFilename in replacedMedia)
{
if (vegas.Project.MediaPool.Contains(replacedFilename))
{
try
{
vegas.Project.MediaPool.Remove(replacedFilename);
}
catch (Exception e)
{
MessageBox.Show(String.Format("Media: {0} could not be removed.\nError: {1}", replacedFilename, e.Message));
}
}
}

MessageBox.Show(String.Format("Processed: {0} out of {1} off-line file(s)", processCount, totalOfflineFiles), "Complete");
}

/// <summary>
/// Does a depth first search of folders looking for a subfolder with the given mame
/// </summary>
/// <param name="startFolder">The folder to start the search</param>
/// <param name="foldername">The folder name to search for</param>
/// <returns></returns>
private string FindFolder(string startFolder, string foldername)
{
// Since MXF disc names are unique, keep them cached to minimize searching
if (MXFFolderCache[foldername] != null)
{
return (string)MXFFolderCache[foldername];
}

string foundFolder = null;

// Search for the folder
DirectoryInfo startDir = new DirectoryInfo(startFolder);
DirectoryInfo[] childDirs = startDir.GetDirectories();

foreach (DirectoryInfo childDirectory in childDirs)
{
if (childDirectory.Name.Equals(foldername))
{
// cache the hit
MXFFolderCache[foldername] = childDirectory.FullName;
foundFolder = childDirectory.FullName;
break;
}

// Recurse through the child folders
if (childDirectory.GetDirectories().Length > 0)
{
foundFolder = FindFolder(childDirectory.FullName, foldername);
if (foundFolder != null)
{
break;
}
}
}

return foundFolder;
}

/// <summary>
/// Returns the right-most path give the number of slashes from the right
/// </summary>
/// <param name="basePath">The path to search</param>
/// <param name="slashCount">The number of slashes from the right</param>
/// <returns>The right-most part of the path from the slash to the end</returns>
private string GetRightPath(string basePath, int slashCount)
{
int lastSlash = basePath.Length;
for (int i = 0; i < slashCount; i++)
{
lastSlash = basePath.LastIndexOf(Path.DirectorySeparatorChar, lastSlash - 1);
}
lastSlash++; // skip over slash
return (basePath.Substring(lastSlash, basePath.Length - lastSlash));
}

}

~jr
busterkeaton wrote on 8/5/2008, 5:20 PM
Wow, give that JohnnyRoy man a hand.


But on another note, you use folders named F822700453705C10045080046020224 ?
That would drive me insane
JohnnyRoy wrote on 8/5/2008, 7:20 PM
> But on another note, you use folders named F822700453705C10045080046020224 ?

LOL, XDCAM generates unique folder names because it uses redundant filenames. So that's an XDCAM "thingy". This is really a unique problem for XDCAM users.

~jr
erikd wrote on 8/5/2008, 10:08 PM
JR, thank you very much for taking time to help out someone you don't know for a problem you don't have. It is embarrasing to add on my part that I am not able to execute your code because of some type of operator error on my part.

I pasted the script into WordPad and saved as a .cs file in the "script menu" folder of Vegas Pro 8.0. I have verified that the "Fix Folders" script shows up in Tools/Scripting inside Vegas.

This is where I am not exactly sure how to proceed. What I am doing is opening the project with offline media without doing any relinking with Vegas. The timeline opens with "offline media" icons.

I have tried the following methods for executing the script.

Highlight a single clip on the timeline and run the script. A dialogue box comes up that says "0 files processed".

Highlight all clips on the timeline using Ctl-A and run the script. A dialogue box comes up that says "0 files processed".

Highlight all clips in a bin from the Project Media Tab and run the script. A dialogue box comes up that says "0 files processed".

This is painfully embarrasing for me to have to ask but may I ask what the correct procedure is for applying the script?

Erik

JohnnyRoy wrote on 8/6/2008, 6:30 AM
Erik, Don't thank me until it works. You are not someone I don't know... you are a member of our community and simply someone I haven't met yet. ;-)

Hmmm... all three ways should have worked because the script doesn't use the timeline at all. It sweeps the media pool so all you have to do is open the project, ignore all media and leave off-line, and then run the script.

It does assume that the files are under:

"E:\Import with Sony Vegas Proxy Library\04-08 Israel\"

From the looks of things, there was no off-line media in your project that used media from that folder. Currently, it will only work for files that are in that folder and also in your project. Are you saying that this doesn't work on the project from which you gave me the sample file names? Try it on that one first. You can change the basePath if you need to use it on other folders.

Do you need a more generic solution to work with several projects and folders that have files outside of ".\04-08 Israel\"? If so, we can work off-line on this. I'll build you a little GUI so you can select different starting points. I just thought you had this problem with one particular project and folder and I was only concerned with that.

~jr
erikd wrote on 8/6/2008, 7:06 AM
Johnny,

It turns out the operator error is with me as here is a cut/paste of the properties/media tab of a clip on the timeline that is showing up "offline".

E:\Import with Sony Vegas Proxy Library\61EF1F00463705C10045080046020224\Sub\C0050S01.MXF

I tried modifying the script by simply eliminating the 04-08 Israel part but that didn't work for me. The result I got was kind of strange in that a dialogue box came that appeared to want me to give a location to save an XML file.?

I do want to ask you about the idea you mentioned for creating a GUI for other jobs so I will contact you offline about that.

This would be a very handy for me and I think for other as well. Assuming XDCam is Sony's future we will have many people who need to deal with realities of large libraries of XDCam proxy footage. The reality being that hundreds of clips will have the same name and will need to be move around for organization purposes after the initial import. My 2 cents anyway.

I am willing to pay you a fee as well as I don't think you should work for free. Or maybe I could buy more VASST products? Whatever you think is appropiate.

Take care and thanks for warm invite into the community.

Erik


JohnnyRoy wrote on 8/6/2008, 9:33 AM
OK, I've made the script more generic without needing a GUI for you (although others might). It should now work with any project with MXF files under: "E:\Import with Sony Vegas Proxy Library\" regardless of how you renamed and moved the folders around and it should find them and resolve the off-line issue.

I updated my original post so please take this new version and let me know how it works.

~jr
erikd wrote on 8/6/2008, 10:44 AM
Gave it a try. This is the error that pops up when I try to run the script.

Sony Vegas Pro 8.0
An error occurred during the execution of the script c:\programfiles\sony\vegaspro 8.0\ScriptMenu|ResolveMXFMedia.cs

Error 0x80131600 (message missing)

C:\Program Files\Sony\Vegas Pro 8.0\Script Menu\ResolveMXFMedia.cs(1) : A namespace does not directly contain members such as fields or methods.


JohnnyRoy wrote on 8/6/2008, 11:46 AM
Since it is complaining about the first line in the file, it looks like you either didn't cut and paste the whole code block or the file is corrupt somehow. I will send you a copy via email.

~jr
erikd wrote on 8/6/2008, 12:20 PM
Are you supposed to copy the small words at the beginning that say "code block"? Maybe I included them when I did the cut and paste.
JohnnyRoy wrote on 8/6/2008, 12:24 PM
Good point. No, code block is added by the we site. the script starts with:

/**

and ends with

}

with everything in between.

~jr
erikd wrote on 8/6/2008, 12:45 PM
I have just witnessed a miracle! I have witnessed it! The code Johnny wrote has saved me countless hours and headaches! It is so elegant to watch. Click the script and wait a few seconds and you get a prompt that says something like 285 clips relinked! And then they gracefully light up my timeline! I am doing this in several different projects as well!

It works! It works!

Johnny for President!
JohnnyRoy wrote on 8/6/2008, 1:09 PM
That's great! I'm happy it worked. Sometimes a simple script can make all the difference. Glad I could help.

~jr
farss wrote on 8/6/2008, 2:25 PM
Excellent work by Johnny.
Can I just add my voice to what Erik said, this could be of benefit to others as well and those who need it would no doubt be willing to pay a reasonable fee.

Bob.