Calls To Add and Rename Take a Long/Are Too Slow

[splatteredbits]

When I make calls to ProcessCommandRename or ProcessCommandAdd, it can take up to a minute per call for the call to return. These same calls via the command line (e.g. vault.exe add and vault.exe rename) also take up to 60 seconds per file being added renamed.

My code is pretty simple:

Code: Select all

	ServerOperations.ProcessCommandAdd(parentDir, files);

and

Code: Select all

	ServerOperations.ProcessCommandRename(vaultPath, newName);

What can I do to speed this up?

[jeremy_sg]

To enable logging, put this code at the start of your program:

Code: Select all

VaultLib.SimpleLogger.Log.EnableClass("all");
VaultLib.SimpleLogger.Log.Enabled = true;
VaultLib.SimpleLogger.Log.LogFilePath = "c:\\logfile.txt"

Looking at the log file will help determine where the slowdown is happening.

[splatteredbits]

Here is the log file. There appears to be a lot of mutex action going on.

[jeremy_sg]

Wow, you're right it's taking a very long time. During the long stretch of <wf> and <mutex> messages, it's scanning all of your working folders for edited files. There are a couple of things you can do to avoid this working folder scan (which happens in the background in the GUI client)

1. Set less working folders. It looks like you have $/ mapped to C:\Build. If you set the workingfolders lower down, you will be able to avoid scanning those directories.
2. Set the "Require Checkout before Checkin" option for the user.

[splatteredbits]

Neither of these are viable solutions for me. I can only move my working directory one level down, as below $/ are all our branches (e.g., $/Current, $/Jul09, etc.). As for working in Check-out mode, well, we don't use Visual Studio integration so I would have to check-out each file by hand, in the Vault GUI. This is unworkable.

I don't understand why vault.exe scans my entire working directory; I don't ask it to. These are the commands I'm issuing:

Code: Select all

// I'm passing in a *specific* list of files to see if there are any pending updates on them
var changeSet = ServerOperations.ProcessCommandListChangeSet(pathsToFiles);

// ...snip...

// checking if a specific file exists in Vault or not
ServerOperations.ProcessCommandListObjectProperties(path);

// ...snip...

// adding specific items to Vault
ServerOperations.ProcessCommandAdd(hashItem.Key, files);

[jeremy_sg]

I've got a temp build which updates only the items that you're requesting in the ListChangeSet. Please email support at sourcegear.com ATTN: jeremy with a link to this thread, and I'll send it to you.

[splatteredbits]

The ProcessCommandListChangeSet command now works much faster. I'm using it to see if an item has already been added, so I have to call ListChangeSet with the parent directory of the file being added. It also throws an exception when passed a file that doesn't exist in the repository whereas before I think it returned an empty change set.

But the ProcessCommandAdd still looks like it does a full repository scan. I've attached a new log file.

[jeremy_sg]

Please add this line to the above logging setup lines:

VaultLib.SimpleLogger.Log.IncludeStackTraces = true;

This will generate a huge log file, but will help me track this down better. Thanks.

[splatteredbits]

Here you go. File was too big, so I had to split it in half.

[splatteredbits]

I'm now having a hard time committing add files, both via the command line and the API. These are my command line invocations:

Code: Select all

> vault.exe add -host vault -repository Repo c:\WorkingFolder c:\WorkingFolder\d69e28dc-c58b-487c-a83c-e118f46ee2c4.txt

> vault.exe commit -host vault -repository Repo c:\WorkingFolder\d69e28dc-c58b-487c-a83c-e118f46ee2c4.txt

The commit fails with the following output:

Code: Select all

<vault>
  <error>
    <exception>System.Exception: No object was found at the repository path: $/d69e28dc-c58b-487c-a83c-e118f46ee2c4.txt
   at VaultClientIntegrationLib.RepositoryUtil.FindVaultTreeObjectAtReposOrLocalPath(String testPath)
   at VaultClientIntegrationLib.RepositoryUtil.MatchWildcardToTreeObjects(String repositoryPath)
   at VaultClientIntegrationLib.ServerOperations.ProcessCommandCommit(String[] objectPaths, UnchangedHandler unchanged, Boolean keepCheckedOut, LocalCopyType localCopy, Boolean resolveMerge, ChangeSetItemColl& csicRemove)
   at VaultCmdLineClient.VaultCmdLineClient.ProcessCommand(Args curArg)
   at VaultCmdLineClient.VaultCmdLineClient.Main(String[] args)</exception>
  </error>
  <result>
    <success>False</success>
  </result>
</vault>

If I commit on the parent directory, however, the commit succeeds:

Code: Select all

> vault.exe commit -host vault -repository core C:\WorkingFolder\

Am I not allowed to commit a specific file? Must I always commit a directory? If I can commit a specific file, what is the proper incantation to do so?

[jeremy_sg]

Add files are a special case, since there is no object at the repository path yet. Probably the easiest thing to do is to add the -commit flag to your vault.exe add command. That will commit the add file immediately.

[splatteredbits]

What about via the API? I don't see any flags on the ProcessCommandAdd that allows me to auto-commit.

Code: Select all

ServerOperations.ProcessCommandAdd(@"c:\WorkingFolder", new string[] { @"c:\WorkingFolder\d69e28dc-c58b-487c-a83c-e118f46ee2c4.txt" })

ServerOperations.ProcessCommandCheckIn(new string[] { @"c:\WorkingFolder\d69e28dc-c58b-487c-a83c-e118f46ee2c4.txt" }, UnchangedHandler.UndoCheckout, false, LocalCopyType.Replace, false);

The second call fails with the same error: unable to find path in repository.

I would prefer to not auto-commit additions. In my case, I have a bunch of modifications, deletions and additions that should get committed/checked-in together.

[jeremy_sg]

In the API, every ProcessCommand* that could result in a commit (if the ServerOperations.Client.AutoCommit flag is set to true) returns a ChangeSetItemColl collection. You can commit that collection later (with ProcessCommandCommit) or add the items to a ChangeSetItemColl that you keep, and then commit all at once.

[splatteredbits]

I've re-written my commit logic to commit a specific directory. But when I do that, my commit fails with a generic System.Exception: Commit failed message.

Here are the two calls I'm making:

Code: Select all

ServerOperations.ProcessCommandAdd(@"C:\WorkingDirectory\", new string[] { @"C:\WorkingDirectory\newfile.txt" } );

// ...snip...

ChangeSetItemColl committedItems;
ServerOperations.ProcessCommandCommit(new[] {@"C:\WorkingDirectory" }, UnchangedHandler.UndoCheckout, false,		                                      LocalCopyType.Replace, false, out committedItems);

This results in the following exception:

Code: Select all

System.Exception: Commit failed
at VaultClientIntegrationLib.ServerOperations.ProcessCommandCommit(ChangeSetItemColl csic, UnchangedHandler unchanged, Boolean keepCheckedOut, LocalCopyType localCopy, Boolean removeLocalCopy, Boolean resolveMerge, ref ChangeSetItemColl csicRemove)
at VaultClientIntegrationLib.ServerOperations.ProcessCommandCommit(String[] objectPaths, UnchangedHandler unchanged, Boolean keepCheckedOut, LocalCopyType localCopy, Boolean resolveMerge, ref ChangeSetItemColl csicRemove)
at WebMD.Vault.Facade.VaultFacade.Commit(String pathToFolder) in VaultFacade.cs: line 34
at WebMD.Vault.Facade.Tests.VaultFacadeTests.Commit() in VaultFacadeTests.cs: line 99 

In addition, it looks like the call to commit is also doing a scan of the entire working directory. Should it be doing this if I give it a specific directory of changes to commit? I've attached a log file for both the exception and entire working directory scan.

[jeremy_sg]

The code that I was imagining goes something like this:

Code: Select all

ChangeSetItemColl mycollection = new ChangeSetItemColl();

ChangeSetItemColl temp = ProcessCommandAdd(...);
foreach (ChangeSetItem item in temp)
   mycollection.Add(item);

//Pend the second add.
temp = ProcessCommandAdd(...);
foreach (ChangeSetItem item in temp)
   mycollection.Add(item);

ServerOperations.ProcessCommandCommit(mycollection,....);

The ProcessCommandCommit does not do a full working folder scan. It only refreshes the items that it already knows about.

If you want information on why the commit failed, you need to add a listener to the ServerOperations.UserMessage event.