How to get just a changeset

[robe070]

ProcessCommandGetVersion gets every file that is in a Version. I just want the files that have changed in that Version. This takes far longer than I need it to take. I need it to be sub-second. Its currently over 13 seconds.

I wouldn't mind if the client-side cache is not used and all the files in the changeset come from the server. I suspect that would be quicker than reading approx 150 MB per second for 13 seconds from the cache! I've made a 2 byte change to a 10 k file!

How can this be done?

[Beth]

Getting to a non-working folder would rule out Vault using the cache.

[robe070]

If it doesn't use the cache it takes even longer!

I just need the changeset, not the whole version. There must be a way to just get the changeset.

Its like I need a Get Latest Version which rather than getting the latest version gets a specific version - comparing to whats there currently and only fetching those files that changed in the changeset since the last version was retrieved.

[Beth]

The Folder / Files will all have a remote date. You could try going through the list looking for a specific date.

Could run a "version" history query for the last X versions or perhaps a time. Then retrieve the history details for each history version, checking for changes.

In Vault 7 we will have a new feature that may help you that will get items changed since a particular label.

[robe070]

So by the looks of it that will give me a list of the files that have changed.

How do I retrieve those files from the Server?

[Beth]

You will then call a Get on the list of files that meet the criteria.

Try calling GetOperations.ProcessCommandGet() or GetOperations.ProcessCommandGetToLocationOutsideWorkingFolder() with an array of repository paths of the files that are wished to be retrieved.

Another option if you know the Transaction ID that created your folder's version, you can call ServerOperations.ProcessCommandTxDetail() and look at the item paths for the changeset item you wish to retrieve This would give you repository paths you could use above with the GetOperations.* methods.

[robe070]

But those APIs will get the latest version of the paths I specify. I need a specific version. So I'll need to use ProcessCommandGetVersion. For that though I'll need to know the Version of the File as opposed to the project. I presume theh hisotry info will provide the file's Version?

[robe070]

I've given it a go and it's worked until the first file that's deleted from the repository in the tip but was there in an earlier version. The exception message is:
No object was found at the repository path
And the path is $/VL/Trunk/SetProject.cmd at Version 1

The exception occurs when using GetOperations.ProcessCommandGetVersion.

I actually need to get the state at this time. How do I get it?

I took the GetOperations.cs file and compiled it into my code.

I have located the code in VaultClientIntegrationLib that's causing the issue. This line:

Code: Select all

				VaultClientTreeObject treeObjectToRetrieve = RepositoryUtil.FindVaultTreeObjectAtReposOrLocalPath(objectPath);

and this line (if comment out the previous)

Code: Select all

				vcFile = new VaultClientFile(ServerOperations.client.ClientInstance.TreeCache.Repository.Root.FindFileRecursive(treeObjectToRetrieve.FullPath));

I can see that these operations work on the latest repository view and do not cope with deleted objects. How can I search for deleted objects?

This is really worthwhile for our repository. Each Version is taking >20 seconds with old method and <1 sec with new - if I catch the exception and let it continue.

[jclausius]

If you call ServerOperations.client.ClientInstance.Refresh(), you're client side tree should be updated. For any files you wish to Get, you can find the parent folder - RepositoryUtil.FindVaultFolderAtReposOrLocalPath() OR ServerOperations.client.ClientInstance.TreeCache.Repository.Root.FindFolderRecursive() in the tree and see if that folder contains the file before doing the GET, or look up each path individually RepositoryUtil.FindVaultFileAtReposOrLocalPath() or ServerOperations.client.ClientInstance.TreeCache.Repository.Root.FindFileRecursive()

[robe070]

I don't actually need to find the file. I know its there at Version 1 because the history told me. I just need to be able to construct a VaultClientFile object for this version of the file. Then I would expect it to be able to return me the actual file from the Server. Its highly unlikely the file is in the cache, because its a very old version of the tree. I have to get it from the server.

I've got all the txn details from the history. Can that be used to construct a VaultClientFile? There isn't a constructor that can do it that I can see. And the elements I think I need to set are read-only.

Those APIs you listed. How do I specify to search for a Version of a File? They all seem to work on the latest view of the repository. This is the critical thing. FindFileRecursive will not find the file. But if I find something existing and set the Version correctly it will retrieve it as the contents of that folder. But I actually only know the verson at $ and the version of the file. I do not know the version of the parent folders so I can't use them.

What if the folder has also been removed in the latest view?

If I can just construct a VaultClientFile object for this file.

[jclausius]

If the file exists in the tree, then you can create a VaultClientFile(VaultClientFile fileFoundinTree), and then setting the Version/ObjVerID members of that instance to whatever version you wish to retrieve.

If the file does not exist, then things get more complicated unless you get a tree structure from history. ServerOperations.client.ClientInstance.Connection.GetBranchStructure() can do this.

While the access modifier doesn't allow it to be called, the source to VaultClientIntegrationLib.GetOperations.performGetVersion() should have some code for trying to get a file at a specific version as well as constructing a VaultClientFile for a specific version.

[robe070]

As I said at the beginning, this is all about deleted files - files that don't exist in the latest tree. And I've already modified GetOperations.cs, as I mentioned.

Do you have any examples of using ServerOperations.client.ClientInstance.Connection.GetBranchStructure?

The only example I can find is in GetOperations.cs:

Code: Select all

if (treeObjectToRetrieve is VaultClientFolder)
{
   VaultClientFolder vcFolder = (VaultClientFolder)treeObjectToRetrieve;
   vcFolder.Version = version;

   VaultFolderDelta vfDelta = new VaultFolderDelta();
   try
   {
   ServerOperations.client.ClientInstance.Connection.GetBranchStructure(ServerOperations.client.ClientInstance.ActiveRepositoryID, treeObjectToRetrieve.FullPath, vcFolder.ID, version, ref vfDelta, false);
   }

It requires vcFolder.ID. That requires a VaultClientFolder. That requires that the folder exists in the latest tree, which it doesn't. I actually want to pass a file name, not a folder. So its the file's ID that I require.

So how to I get this ID without requiring a VaultClientFolder? I cannot see it in VaultTxDetailHistoryItem.

I have tried this and it throws an exception:

Code: Select all

ServerOperations.client.ClientInstance.Connection.GetBranchStructure(ServerOperations.client.ClientInstance.ActiveRepositoryID, txdetailitem.ItemPath1, txdetailitem.OtherID, txdetailitem.Version, ref vfd, false);

Thats because its a file and not a folder.

It seems to be a catch-22. I am forced to recurse the directory structure from the top to get the changes for a file that no longer exists and for arbitrary folders that no longer exist.

[jclausius]

If you want the changes of files that have been modified by a particular transaction/changeset, you could just skip individual delete items within the changeset. ServerOperations.ProcessCommandVersionHistory() will get you some info. Using the Transaction ID (TxID) from that you can call client.ClientInstance.Connection.GetTxDetail. This should have all the changeset items for a given transaction. The VaultTxDetailHistoryItem.RequestType is represented by VaultLib.VaultRequestType. You could then skip deleted changeset items.

However, getting back to your original question, since you're dealing with historical trees within the repository, then you probably need to work with historical versions of a tree - client.ClientInstance.Connection.GetRepositoryStructure(int nRepID, long nSrcRevision, long nDestRevision, ref long nReturnDestRevision, VaultDateTime dtLastCheck, ref VaultDateTime dtLatestCheck, ref VaultRepositoryDelta rd)

You'll need to know your repository ID for nRepID. nSrcVersion and nDestRevision are the TxIDs of a "beginning tree" and "ending tree". Since you may not have a beginning, you can pass 0 (zero) for nSrcVersion, nReturnDestRevision needs to be passed in, but can be ignored. dtLastCheck can be a date/time set to a VaultDateTime min time, and the VaultRepositoryDelta needs to be a ref which can initially point to null.

After making the call, you'll notice the rd is a data structure containing a def'n of what the repository tree should look like. The main things to know if the VaultFSObjectDelta.Change member of each node will have a VaultLib.VaultDeltaChangeType. So, you can tell if the delta will results in files/folders that were modified, deleted, or added. Also, If you pass that data structure into a VaultClientFolder constructor, you'll have a full tree.

Since you are trying to detect if a folder/file exists, you'll need to use the Transaction ID right BEFORE a transaction, and the Transaction ID at that transaction. Then traversing this delta data structure should show you which files/folders were deleted.

I hope this is what you're looking for.

[robe070]

Thats close to what I need. Thanks for thinking about this. I had already started processing Delete, Rename, Move and Shared directly in the way you mention in the first paragraph. Though I actually perform the delete, etc directly myself. I delete the file. Thats sort of a side issue.

The main issue is actually the Checkins when that file no longer exists in the Latest Version. So checking in something that is later deleted. I can't bypass Vault, of course. I have to get the Version I require from the server.

I'm well past the effort my boss wants me to be spending on making this better. So I need a few more details to see if it will work for us and its just a matter of coding it.

After making the call, you'll notice the rd is a data structure containing a def'n of what the repository tree should look like. The main things to know if the VaultFSObjectDelta.Change member of each node will have a VaultLib.VaultDeltaChangeType. So, you can tell if the delta will results in files/folders that were modified, deleted, or added.

So, is this a way to get the Changeset rather than through GetTxDetail? Could you specify the previous TxnID and the current and will that result in the Changeset?

Also, If you pass that data structure into a VaultClientFolder constructor, you'll have a full tree

A full tree of what? The delta tree or the state of the entire set of files at that Version?

[jclausius]

So, is this a way to get the Changeset rather than through GetTxDetail? Could you specify the previous TxnID and the current and will that result in the Changeset?

No. Not really. With a little bit of coding, GetRepositoryStructure() can show you is "how" the tree was affected -> which files are moved, modified, renamed, etc. It does not explain the "why". For example, a MOVE operation will show up as an ADD/DELETE pair in the repository delta. However it does not explain WHY the ADD/DELETE occur. But that can be seen in GetTxDetail.

A full tree of what? The delta tree or the state of the entire set of files at that Version?

I slightly misstated this, but meant the latter. You can apply a repository delta to an existing Repository (not a VaultClientFolder) with "Update()", and that will build a data structure of what the repository would look like at a given point in time.