Problems with an automated Vault client

[Igor.Samoylenko]

Hi,

I've been working on an automated service to do our internal builds. It is a simple custom build web service that gets the source from Vault first via API and then executes the tasks in turn.

The version of Vault I am using is v4.1.2. I am having concurrency problems: the web service sets a label first and then gets the source into a local subdirectory using:

Code: Select all

GetOperations.ProcessCommandGetLabelToLocationOutsideWorkingFolder(vaultFolderPath, labelName, Nothing, getOptions, localFolderPath)

The problem is when this code executes concurrently (i.e. trying to get another label locally at the same time), the second thread throws an exception:

Code: Select all

VaultClientOperationsLib.GetLatestVersionFailedException: Exception of type 'VaultClientOperationsLib.GetLatestVersionFailedException' was thrown.
   at VaultClientOperationsLib.ClientInstance.ProcessGetFileRequests(GetFileInfo[] infos, MakeWritableType makeWritable, SetFileTimeType setFileTime, MergeType merge, Boolean updateHiddenFilesOnly, String ancestorFullPath, Boolean flat, String ancestorDiskPath, OverwritePrompt PromptData, Boolean isLabelGet, String currentPathToLabelItem, Int64 labelID, Boolean isRetry, Boolean isGetByDisplayVersion)
   at VaultClientOperationsLib.ClientInstance.GetByLabelToNonWorkingFolder_GetData(VaultClientFile[] files, Boolean overwrite, MakeWritableType makeWritable, SetFileTimeType setFileTime, String ancestorFullPath, String ancestorDiskPath, OverwritePrompt PromptData, Int64 labelID, String currentPathToLabelItem, String labelSubItemPath, Boolean isLabelGetForSingleFile)
   at VaultClientOperationsLib.ClientInstance.GetByLabelToNonWorkingFolder_GetData(VaultClientFolder folder, Boolean recursive, Boolean overwrite, MakeWritableType makeWritable, SetFileTimeType setFileTime, String destinationDiskPath, OverwritePrompt PromptData, Int64 labelID, String currentPathToLabelItem, String labelSubItemPath)
   at VaultClientIntegrationLib.GetOperations.performLabelGet(String objectPath, String label, String labelSubItem, String labelWorkingFolder, String destPath, GetOptions go)
   at VaultClientIntegrationLib.GetOperations.ProcessCommandGetLabelToLocationOutsideWorkingFolder(String objectPath, String label, String labelSubItem, GetOptions getOptions, String destPath)

I am looking at the way web service handles the connections to Vault as the possible reason. At the moment, before executing a task, the service does this:

Code: Select all

        If (Not ServerOperations.isConnected) Then
            ServerOperations.client.LoginOptions.URL = ...
            ServerOperations.client.LoginOptions.User = ...
            ServerOperations.client.LoginOptions.Password = ...
            ServerOperations.client.LoginOptions.Repository = ...
            ServerOperations.Login(VaultConnection.AccessLevelType.Client, true, true)
        End If

So the two concurrent threads will share the same connection right? So calling ProcessCommandGetLabelToLocationOutsideWorkingFolder() concurrently on the same connection does not work.

How else can I do it? How would you recommend handling Vault connections generally? Maintain a pool and allocate to each thread? What is the most optimal way to do it for me (performance etc)?

Also, what do "bAllowAuto", "bSaveSession" parameters mean in the call to Login? What are their defaults (in Login())? Also, can you explain what VaultConnection.AccessLevelType options mean, especially when used in a web service?

Thanks a lot.

[shannon]

Have you looked at CruiseControl.Net? We have a plugin that works with CruiseControl.Net and handles both label and get.

Can you set up some event handlers to determine what the specific errors are? You'd do it like this:

After login:

Code: Select all

ServerOperations.client.ClientInstance.EventEngine.addListener(ServerOperations.client, typeof(MessageEvent));
ServerOperations.client.ClientInstance.EventEngine.addListener(ServerOperations.client, typeof(BulkMessageEvent));

Then implement these two methods to output the errors somewhere:

Code: Select all

public void HandleEvent(MessageEvent e) {
     if (e.Message.Level == ProgressMessage.MessageLevel.Error)
         // output error here
}

public void HandleEvent(BulkMessageEvent e) {
     // same as above, but loop over e.Messages
}

[Igor.Samoylenko]

Yes I am looking at CruiseControl.NET but for now I would like to get our service to work for now.

The error I get is this (in Sub HandleEvent(ByVal e As MessageEvent)):

Code: Select all

An exception was encountered during the get latest operation.  
Exception: VaultServiceAPILib.VaultSoapException: 
2100 : FailExistingDownload    at VaultService.VaultService.BeginDownloadLabelFiles(Int32 nRepID, String strPathToLabelItem, Int64 nLabelID, VaultRequestGetFile[]& requests, String& strDownloadID)

[shannon]

ServerOperations only has one client connection. The server will only handle one download request per client connection. You can either use the old api style or implement some locking on your own to prevent two simultaneous downloads.

For some of your Login questions:
--Login() calls Login(VaultConnection.AccessLevelType.Client, true, false);
--bAllowAuto determines whether we try to retrieve connection parameters from a file (if they were stored previously) when the connection parameters haven't been provided
--bSaveSession determines whether we write the connection parameters (username, password, etc) to a file and use them for future connections if no parameters are provided. (The rememberlogin command on the clc is the best example of this.)
--Unless you need to be doing something that only an Admin can do (i.e. obliterate) the only access level you should ever need is Client

[Igor.Samoylenko]

Thanks a lot for your quick (as usual) response. Now that I know what is going on, I will see what I can do to fix it.