Sporadic failures with command line GET from Vault.exe

[jvel]

We are a long time user of Vault - we're currently using 4.1.1 (I've bought 5.1 licences but yet upgraded). We have about 10 users plus a machine running continuous integration builds using CCNet and NAnt. The CI build machine is sporadically failing when attempting GET or GETVERSION operations.

The error is "System.Exception: The connection to the server failed: server cannot be contacted or uses a protocol that is not supported by this client. Server was unable to process request."

We're simply using a NAnt EXEC task, calling Vault.exe, not using any built-in or plug-in CCNet/NAnt features. The NAnt script does a couple of GETs and sometimes it's the first that fails and sometime's it's the second.

I've set LogLevel to Debug in the Vault config file. I can't see anything in the sgvault.log which resembles a failure. (I can send you the log if necessary.)

Please can you tell me how to troubleshoot this. Could it be the server is not responsive enough, and if so are there any settings that I can tweak? (I mean the Vault server - I guess the SQL Server is behaving otherwise I'd expect to see errors in the Vault log).

Thanks.
John Vel, Arden Software Ltd, UK

[lbauer]

Is this a new problem? Did it used to work with no problems?

Are you connecting to the Vault Server via http or https?

[jvel]

This is a newish problem - I guess it's been happening for two or three months. The build scripts haven't changed much in the last year or so, except that we're often changing the branches that we're building.

We're not using SSL. Would that be better? I don't think we've made an informed choice there, we've probably just gone with the defaults.

My next reply contains the relevant part of the NAnt log, including the Vault call stack, in case it's useful.

Our Vault server is running Windows Server 2003 R2 Standard x64 Edition Service Pack 2, with 4GB RAM.
Our database server is running Windows Server 2003 R2 Standard Service Pack 2 (32bit), with 4GB RAM. The database is on SQL Server 2005 (which is hosting a few other databases too).

[jvel]

vault-getversion:

[echo] GETVERSION -host soft05.arden.group -user build -password ****** -repository Impact 24226 "$/Impact Source/Developer Branches/John/47825/" "D:\Build Area\Impact John 47825\Build"
[exec]
<vault>
[exec] <result>
[exec] <success>True</success>
[exec] </result>
[exec] </vault>

BUILD SUCCEEDED

Total time: 855.3 seconds.

[nant] D:\Build Area\Impact John 47825\Scripts\vaultoperations.build vault-get
Buildfile: file:///D:/Build Area/Impact John 47825/Scripts/vaultoperations.build
Target framework: Microsoft .NET Framework 2.0
Target(s) specified: vault-get


vault-get:

[echo] GET -host soft05.arden.group -user build -password ****** -repository Impact -destpath "D:\Build Area\Impact John 47825\Build\Support Files" "$/IDE/Dev/Support Files"
[exec]
<vault>
[exec] <error>
[exec] <exception>System.Exception: The connection to the server failed: server cannot be contacted or uses a protocol that is not supported by this client. Server was unable to process request. ---> Object reference not set to an instance of an object. ---> System.Web.Services.Protocols.SoapException: Server was unable to process request. ---> Object reference not set to an instance of an object.
[exec] at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)
[exec] at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
[exec] at VaultClientNetLib.ClientService.VaultService.Login(String strHostname, Boolean bUseFullFiles, String username, String strEncryptedPassword, String strRMKey, String& strAuthToken)
[exec] at VaultClientNetLib.VaultConnection.Login(String strURLBase, String strUserLogin, String strPassword)
[exec] at VaultClientOperationsLib.ClientInstance.Login(String urlbase, String username, String password)
[exec] at VaultClientIntegrationLib.ServerOperations.Login(AccessLevelType altCommand, Boolean bAllowAuto, Boolean bSaveSession)
[exec] --- End of inner exception stack trace ---
[exec] at VaultClientIntegrationLib.ServerOperations.Login(AccessLevelType altCommand, Boolean bAllowAuto, Boolean bSaveSession)
[exec] at VaultClientIntegrationLib.ServerOperations.Login()
[exec] at VaultCmdLineClient.VaultCmdLineClient.ProcessCommand(Args curArg)
[exec] at VaultCmdLineClient.VaultCmdLineClient.Main(String[] args)</exception>
[exec] </error>
[exec] <result>
[exec] <success>False</success>
[exec] </result>
[exec] </vault>

BUILD FAILED

[lbauer]

We'd like to see a copy of the Vault Server log that covers the time of this failure. It's called sgvault.log and is in %windir%\temp\sgvault on the server machine. Send the logs zipped up to support at sourcegear.com, Attn: Linda. Please include a link to this forum post.

HS:222547

[jvel]

If any readers are wondering how this turned out in the end, Linda said "Most likely there are some sporadic network or connectivity issues that caused the build to lose contact with the Vault Server" but our networking/server guys are yet to be convinced. My next step will be to update all the drivers at both ends and see if that makes any difference.

[BSayatovic]

Did you ever reach a conclusion. We've been having this with years. We have dozens of servers in our environment (development sites, SharePoint, SQL servers, FogBugz server, CI servers, etc.) and the ONLY server that ever experiences these interruptions is Vault, so I, too, doubt the network explanation.

[lbauer]

We didn't hear back from the user, so not sure if they found any network issues.

If you're experiencing problems, we'd like to see a copy of the Vault Server log and a corresponding client log.

This KB article describes logging for the various Vault Clients.
http://support.sourcegear.com/viewtopic.php?t=1534

Once you've reproduced the error, send the logs zipped up to support at sourcegear.com, Attn: Linda. Please include a link to this forum post.