Problem Accessing Database (FailDBConn)

[nattyBoy]

Hello,

We recently encountered problems accessing Vault on our server, giving the message:
"The Vault Server could not establish a connection to the database. Please check the server's log for detailed information. (FailDBConn)" From the Client.

Nothing on the server has changed apart from some windows updates being installed (including Service Pack 1) and the server being restarted. Sql Server (2000) is on the same server as the Web Service and is setup to use mixed authentication. IIS is parsing the default IIS user (SERVERNAME_IUSR) to SQL and that user has DBO permissions on the SGVault database.

Strangely there are no errors logged in either the Windows Event Viewer or Vault Error Log.

We are at a complete loss, we do not know where to look for the cause, any help much appreciated!!

[Beth]

It could be a case here of where something changed that you didn't expect.

Try this first:
Run iisreset.exe from a command line. (per this article: http://support.sourcegear.com/viewtopic.php?t=5690)

[nattyBoy]

I've tried iisreset and they had no effect. We've even now tried a complete reboot of the server, and that had no effect either.

[Beth]

It would appear that something may have changed in the sgvaultuser. The easiest way to fix everything would be to reinstall the Vault server, or you can check through the following.

Here are some things to check and do:

• 1) check the web.config file and make sure the server is pointing to the correct sql server host name
  2) drop the old sgvaultuser database user from sgvault database.

if using sql server authentication:

• 1) create new sgvaultuser login - the password MUST be your vault admin password (to match web.config)
  2) create a new sgvaultuser database user for the sql login
  3) grant db access to sgvaultuser; add the role member db_owner

if using windows authentication:

• 1) you will need to know the asp.net process configuration. for example, in iis 5 (assuming the process model is using the default ASPNET windows account), you would need to grant IISMACHINEHOSTNAME\ASPNET access to the database.
  2) create a new database user for the windows login in step 1
  3) grant db access to that account; add the role member db_owner

[Beth]

For further looking into this if needed, you can feel free to send a Vault server log to support at sourcegear.com (attn: Beth) and reference this post.

[nattyBoy]

The easiest way to fix everything would be to reinstall the Vault server

If I reinstall the Vault server, will the new installation automatically link to the existing repositories?

We are using Windows Auth, so for now I will look into re-creating the login in the database...

Your help is much appreciated

[Beth]

As long as you save your database, all the information that was in Vault previously would still be there.

[nattyBoy]

I've reinstalled Vault, and managed to keep the database. I get a different error now, which I think is a good thing. Actually I get two errors. The first time I access vault (through the client / admin tool) I get this:

Collection cannot be null.
Parameter name: c

After I've recieved that error once, I get this on subsequent attempts:

System.Web.Services.Protocols.SoapException: Server was unable to process request. ---> System.IndexOutOfRangeException: Index was outside the bounds of the array.
at VaultServiceAPILib.VaultServiceAPISystem.GetPublicKey(Byte[]& publicKey, Byte[]& exponent)
at VaultService.VaultAdminService.GetPublicKey(Byte[]& publicKey, Byte[]& exponent)
--- End of inner exception stack trace ---

Wierd things is the Windows\Temp\SGVault folder still has no new logs in it.

[Beth]

From here I think you can do one of the following:

1. Detail out for me how you performed the uninstall and reinstall. Send me a copy of the installation log. The vault_install.log file is located in the windows temp directory (%temp%).

2. Perform another uninstall/reinstall but follow the following steps:

• a) Uninstall - keep the vault database when prompted
  
  b) Manually inspect IIS for VaultService / VaultShadowFolder virtual
  directories. Delete them if found.
  
  c) Inspect the Vault installation directory
  (%SYSTEMROO%\Inetpub\wwwroot\VaultService) make sure it is empty.
  
  d) Delete %windir%\temp\sgvault
  
  e) Reinstall - keep the database when prompted. Use the defaults for
  .Net Process Model, unless there is a need to change.

[nattyBoy]

1. I Unistalled Vault, keeping the database. Reinstalled Vault, set the IIS process model to use the System account (DOMAIN\SERVER$) and set it to use the existing database. I did a search on the WINDOWS directory and could not find any files named "vault_install.log".

2. I decided to do this aswell, this time doing the following:
a) Uninstalled, keeping the database
b) Inspected IIS, finding no evidence of vaultservice or vaultshadowfolder virtual directories
c) There was no evidence of Vault folders in wwwroot folder
d) Deleted the WINDOWS\temp\sgvault folder
e) Reinstalled keeping the database, and this time going with the default IISProcess Model (NT AUTHORITY\NETWORK SERVICE).

Tried to access Vault throught the admin tool, first time I get this:
Collection cannot be null.
Parameter name: c

Second Time I get this:
The Vault Server could not establish a connection to the database. Please check the server's log for detailed information. (FailDBConn)

[Beth]

Can you send your Vault Server Log to me using either the private function here or sending it to support at sourcegear.com (attn: Beth) with a link to this thread?

Is your SQL server and your Vault server on the same or separate machines?

[Beth]

Well, you're in luck. I somehow accidentally messed up my own server and came up with the exact same scenario. Unfortunately, I can't tell you exactly what caused it except that I was poking around in IIS, and I think I changed something in the Vault App Pool.

What finally got my server up and running again was:
1. Completely uninstall Vault but keep the SQL server.
2. Remove the Vault Cache.
3. Uninstall and reinstall the 1.1 .NET framework. At a command prompt, type first C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\aspnet_regiis.exe -u, and then type C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\aspnet_regiis.exe -i.
4. Install Vault (From here there was still an IIS issue)
5. Go to the IIS Manager, open the properties page for the Vault service in it's website.
6. On the Virtual Directory tab, I removed and created the application name.
7. On the ASP.NET tab, I set the version to the 1.1 .NET framework and clicked Apply. (This won't prevent you from programming for the 2.0 .NET framework.

Then at last I could get into Vault.

[nattyBoy]

Hi Beth,

Thankyou for your post, but I'm afraid that didn't help either... What we've done now instead, is to setup a new instance of Sourcegear on a different server and re-insert our projects into source safe. We are considering the broken server to be unrecoverable now so you need not investigate this further.

Regarding your post before last, there are still no vault logs or event viewer logs relating to vault at all.

Your assistance in trying to solve this problem has been much appreciated.

[Beth]

I did find that my server stopped logging right at the point where I made some change to my App Pool. Hopefully your new server will be able to perform logging for you.

[Beth]

An update on this one on another item I found that can cause the Collection Cannot Be Null error. If the website location was moved without telling IIS where it went, this error will occur. (Yes, I made it happen twice now.)

In my case, I had Vault under Default Website, but when I would click Default Website when in the IIS manager, I would get the error “The system cannot find the path specified.” I had removed the drive that it was originally configured for, so that drive no longer existed.

I went into the properties page for the Default Website and went to the Home Directory tab and changed the home directory to a location that existed.