Vault server restarting every 2-5 minutes.

mamacker · Post by **mamacker** » Wed Dec 21, 2005 10:52 am

Hi there,
We've been having trouble with our vault server constantly restarting every 2-5 minutes. I've dug through a bunch of entries on the Knowledgebase and changed all the IIS settings I could find that cause timeouts.

My questions:
When IIS restarts the service is there any indication in a log somewhere as to why it thought it needed to restart the service?
Do http keep-alives have anything to do with vault connections?
What is the latest kb article describing all the IIS settings and what they should be?

My setup is completely under my control the IIS server is in house, and its a pretty basic install. I do have sharepoint installed, but I cannot tell how that would affect my Vault installation. I only mention it because it is mentioned in a few other kb posts.

Thanks in advance for the help.

lbauer · Post by **lbauer** » Wed Dec 21, 2005 12:54 pm

Some ideas --

Is this new behavior? Was Vault *not* restarting frequently before?

What operating system is the Vault Server installed on? If Windows 2003 with IIS 6, have you seen this KB article about recycling?

http://support.sourcegear.com/viewtopic.php?t=1014

What is your Vault session timeout in the Admin Tool->Server Options?

Check the Windows Event Viewer on the server machine. Are there any messages regarding IIS restarting or the ASPNET process being recycled?

Is IIS restarting? Check the IIS log, default location: %WinDir%\System32\LogFiles

mamacker · Post by **mamacker** » Wed Dec 21, 2005 1:41 pm

Hi!
Thanks for the quick respose. I had seen that article and just now double checked all the settings listed.
I am running Windows 2003 with IIS 6.

It appears that we started seeing the trouble five days after starting up the service. To see if it was just a crufty run of windows I rebooted the server machine but the vault reboots happened again after 10 minutes of uptime.

Here is a snippet of the debug log to see what Vault is going through. At the time of this log I turned off all clients to the vault server(I think).:

----12/21/2005 10:33:28 AM sgvaultsystem--()--
System Started
Version 3.1.5.3546
Cache Level = 1
DataBase Buffer Size (KB) = 256
LogFile Path = C:\WINDOWS\Temp\sgvault
Log Level = Debug
Archive Log = Weekly
ReverseDNS Lookup = True
Maximum HTTP Request Length = 102400
Overwrite Log on Startup = False
Session Timeout = 4320
Active Directory Domain =
SGVault Working Directory = C:\WINDOWS\Temp
SGVault Server URL = http://toad/VaultService
Identity = NT AUTHORITY\NETWORK SERVICE
----12/21/2005 10:43:34 AM sgvaultsystem--()--
System Started
Version 3.1.5.3546
Cache Level = 1
DataBase Buffer Size (KB) = 256
LogFile Path = C:\WINDOWS\Temp\sgvault
Log Level = Debug
Archive Log = Weekly
ReverseDNS Lookup = True
Maximum HTTP Request Length = 102400
Overwrite Log on Startup = False
Session Timeout = 4320
Active Directory Domain =
SGVault Working Directory = C:\WINDOWS\Temp
SGVault Server URL = http://toad/VaultService
Identity = NT AUTHORITY\NETWORK SERVICE
----12/21/2005 10:53:33 AM sgvaultsystem--()--
System Started
Version 3.1.5.3546
Cache Level = 1
DataBase Buffer Size (KB) = 256
LogFile Path = C:\WINDOWS\Temp\sgvault
Log Level = Debug
Archive Log = Weekly
ReverseDNS Lookup = True
Maximum HTTP Request Length = 102400
Overwrite Log on Startup = False
Session Timeout = 4320
Active Directory Domain =
SGVault Working Directory = C:\WINDOWS\Temp
SGVault Server URL = http://toad/VaultService
Identity = NT AUTHORITY\NETWORK SERVICE

The logs from IIS, at this time, look like this:

2005-12-21 18:33:26 W3SVC1 192.168.50.220 POST /VaultService/VaultService.asmx - 80 - 64.203.90.133 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+2.0.50727.42) 500 0 0
2005-12-21 18:33:35 W3SVC1 192.168.50.220 POST /VaultService/VaultService.asmx - 80 - 73.235.83.164 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+2.0.50727.26) 500 0 0
2005-12-21 18:34:26 W3SVC1 192.168.50.220 POST /VaultService/VaultService.asmx - 80 - 64.203.90.133 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+2.0.50727.42) 500 0 0
2005-12-21 18:35:26 W3SVC1 192.168.50.220 POST /VaultService/VaultService.asmx - 80 - 64.203.90.133 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+2.0.50727.42) 500 0 0
2005-12-21 18:36:26 W3SVC1 192.168.50.220 POST /VaultService/VaultService.asmx - 80 - 64.203.90.133 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+2.0.50727.42) 500 0 0
2005-12-21 18:37:26 W3SVC1 192.168.50.220 POST /VaultService/VaultService.asmx - 80 - 64.203.90.133 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+2.0.50727.42) 500 0 0
2005-12-21 18:38:26 W3SVC1 192.168.50.220 POST /VaultService/VaultService.asmx - 80 - 64.203.90.133 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+2.0.50727.42) 500 0 0
2005-12-21 18:39:26 W3SVC1 192.168.50.220 POST /VaultService/VaultService.asmx - 80 - 64.203.90.133 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+2.0.50727.42) 500 0 0
2005-12-21 18:40:26 W3SVC1 192.168.50.220 POST /VaultService/VaultService.asmx - 80 - 64.203.90.133 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+2.0.50727.42) 500 0 0
2005-12-21 18:41:26 W3SVC1 192.168.50.220 POST /VaultService/VaultService.asmx - 80 - 64.203.90.133 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+2.0.50727.42) 500 0 0
2005-12-21 18:42:26 W3SVC1 192.168.50.220 POST /VaultService/VaultService.asmx - 80 - 64.203.90.133 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+2.0.50727.42) 500 0 0
2005-12-21 18:43:31 W3SVC1 192.168.50.220 POST /VaultService/VaultService.asmx - 80 - 64.203.90.133 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+2.0.50727.42) 500 0 0
2005-12-21 18:43:45 W3SVC1 192.168.50.220 POST /VaultService/VaultService.asmx - 80 - 73.235.83.164 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+2.0.50727.26) 500 0 0
2005-12-21 18:44:31 W3SVC1 192.168.50.220 POST /VaultService/VaultService.asmx - 80 - 64.203.90.133 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+2.0.50727.42) 500 0 0
2005-12-21 18:45:31 W3SVC1 192.168.50.220 POST /VaultService/VaultService.asmx - 80 - 64.203.90.133 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+2.0.50727.42) 500 0 0
2005-12-21 18:46:31 W3SVC1 192.168.50.220 POST /VaultService/VaultService.asmx - 80 - 64.203.90.133 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+2.0.50727.42) 500 0 0
2005-12-21 18:47:31 W3SVC1 192.168.50.220 POST /VaultService/VaultService.asmx - 80 - 64.203.90.133 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+2.0.50727.42) 500 0 0
2005-12-21 18:48:31 W3SVC1 192.168.50.220 POST /VaultService/VaultService.asmx - 80 - 64.203.90.133 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+2.0.50727.42) 500 0 0
2005-12-21 18:49:31 W3SVC1 192.168.50.220 POST /VaultService/VaultService.asmx - 80 - 64.203.90.133 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+2.0.50727.42) 500 0 0
2005-12-21 18:50:31 W3SVC1 192.168.50.220 POST /VaultService/VaultService.asmx - 80 - 64.203.90.133 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+2.0.50727.42) 500 0 0
2005-12-21 18:51:31 W3SVC1 192.168.50.220 POST /VaultService/VaultService.asmx - 80 - 64.203.90.133 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+2.0.50727.42) 500 0 0
2005-12-21 18:52:31 W3SVC1 192.168.50.220 POST /VaultService/VaultService.asmx - 80 - 64.203.90.133 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+2.0.50727.42) 500 0 0
2005-12-21 18:53:31 W3SVC1 192.168.50.220 POST /VaultService/VaultService.asmx - 80 - 64.203.90.133 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+2.0.50727.42) 500 0 0
2005-12-21 18:53:40 W3SVC1 192.168.50.220 POST /VaultService/VaultService.asmx - 80 - 73.235.83.164 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+2.0.50727.26) 500 0 0
2005-12-21 18:54:31 W3SVC1 192.168.50.220 POST /VaultService/VaultService.asmx - 80 - 64.203.90.133 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+2.0.50727.42) 500 0 0

From HTTPERR:

2005-12-21 18:21:18 85.244.2.252 45640 192.168.50.220 80 - - - - - Timer_ConnectionIdle -
2005-12-21 18:21:38 85.244.2.252 46433 192.168.50.220 80 HTTP/1.1 POST /wordpress/xmlrpc.php - 1 Timer_ConnectionIdle DefaultAppPool
2005-12-21 19:16:31 64.203.90.133 59353 192.168.50.220 80 HTTP/1.1 POST /VaultService/VaultService.asmx 503 1 N/A VaultAppPool
2005-12-21 19:17:31 64.203.90.133 62307 192.168.50.220 80 HTTP/1.1 POST /VaultService/VaultService.asmx 503 1 N/A VaultAppPool

Thanks again for the help. Any further thoughts?

Last Note: I can send my complete log file to a person but I cannot post it to the public knowledge base.

lbauer · Post by **lbauer** » Wed Dec 21, 2005 5:32 pm

You can email the log (or part of it) to linda at sourcegear.com.

Did you check your Windows Event viewer for any events related to the times of the Vault Server restart?

Also, send me the web.config file from the VaultServer directory on the Vault Server machine.

mamacker · Post by **mamacker** » Wed Dec 21, 2005 5:37 pm

I did check the event logs and there does not appear to be anything relating to IIS anything.

I'll send you the request log files.