Large checkin kills server

If you are having a problem using Vault, post a message here.

Moderator: SourceGear

Post Reply
edboykin
Posts: 9
Joined: Wed Oct 03, 2007 6:13 am

Large checkin kills server

Post by edboykin » Tue Feb 26, 2008 10:32 am

I'm doing a large number of changes to a large number of files. About 7500 files. I have to be able to process files in large batches like this. I just tried to check them in and the server exploded. Nobody can login and those that were logged in have been disconnected.

The database appears to be rolling back a huge transaction. This has happened before and continues to cause problems. Our vault server will be down now for over an hour or more while this finsihes.

We are running client and server version 4.0.6

Anyway, here is what my clietn log window said:
Client Log:
[2/26/2008 11:17:06 AM] Check in $/Delphi_60/Trunk/MGU Office/BlockAnalysis/History_60/Conversion/DOS/TransferTables/Source/frm_TransferHistoryDOSTables.dfm
[2/26/2008 11:17:37 AM] Upload for item $/Delphi_60/Trunk/MGU Office/Underwriting2/UwrCensus_60 MU/Source/frm_UwrCensusZip.dfm failed, retrying...
[2/26/2008 11:17:37 AM] Upload for item $/Delphi_60/Trunk/MGU Office/Underwriting2/UwrCensus_60 MU/Source/frm_UwrImportCensusZipFormat.dfm failed, retrying...
[2/26/2008 11:17:53 AM] Upload for item $/Delphi_60/Trunk/MGU Office/Underwriting2/UwrCensus_60 MU/Source/frm_UwrImportCensusZipFormat.dfm failed, retrying...
[2/26/2008 11:17:53 AM] Upload for item $/Delphi_60/Trunk/MGU Office/Underwriting2/UwrCensus_60 MU/Source/frm_UwrCensusZip.dfm failed, retrying...
[2/26/2008 11:18:08 AM] Upload for item $/Delphi_60/Trunk/MGU Office/Underwriting2/UwrCensus_60 MU/Source/frm_UwrImportCensusZipFormat.dfm failed, retrying...
[2/26/2008 11:18:08 AM] Upload for item $/Delphi_60/Trunk/MGU Office/Underwriting2/UwrCensus_60 MU/Source/frm_UwrCensusZip.dfm failed, retrying...
[2/26/2008 11:18:08 AM] Ending the transaction
[2/26/2008 11:18:08 AM] Upload for item $/Delphi_60/Trunk/MGU Office/Underwriting2/UwrCensus_60 MU/Source/frm_UwrImportCensusZipFormat.dfm failed too many times, aborting transaction.
Please verify your network settings using the Options dialog under the Tools menu in the Vault GUI Client.
The specific error was "The server returned an unknown error header: FailDBConn"
An exception was encountered during the transaction. Exception: The server returned an unknown error header: FailDBConn at VaultClientOperationsLib.ClientInstance.UploadItem(ChangeSetItem item, String txID, Byte[]& streamBuffer, Int32& bytesWrittenThisFile, Boolean bIsImport)
at VaultClientOperationsLib.UploadThread.ProcessCommand(UploadThreadCommand command, UploadThreadCommandResult& outputResult)
[2/26/2008 11:18:08 AM] Upload for item $/Delphi_60/Trunk/MGU Office/Underwriting2/UwrCensus_60 MU/Source/frm_UwrCensusZip.dfm failed too many times, aborting transaction.
Please verify your network settings using the Options dialog under the Tools menu in the Vault GUI Client.
The specific error was "The server returned an unknown error header: FailDBConn"
An exception was encountered during the transaction. Exception: The server returned an unknown error header: FailDBConn at VaultClientOperationsLib.ClientInstance.UploadItem(ChangeSetItem item, String txID, Byte[]& streamBuffer, Int32& bytesWrittenThisFile, Boolean bIsImport)
at VaultClientOperationsLib.UploadThread.ProcessCommand(UploadThreadCommand command, UploadThreadCommandResult& outputResult)
[2/26/2008 11:18:23 AM] Transaction failed
[2/26/2008 11:18:54 AM] Transaction failed


And here is what the server log said:
----2/26/2008 11:22:28 AM eb--64.238.124.114(64.238.124.114)--SSL Disabled Vault Failed to establish a connection to the database.
----2/26/2008 11:22:28 AM eb--64.238.124.114(64.238.124.114)--SSL Disabled System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn) at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn)
----2/26/2008 11:22:28 AM eb--64.238.124.114(64.238.124.114)--SSL Disabled VaultFileUpload.aspx encountered: FailDBConn
----2/26/2008 11:22:29 AM eb--64.238.124.114(64.238.124.114)--SSL Disabled Vault Failed to establish a connection to the database.
----2/26/2008 11:22:29 AM eb--64.238.124.114(64.238.124.114)--SSL Disabled System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn) at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn)
----2/26/2008 11:22:29 AM eb--64.238.124.114(64.238.124.114)--SSL Disabled VaultFileUpload.aspx encountered: FailDBConn
----2/26/2008 11:22:36 AM ft--64.238.124.114(64.238.124.114)--SSL Disabled Vault Failed to establish a connection to the database.
----2/26/2008 11:22:36 AM ft--64.238.124.114(64.238.124.114)--SSL Disabled System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn) at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn)
----2/26/2008 11:22:36 AM ft--64.238.124.114(64.238.124.114)--SSL Disabled FailDBConn
----2/26/2008 11:22:39 AM bt--64.238.124.114(64.238.124.114)--SSL Disabled Vault Failed to establish a connection to the database.
----2/26/2008 11:22:39 AM bt--64.238.124.114(64.238.124.114)--SSL Disabled System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn) at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn)
----2/26/2008 11:22:39 AM bt--64.238.124.114(64.238.124.114)--SSL Disabled The connection to the database failed for VaultServiceAPI.Logout: FailDBConn
----2/26/2008 11:22:44 AM eb--64.238.124.114(64.238.124.114)--SSL Disabled Vault Failed to establish a connection to the database.
----2/26/2008 11:22:44 AM eb--64.238.124.114(64.238.124.114)--SSL Disabled Vault Failed to establish a connection to the database.
----2/26/2008 11:22:44 AM eb--64.238.124.114(64.238.124.114)--SSL Disabled System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn) at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn)
----2/26/2008 11:22:44 AM eb--64.238.124.114(64.238.124.114)--SSL Disabled System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn) at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn)
----2/26/2008 11:22:44 AM eb--64.238.124.114(64.238.124.114)--SSL Disabled VaultFileUpload.aspx encountered: FailDBConn
----2/26/2008 11:22:44 AM eb--64.238.124.114(64.238.124.114)--SSL Disabled VaultFileUpload.aspx encountered: FailDBConn
----2/26/2008 11:22:45 AM ma--64.238.124.114(64.238.124.114)--SSL Disabled Vault Failed to establish a connection to the database.
----2/26/2008 11:22:45 AM ma--64.238.124.114(64.238.124.114)--SSL Disabled System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn) at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn)
----2/26/2008 11:22:45 AM ma--64.238.124.114(64.238.124.114)--SSL Disabled FailDBConn
----2/26/2008 11:22:51 AM ft--64.238.124.114(64.238.124.114)--SSL Disabled Vault Failed to establish a connection to the database.
----2/26/2008 11:22:51 AM ft--64.238.124.114(64.238.124.114)--SSL Disabled System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn) at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn)
----2/26/2008 11:22:51 AM ft--64.238.124.114(64.238.124.114)--SSL Disabled FailDBConn
----2/26/2008 11:22:54 AM --()-- Vault Failed to establish a connection to the database.
----2/26/2008 11:22:54 AM --()-- System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn) at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn)
----2/26/2008 11:22:54 AM --()-- The connection to the database failed for VaultServiceAPI.Logout: FailDBConn
----2/26/2008 11:23:00 AM eb--64.238.124.114(64.238.124.114)--SSL Disabled Vault Failed to establish a connection to the database.
----2/26/2008 11:23:00 AM eb--64.238.124.114(64.238.124.114)--SSL Disabled Vault Failed to establish a connection to the database.
----2/26/2008 11:23:00 AM eb--64.238.124.114(64.238.124.114)--SSL Disabled System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn) at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn)
----2/26/2008 11:23:00 AM eb--64.238.124.114(64.238.124.114)--SSL Disabled System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn) at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn)
----2/26/2008 11:23:00 AM eb--64.238.124.114(64.238.124.114)--SSL Disabled VaultFileUpload.aspx encountered: FailDBConn
----2/26/2008 11:23:00 AM eb--64.238.124.114(64.238.124.114)--SSL Disabled VaultFileUpload.aspx encountered: FailDBConn
----2/26/2008 11:23:00 AM ma--64.238.124.114(64.238.124.114)--SSL Disabled Vault Failed to establish a connection to the database.
----2/26/2008 11:23:00 AM ma--64.238.124.114(64.238.124.114)--SSL Disabled System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn) at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn)
----2/26/2008 11:23:00 AM ma--64.238.124.114(64.238.124.114)--SSL Disabled FailDBConn
----2/26/2008 11:23:15 AM eb--64.238.124.114(64.238.124.114)--SSL Disabled Vault Failed to establish a connection to the database.
----2/26/2008 11:23:15 AM eb--64.238.124.114(64.238.124.114)--SSL Disabled System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn) at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn)
----2/26/2008 11:23:30 AM eb--64.238.124.114(64.238.124.114)--SSL Disabled Vault Failed to establish a connection to the database.
----2/26/2008 11:23:30 AM eb--64.238.124.114(64.238.124.114)--SSL Disabled System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn) at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn)
----2/26/2008 11:23:30 AM eb--64.238.124.114(64.238.124.114)--SSL Disabled FailDBConn
----2/26/2008 11:23:40 AM ft--64.238.124.114(64.238.124.114)--SSL Disabled Vault Failed to establish a connection to the database.
----2/26/2008 11:23:40 AM ft--64.238.124.114(64.238.124.114)--SSL Disabled System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn) at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn)
----2/26/2008 11:23:40 AM ft--64.238.124.114(64.238.124.114)--SSL Disabled FailDBConn
----2/26/2008 11:23:46 AM eb--64.238.124.114(64.238.124.114)--SSL Disabled Vault Failed to establish a connection to the database.
----2/26/2008 11:23:46 AM eb--64.238.124.114(64.238.124.114)--SSL Disabled System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn) at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn)
----2/26/2008 11:23:46 AM eb--64.238.124.114(64.238.124.114)--SSL Disabled FailDBConn
----2/26/2008 11:23:55 AM ft--64.238.124.114(64.238.124.114)--SSL Disabled Vault Failed to establish a connection to the database.
----2/26/2008 11:23:55 AM ft--64.238.124.114(64.238.124.114)--SSL Disabled System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn) at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn)
----2/26/2008 11:23:55 AM ft--64.238.124.114(64.238.124.114)--SSL Disabled FailDBConn
----2/26/2008 11:24:27 AM eb--64.238.124.114(64.238.124.114)--SSL Disabled Vault Failed to establish a connection to the database.
----2/26/2008 11:24:27 AM eb--64.238.124.114(64.238.124.114)--SSL Disabled System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn) at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn)
----2/26/2008 11:24:27 AM eb--64.238.124.114(64.238.124.114)--SSL Disabled FailDBConn
----2/26/2008 11:24:42 AM eb--64.238.124.114(64.238.124.114)--SSL Disabled Vault Failed to establish a connection to the database.
----2/26/2008 11:24:42 AM eb--64.238.124.114(64.238.124.114)--SSL Disabled System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn) at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at VaultServiceSQL.VaultSqlConn.OpenConn()
at VaultServiceAPILib.VaultServiceAPISystem.GetOpenSqlConn(VaultSqlConn& conn)
----2/26/2008 11:24:42 AM eb--64.238.124.114(64.238.124.114)--SSL Disabled FailDBConn
Top
Beth
Posts: 8550
Joined: Wed Jun 21, 2006 8:24 pm
Location: SourceGear
Contact:
Contact Beth

Post by Beth » Tue Feb 26, 2008 2:03 pm

It looks like a timeout. Could you go to your vault.config file and find the setting for the SQL timeout? By default, it's set to 360. Add another 0 to that.

Also, are all the files actually checking in, or are some unmodified and then actually going through an undo check out? An undo check out is the default action for files that are not modified.

What is the purpose of checking out 7500 files and checking them in at once? I might have some other ideas for you of things that may help if I understood the entire situation.
Top
edboykin
Posts: 9
Joined: Wed Oct 03, 2007 6:13 am

Post by edboykin » Tue Feb 26, 2008 2:21 pm

I have changed the timeout per your recomendation.

We're undergoing a migration to a new version of our development tool. This required us to modify certain source code files so they now reference different included libraries, etc. (Roughly 7k files)

I checked out all the files I needed to. Then did a grep serach/replace on the stuff I needed to change and then tried to check them back in.

Note: I was able to get everything checked back in by doing it in chunks but that took a considarable amount of time.
Top
Beth
Posts: 8550
Joined: Wed Jun 21, 2006 8:24 pm
Location: SourceGear
Contact:
Contact Beth

Post by Beth » Wed Feb 27, 2008 7:20 pm

If you were still getting a timeout, let's try adding one more '0' in the SQL timeout. Mostly, I'm hoping that it would give some other error than a timeout as that will give more information.

Do you have any errors in your Event Viewer on the server?

What happens if you try the same actions right on the server and use 'localhost' rather than a machine name or IP address for connecting to the server?
Top
Post Reply

Return to “Support (Vault)”