Missing Stored Procedure - Standard v5.1.2

[kagamat]

Hi,

I've inherited a Vault Server install v5.1.2. We are seeing exceptions in the server logs due to a missing stored procedure: spgetintegrations. This happens regularly, but not always, for any client. The procedure is not in the database, so the error is accurate.

This seems like a serious problem but no one knows how it might have happened. More importantly, is there a fix?

Any help is appreciated.

Thanks,
-Kevin

Here is an example cmd line invocation (via bat file), but the problem happens for GUI clients too.

%PATH_TO_VAULT%\vault getwildcard -setfiletime checkin -user %REPO_USER% -password %REPO_PASSWORD% -host %REPO_SERVER_NAME% -repository %REPO_NAME% %REPO_DIAGRAM_FOLDER% *.vsd

This is the result in the log:

System.Data.SqlClient.SqlException: Could not find stored procedure 'dbo.spgetintegrations'.
at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)
at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection)
at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj)
at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
at System.Data.SqlClient.SqlDataReader.ConsumeMetaData()
at System.Data.SqlClient.SqlDataReader.get_MetaData()
at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async)
at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, DbAsyncResult result)
at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method)
at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior, String method)
at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior)
at VaultServiceSQL.VaultSqlSystem.GetIntegrations(VaultSqlConn conn, Int32 nRepID, Boolean bActiveOnly, List`1& listIntegrationStrings) at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)
at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection)
at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj)
at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
at System.Data.SqlClient.SqlDataReader.ConsumeMetaData()
at System.Data.SqlClient.SqlDataReader.get_MetaData()
at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async)
at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, DbAsyncResult result)
at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method)
at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior, String method)
at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior)
at VaultServiceSQL.VaultSqlSystem.GetIntegrations(VaultSqlConn conn, Int32 nRepID, Boolean bActiveOnly, List`1& listIntegrationStrings)

[Beth]

Open your IIS Manager and find the website that Vault was installed to. Does it have a vaultpro folder listed with it? If so, delete the vaultpro folder on disk where the website is physically located. Then see if you still get the errors.

[kagamat]

Hi,

I did find and remove a vaultpro folder and restarted the web server.

After watching the logs a few minutes, I confirmed that the exception is still getting thrown.

[Beth]

Do you see an action happening prior to the error?

Please rename your current sgvault.log so that a new one is created.

If all you see are logins/logouts, then let's put the server into debug logging mode. Go to the Vault admin web page, click on logging, set it from Quiet to Debug. This will disconnect users as it recycles Vault, so you may need to do it at a convenient time.

[kagamat]

I've made the change. There is a lot more in the log now. Here is a snippet. Can I send you a larger file?

----11/20/2014 5:35:54 PM COMPUTER-NAME(ip-address)--SSL Disabled DB: GetUserEmailOptions started.
----11/20/2014 5:35:54 PM COMPUTER-NAME(ip-address)--SSL Disabled DB: GetKeywordExtensions started.
----11/20/2014 5:35:54 PM COMPUTER-NAME(ip-address)--SSL Disabled DB: GetUserEmailOptions finished.
----11/20/2014 5:35:54 PM COMPUTER-NAME(ip-address)--SSL Disabled GetUserEmailOptions returned Success
----11/20/2014 5:35:54 PM COMPUTER-NAME(ip-address)--SSL Disabled ASMX: GetUserEmailOptions finished.
----11/20/2014 5:35:54 PM COMPUTER-NAME(ip-address)--SSL Disabled DB: GetUserRepositoryLastChange started.
----11/20/2014 5:35:54 PM COMPUTER-NAME(ip-address)--SSL Disabled System.Data.SqlClient.SqlException: Could not find stored procedure 'dbo.spgetintegrations'.
at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)
at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection)
at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj)
at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
at System.Data.SqlClient.SqlDataReader.ConsumeMetaData()
at System.Data.SqlClient.SqlDataReader.get_MetaData()

[Beth]

Let's try one thing first.

First, do you use Shadow Folders? If so, backup the web.config that is in the vaultshadowfolder virtual directory. If it's in the default location, then it's at C:\inetpub\wwwroot\vaultservice\vaultshadowfolder\web.config. If not, then don't worry about backing up that file.

Next, backup your Vault databases (sgmaster, sgvault).

Uninstall Vault, but choose the option to keep the databases. Then install Vault again and allow it to pick up and reuse the databases.

If you're unsure of what options were chosen in the past install, then just email the vault_install.log from the server (usually in %temp%) to support at sourcegear.com with a link to this thread..

[kagamat]

It seems to be consistently related to DB: GetIntegrations started. There is always an exception after that call.

I'll take a look at your suggestions from the last reply.

----11/20/2014 5:35:54 PM USER--COMPUTER-NAME(ip-address)--SSL Disabled DB: GetIntegrations started.
----11/20/2014 5:35:54 PM USER--COMPUTER-NAME(ip-address)--SSL Disabled System.Data.SqlClient.SqlException: Could not find stored procedure 'dbo.spgetintegrations'.
at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)
at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection)
at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj)
at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
at System.Data.SqlClient.SqlDataReader.ConsumeMetaData()
at System.Data.SqlClient.SqlDataReader.get_MetaData()
at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async)
at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, DbAsyncResult result)
at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method)
at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior, String method)
at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior)
at VaultServiceSQL.VaultSqlSystem.GetIntegrations(VaultSqlConn conn, Int32 nRepID, Boolean bActiveOnly, List`1& listIntegrationStrings) at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)
at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection)
at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj)
at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
at System.Data.SqlClient.SqlDataReader.ConsumeMetaData()
at System.Data.SqlClient.SqlDataReader.get_MetaData()
at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async)
at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, DbAsyncResult result)
at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method)
at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior, String method)
at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior)
at VaultServiceSQL.VaultSqlSystem.GetIntegrations(VaultSqlConn conn, Int32 nRepID, Boolean bActiveOnly, List`1& listIntegrationStrings)

[kagamat]

We won't be able to re-install until next weekend.

The shadow service is running but seems unused. There was a bad path in the config file.

I'll reply once we do the re-install. Is there anything else to consider in the meantime?