Unable to branch/export

[WHSomerville]

We have a large (15GB or so) Vault DB. Vault version is 3.5.2. I'm trying to create a new branch (via branch or share) of our current trunk, and it fails. So, I decided to go the export/import route. That fails as well.

Basically, my source code is trapped.

Looking around on the board, I've done the following:

1. Backup/restore the Vault DB (i.e. into a fresh DB).
2. DBCC CHECKDB (passes with flying colors).

Server log from the branch (FailDBReader):

----11/3/2009 8:28:31 AM admin--::1(::1)--SSL Disabled Login
----11/3/2009 8:37:47 AM admin--::1(::1)--SSL Disabled System.Data.SqlClient.SqlException: Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)
at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection)
at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj)
at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
at System.Data.SqlClient.SqlDataReader.ConsumeMetaData()
at System.Data.SqlClient.SqlDataReader.get_MetaData()
at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async)
at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, DbAsyncResult result)
at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method)
at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior, String method)
at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior)
at VaultServiceSQL.VaultSqlSCC.BranchFolder(VaultSqlConn conn, String strSessionID, Int32 nRepID, Hashtable htSharedItems, HybridDictionary htTxModifiedItems, Int64 nTxID, Int32 nTxItem, Byte nTxType, VaultDateTime vdTxBegin, String strItemPath, String strNewBranchName, String strXml, BranchingModTime bmt, VaultFolder vfRoot, String strTxComment, Int32& nCopiedSecurityFolderRights, VaultFolder& vfOut) at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)
at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection)
at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj)
at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
at System.Data.SqlClient.SqlDataReader.ConsumeMetaData()
at System.Data.SqlClient.SqlDataReader.get_MetaData()
at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async)
at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, DbAsyncResult result)
at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method)
at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior, String method)
at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior)
at VaultServiceSQL.VaultSqlSCC.BranchFolder(VaultSqlConn conn, String strSessionID, Int32 nRepID, Hashtable htSharedItems, HybridDictionary htTxModifiedItems, Int64 nTxID, Int32 nTxItem, Byte nTxType, VaultDateTime vdTxBegin, String strItemPath, String strNewBranchName, String strXml, BranchingModTime bmt, VaultFolder vfRoot, String strTxComment, Int32& nCopiedSecurityFolderRights, VaultFolder& vfOut)
----11/3/2009 8:37:51 AM admin--::1(::1)--SSL Disabled (d16b803f-1045-4399-97a0-2302d34909d1) EndTx (Revision - 0) returned: FailDBReader
----11/3/2009 8:37:51 AM admin--::1(::1)--SSL Disabled (d16b803f-1045-4399-97a0-2302d34909d1) CopyBranch: $/XXXXX/Current to $/XXXXX/10.0 returned: FailDBReader


Server log during the export:

----11/2/2009 5:26:21 PM admin--::1(::1)--SSL Disabled Login
----11/3/2009 7:32:06 AM admin--::1(::1)--SSL Disabled Logout

Export log at the time of failure (fails at around 6500 out of 123000 transactions):

11/2/2009 9:17:07 PM <generic>: [<No Name>:1816] Folder Export: Retrieving transaction for XXXXX on 1/25/2007 2:07:26 PM TxID: 283724.
11/2/2009 9:17:09 PM <generic>: [<No Name>:1816] Folder Export: An exception occurred in ProcessResults_ProcessTransaction_BuildChangesetForTransaction(). Tx: 283724 Transaction before exception: 283724 on 1/25/2007 2:07:26 PM
Change Set Items. Object reference not set to an instance of an object. at VaultFolderExportImport.frmExport.PickDeltaSummaryItem(ArrayList alSummaryItems, String strItemPath, Int32 nChange, PinStatus ps, Int64 nObjVerID, String strPrefObjPath, String strPrefObjPathWithSlash)
at VaultFolderExportImport.frmExport.BuildChangesetForTransaction(String strCalleePrefix, String strDeltaObjPath, String strDeltaObjPathWithSlash, HybridDictionary hdObjectIDsToExport, VaultFolderExportTxInfo tx, VaultTxDetail txDetail, VaultRepositoryDeltaSummary summary) [System.NullReferenceException: Object reference not set to an instance of an object.
at VaultFolderExportImport.frmExport.PickDeltaSummaryItem(ArrayList alSummaryItems, String strItemPath, Int32 nChange, PinStatus ps, Int64 nObjVerID, String strPrefObjPath, String strPrefObjPathWithSlash)
at VaultFolderExportImport.frmExport.BuildChangesetForTransaction(String strCalleePrefix, String strDeltaObjPath, String strDeltaObjPathWithSlash, HybridDictionary hdObjectIDsToExport, VaultFolderExportTxInfo tx, VaultTxDetail txDetail, VaultRepositoryDeltaSummary summary)]Object reference not set to an instance of an object.

11/2/2009 9:17:09 PM <generic>: [<No Name>:1816] Folder Export: An exception occurred in ProcessResults_ProcessTransaction() Object reference not set to an instance of an object. at VaultFolderExportImport.frmExport.PickDeltaSummaryItem(ArrayList alSummaryItems, String strItemPath, Int32 nChange, PinStatus ps, Int64 nObjVerID, String strPrefObjPath, String strPrefObjPathWithSlash)
at VaultFolderExportImport.frmExport.BuildChangesetForTransaction(String strCalleePrefix, String strDeltaObjPath, String strDeltaObjPathWithSlash, HybridDictionary hdObjectIDsToExport, VaultFolderExportTxInfo tx, VaultTxDetail txDetail, VaultRepositoryDeltaSummary summary)
at VaultFolderExportImport.frmExport.ProcessResults_ProcessTransaction(ExportCommand ec) [System.NullReferenceException: Object reference not set to an instance of an object.
at VaultFolderExportImport.frmExport.PickDeltaSummaryItem(ArrayList alSummaryItems, String strItemPath, Int32 nChange, PinStatus ps, Int64 nObjVerID, String strPrefObjPath, String strPrefObjPathWithSlash)
at VaultFolderExportImport.frmExport.BuildChangesetForTransaction(String strCalleePrefix, String strDeltaObjPath, String strDeltaObjPathWithSlash, HybridDictionary hdObjectIDsToExport, VaultFolderExportTxInfo tx, VaultTxDetail txDetail, VaultRepositoryDeltaSummary summary)
at VaultFolderExportImport.frmExport.ProcessResults_ProcessTransaction(ExportCommand ec)]Object reference not set to an instance of an object.

11/3/2009 7:31:58 AM <generic>: [<No Name>:1816] Folder Export: Closing the Export Dialog with Dialog Result: Cancel
11/3/2009 7:31:59 AM <generic>: [<No Name>:1816] Folder Export: ## FOLDER EXPORT END ##
11/3/2009 7:32:06 AM <generic>: [<No Name>:3988] SimpleLogger finalized


Output of DBCC CHECKDB:

1> DBCC CHECKDB (sgvault)
2> go
DBCC results for 'sgvault'.
Service Broker Msg 9675, State 1: Message Types analyzed: 14.
Service Broker Msg 9676, State 1: Service Contracts analyzed: 6.
Service Broker Msg 9667, State 1: Services analyzed: 3.
Service Broker Msg 9668, State 1: Service Queues analyzed: 3.
Service Broker Msg 9669, State 1: Conversation Endpoints analyzed: 0.
Service Broker Msg 9674, State 1: Conversation Groups analyzed: 0.
Service Broker Msg 9670, State 1: Remote Service Bindings analyzed: 0.
DBCC results for 'sys.sysrowsetcolumns'.
There are 1084 rows in 10 pages for object "sys.sysrowsetcolumns".
DBCC results for 'sys.sysrowsets'.
There are 199 rows in 2 pages for object "sys.sysrowsets".
DBCC results for 'sysallocunits'.
There are 211 rows in 3 pages for object "sysallocunits".
DBCC results for 'sys.sysfiles1'.
There are 2 rows in 1 pages for object "sys.sysfiles1".
DBCC results for 'sys.syshobtcolumns'.
There are 1084 rows in 10 pages for object "sys.syshobtcolumns".
DBCC results for 'sys.syshobts'.
There are 199 rows in 2 pages for object "sys.syshobts".
DBCC results for 'sys.sysftinds'.
There are 0 rows in 0 pages for object "sys.sysftinds".
DBCC results for 'sys.sysserefs'.
There are 211 rows in 1 pages for object "sys.sysserefs".
DBCC results for 'sys.sysowners'.
There are 16 rows in 1 pages for object "sys.sysowners".
DBCC results for 'sys.sysprivs'.
There are 147 rows in 1 pages for object "sys.sysprivs".
DBCC results for 'sys.sysschobjs'.
There are 676 rows in 16 pages for object "sys.sysschobjs".
DBCC results for 'sys.syscolpars'.
There are 2200 rows in 43 pages for object "sys.syscolpars".
DBCC results for 'sys.sysnsobjs'.
There are 1 rows in 1 pages for object "sys.sysnsobjs".
DBCC results for 'sys.syscerts'.
There are 0 rows in 0 pages for object "sys.syscerts".
DBCC results for 'sys.sysxprops'.
There are 21 rows in 1 pages for object "sys.sysxprops".
DBCC results for 'sys.sysscalartypes'.
There are 52 rows in 1 pages for object "sys.sysscalartypes".
DBCC results for 'sys.systypedsubobjs'.
There are 0 rows in 0 pages for object "sys.systypedsubobjs".
DBCC results for 'sys.sysidxstats'.
There are 399 rows in 10 pages for object "sys.sysidxstats".
DBCC results for 'sys.sysiscols'.
There are 703 rows in 5 pages for object "sys.sysiscols".
DBCC results for 'sys.sysbinobjs'.
There are 23 rows in 1 pages for object "sys.sysbinobjs".
DBCC results for 'sys.sysobjvalues'.
There are 726 rows in 326 pages for object "sys.sysobjvalues".
DBCC results for 'sys.sysclsobjs'.
There are 16 rows in 1 pages for object "sys.sysclsobjs".
DBCC results for 'sys.sysrowsetrefs'.
There are 0 rows in 0 pages for object "sys.sysrowsetrefs".
DBCC results for 'sys.sysremsvcbinds'.
There are 0 rows in 0 pages for object "sys.sysremsvcbinds".
DBCC results for 'sys.sysxmitqueue'.
There are 0 rows in 0 pages for object "sys.sysxmitqueue".
DBCC results for 'sys.sysrts'.
There are 1 rows in 1 pages for object "sys.sysrts".
DBCC results for 'sys.sysconvgroup'.
There are 0 rows in 0 pages for object "sys.sysconvgroup".
DBCC results for 'sys.sysdesend'.
There are 0 rows in 0 pages for object "sys.sysdesend".
DBCC results for 'sys.sysdercv'.
There are 0 rows in 0 pages for object "sys.sysdercv".
DBCC results for 'sys.syssingleobjrefs'.
There are 517 rows in 7 pages for object "sys.syssingleobjrefs".
DBCC results for 'sys.sysmultiobjrefs'.
There are 2952 rows in 28 pages for object "sys.sysmultiobjrefs".
DBCC results for 'sys.sysdbfiles'.
There are 2 rows in 1 pages for object "sys.sysdbfiles".
DBCC results for 'sys.sysguidrefs'.
There are 0 rows in 0 pages for object "sys.sysguidrefs".
DBCC results for 'sys.sysqnames'.
There are 91 rows in 1 pages for object "sys.sysqnames".
DBCC results for 'sys.sysxmlcomponent'.
There are 93 rows in 1 pages for object "sys.sysxmlcomponent".
DBCC results for 'sys.sysxmlfacet'.
There are 97 rows in 1 pages for object "sys.sysxmlfacet".
DBCC results for 'sys.sysxmlplacement'.
There are 17 rows in 1 pages for object "sys.sysxmlplacement".
DBCC results for 'sys.sysobjkeycrypts'.
There are 0 rows in 0 pages for object "sys.sysobjkeycrypts".
DBCC results for 'sys.sysasymkeys'.
There are 0 rows in 0 pages for object "sys.sysasymkeys".
DBCC results for 'sys.syssqlguides'.
There are 0 rows in 0 pages for object "sys.syssqlguides".
DBCC results for 'sys.sysbinsubobjs'.
There are 0 rows in 0 pages for object "sys.sysbinsubobjs".
DBCC results for 'tblcheckoutlists'.
There are 66 rows in 4 pages for object "tblcheckoutlists".
DBCC results for 'tblclientoptions'.
There are 1920 rows in 28 pages for object "tblclientoptions".
DBCC results for 'tblcloaks'.
There are 4 rows in 1 pages for object "tblcloaks".
DBCC results for 'tblcurrenttreerevisionfolders'.
There are 5282 rows in 46 pages for object "tblcurrenttreerevisionfolders".
DBCC results for 'tbldelayusersecuritychanges'.
There are 0 rows in 0 pages for object "tbldelayusersecuritychanges".
DBCC results for 'tbldelaysecurityassignments'.
There are 0 rows in 0 pages for object "tbldelaysecurityassignments".
DBCC results for 'tbldeletedfsobjects'.
There are 14310 rows in 70 pages for object "tbldeletedfsobjects".
DBCC results for 'tbldeletions'.
There are 11349 rows in 69 pages for object "tbldeletions".
DBCC results for 'tbleolconversion'.
There are 5 rows in 1 pages for object "tbleolconversion".
DBCC results for 'tblfileinfo'.
There are 273830 rows in 1609 pages for object "tblfileinfo".
DBCC results for 'tblfolderentries'.
There are 71030540 rows in 414172 pages for object "tblfolderentries".
DBCC results for 'tblfolderentrypins'.
There are 2685173 rows in 15446 pages for object "tblfolderentrypins".
DBCC results for 'tblfsobjectattrs'.
There are 0 rows in 0 pages for object "tblfsobjectattrs".
DBCC results for 'tblfsobjectattrvalues'.
There are 0 rows in 0 pages for object "tblfsobjectattrvalues".
DBCC results for 'tblfsobjecthistoryitemmisc'.
There are 135241 rows in 1309 pages for object "tblfsobjecthistoryitemmisc".
DBCC results for 'tblfsobjecthistoryitems'.
There are 395327 rows in 2247 pages for object "tblfsobjecthistoryitems".
DBCC results for 'tblfsobjectlabelchanges'.
There are 9874 rows in 63 pages for object "tblfsobjectlabelchanges".
DBCC results for 'tblfsobjectlabelchangedeltas'.
There are 61808 rows in 889 pages for object "tblfsobjectlabelchangedeltas".
DBCC results for 'tblfsobjectlabels'.
There are 9867 rows in 2772 pages for object "tblfsobjectlabels".
DBCC results for 'tblfsobjectlabelviews'.
There are 659358 rows in 15774 pages for object "tblfsobjectlabelviews".
DBCC results for 'tblfsobjectorigins'.
There are 3849 rows in 28 pages for object "tblfsobjectorigins".
DBCC results for 'tblsecurityassignments'.
There are 0 rows in 0 pages for object "tblsecurityassignments".
DBCC results for 'tblsecurityassignmentspaths'.
There are 0 rows in 0 pages for object "tblsecurityassignmentspaths".
DBCC results for 'tblfsobjects'.
There are 72859 rows in 583 pages for object "tblfsobjects".
DBCC results for 'tblfsobjectversionattrs'.
There are 0 rows in 0 pages for object "tblfsobjectversionattrs".
DBCC results for 'tblfsobjectversionattrvalues'.
There are 0 rows in 0 pages for object "tblfsobjectversionattrvalues".
DBCC results for 'tblfsobjectversionorigattrvalues'.
There are 0 rows in 0 pages for object "tblfsobjectversionorigattrvalues".
DBCC results for 'tblfsobjectversionfileinfos'.
There are 252380 rows in 1416 pages for object "tblfsobjectversionfileinfos".
DBCC results for 'tblfsobjectversionnames'.
There are 70626 rows in 471 pages for object "tblfsobjectversionnames".
DBCC results for 'tblfsobjectversions'.
There are 1822216 rows in 13815 pages for object "tblfsobjectversions".
DBCC results for 'tblfulltreerevisionfolders'.
There are 153 rows in 8 pages for object "tblfulltreerevisionfolders".
DBCC results for 'tblgroups'.
There are 3 rows in 1 pages for object "tblgroups".
DBCC results for 'tblgroupmembers'.
There are 16 rows in 1 pages for object "tblgroupmembers".
DBCC results for 'tblkeywordextensions'.
There are 29 rows in 1 pages for object "tblkeywordextensions".
DBCC results for 'tbllicensekeys'.
There are 3 rows in 1 pages for object "tbllicensekeys".
DBCC results for 'tblloginattempts'.
There are 0 rows in 1 pages for object "tblloginattempts".
DBCC results for 'tblmergeablefileextensions'.
There are 136 rows in 1 pages for object "tblmergeablefileextensions".
DBCC results for 'tblfilefolderexclusions'.
There are 69 rows in 1 pages for object "tblfilefolderexclusions".
DBCC results for 'tblpendingtransactionobjects'.
There are 0 rows in 1 pages for object "tblpendingtransactionobjects".
DBCC results for 'tblplugins'.
There are 0 rows in 0 pages for object "tblplugins".
DBCC results for 'tblrepositories'.
There are 3 rows in 1 pages for object "tblrepositories".
DBCC results for 'tbltransactiondetailattrs'.
There are 1 rows in 1 pages for object "tbltransactiondetailattrs".
DBCC results for 'tbltransactiondetailattrvalues'.
There are 214689 rows in 4207 pages for object
"tbltransactiondetailattrvalues".
DBCC results for 'tbltransactiondetails'.
There are 321827 rows in 6018 pages for object "tbltransactiondetails".
DBCC results for 'tbltransactionattrs'.
There are 1 rows in 1 pages for object "tbltransactionattrs".
DBCC results for 'tbltransactionattrvalues'.
There are 5697 rows in 106 pages for object "tbltransactionattrvalues".
DBCC results for 'tbltransactionfsobjectversions'.
There are 0 rows in 1 pages for object "tbltransactionfsobjectversions".
DBCC results for 'tbltransactionrepositorycounts'.
There are 0 rows in 1 pages for object "tbltransactionrepositorycounts".
DBCC results for 'tbltransactions'.
There are 295521 rows in 1501 pages for object "tbltransactions".
DBCC results for 'tbltreerevisionfolderdeltas'.
There are 2861283 rows in 45762 pages for object "tbltreerevisionfolderdeltas".
DBCC results for 'tbltreerevisionfolders'.
There are 426445 rows in 3242 pages for object "tbltreerevisionfolders".
DBCC results for 'tblschemaversions'.
There are 1 rows in 1 pages for object "tblschemaversions".
DBCC results for 'tblsessionbranchsandbox'.
There are 0 rows in 0 pages for object "tblsessionbranchsandbox".
DBCC results for 'tblsessionfolderexports'.
There are 0 rows in 71 pages for object "tblsessionfolderexports".
DBCC results for 'tblsessionlabelqueries'.
There are 0 rows in 1 pages for object "tblsessionlabelqueries".
DBCC results for 'tblsessionrecursivelabelqueries'.
There are 0 rows in 85 pages for object "tblsessionrecursivelabelqueries".
DBCC results for 'tblsessions'.
There are 0 rows in 1 pages for object "tblsessions".
DBCC results for 'tbluserrepositories'.
There are 189 rows in 1 pages for object "tbluserrepositories".
DBCC results for 'tblusers'.
There are 63 rows in 3 pages for object "tblusers".
DBCC results for 'tblusersecuritychanges'.
There are 189 rows in 1 pages for object "tblusersecuritychanges".
DBCC results for 'tblvaultsettings'.
There are 1 rows in 1 pages for object "tblvaultsettings".
DBCC results for 'tblwatchers'.
There are 11 rows in 1 pages for object "tblwatchers".
DBCC results for 'tblhistoryfavorites'.
There are 15 rows in 1 pages for object "tblhistoryfavorites".
DBCC results for 'tblhistoryfavoriteitems'.
There are 215 rows in 1 pages for object "tblhistoryfavoriteitems".
DBCC results for 'tblfsobjectshares'.
There are 18237 rows in 319 pages for object "tblfsobjectshares".
DBCC results for 'sys.queue_messages_1977058079'.
There are 0 rows in 0 pages for object "sys.queue_messages_1977058079".
DBCC results for 'sys.queue_messages_2009058193'.
There are 0 rows in 0 pages for object "sys.queue_messages_2009058193".
DBCC results for 'sys.queue_messages_2041058307'.
There are 0 rows in 0 pages for object "sys.queue_messages_2041058307".
DBCC results for 'tblbackups'.
There are 0 rows in 0 pages for object "tblbackups".
DBCC results for 'tblbranchedhistoryitems'.
There are 49249 rows in 246 pages for object "tblbranchedhistoryitems".
DBCC results for 'tblbulkcreatefsobjects'.
There are 0 rows in 1 pages for object "tblbulkcreatefsobjects".
DBCC results for 'tblcheckoutlistitemdeltas'.
There are 106 rows in 16 pages for object "tblcheckoutlistitemdeltas".
DBCC results for 'tblcheckoutlistitems'.
There are 158 rows in 9 pages for object "tblcheckoutlistitems".
CHECKDB found 0 allocation errors and 0 consistency errors in database
'sgvault'.
DBCC execution completed. If DBCC printed error messages, contact your system
administrator.
1>

[lbauer]

If you're branching a large trunk, it can take some time, and it looks like SQL Server is timing out.

You can increase the timeout in the Vault.config file in the VaultService directory on the server machine. Look for the <SqlCommandTimeout> line, and increase the value to 7200 or so.

We made some improvements in branch speed in Vault 4.1.4, so upgrading may also help.

If you want to try the export again, we'd suggest upgrading to at least 3.5.3, because we did fix some exportimport issues in that release.

[WHSomerville]

Linda:
Thanks for the reply. Changing the timeout did allow me to branch. I'll look at upgrading, but I think I already tried with 4.0, and it still failed. I'll let you know for sure in a while...

-- Bill

[lbauer]

Ok, let us know if you have further questions.