VSS Import troubles

Mike Mestemaker · Post by **Mike Mestemaker** » Mon Oct 09, 2006 7:23 am

I'm stuck in a conundrum with this import. Our VSS db is 10gb in size. It takes almost 20 hours just to run the pre-scan and estimates the full import at almost 4 days. I've tried twice now and failed. The first time, the server rebooted and we can't find any reason for it. The 2nd, it failed 2 hours into the main import with this message in the log:

Upload for item $/Main/eLearningCDROM/E-Learning CDROM Builds 10-31-03/Effective Leadership/Course/Runtime/Win9x/COMCTL32.OCX failed too many times, aborting transaction.
Please verify your network settings using the Options dialog under the Tools menu in the Vault GUI Client.
The specific error was "The request was aborted: The request was canceled."
An exception was encountered during the transaction. Exception: The request was aborted: The request was canceled. at System.Net.HttpWebRequest.CheckFinalStatus()
at System.Net.HttpWebRequest.GetResponse()
at VaultClientOperationsLib.ClientInstance.UploadItem(ChangeSetItem item, String txID, Byte[]& streamBuffer, Int32& bytesWrittenThisFile, Boolean bIsImport)
at VaultClientOperationsLib.UploadThread.ProcessCommand(UploadThreadCommand command, UploadThreadCommandResult& outputResult)

From there, it tries to reestablish a connection, but it can't.

Every test import I've done on individual projects works fine. But trying to do the whole db is just too much for it, it seems.

The problem is that we use a lot of shared code modules, so doing the import one project at a time will cause me to lose those links.

Any suggestions would be great. At the moment, I'm sitting here with an expensive Source Code Control system that I am not using.

Beth · Post by **Beth** » Mon Oct 09, 2006 8:57 am

Have you reviewed our article for a successful import? http://support.sourcegear.com/viewtopic.php?t=7

Can you provide me details on the set-up you are using for performing this? (server OS, RAM, processor, Vault version)

Mike Mestemaker · Post by **Mike Mestemaker** » Mon Oct 09, 2006 9:19 am

Beth wrote:Have you reviewed our article for a successful import? http://support.sourcegear.com/viewtopic.php?t=7

Can you provide me details on the set-up you are using for performing this? (server OS, RAM, processor, Vault version)

Yes, I have read that document and tried to followed as best I can.

This is a single server running Windows 2003 server, 2gb ram, fairly recent xeon processor, current vault version, SQL 2005 for the db. Over 100gb free drive space.

Beth · Post by **Beth** » Mon Oct 09, 2006 9:26 am

Can you send the import log located in:
C:\Program Files\SourceGear\SourceSafe Import Tool

You can send it to either the private function here or to support at sourcegear.com and just reference this posting.

Beth · Post by **Beth** » Mon Oct 09, 2006 9:47 am

I'm checking on whether we need to take a look at your database or not. In the meantime, I did see a few errors in the log that state:
"Exception of type System.OutOfMemoryException was thrown."

Have you watched the memory usage while trying to perform the import?

Could you also send a Vault Server Log and an Event Viewer Log that details what's going on during the import?

If your Vault logging has not been increased, you mean wish to set that to debug mode. To place the Vault Server into Debug mode, open the Vault Admin Tool and go to the Server Options tab. The second half of the window covers logging. Set the Log Level to Debug.

Mike Mestemaker · Post by **Mike Mestemaker** » Mon Oct 09, 2006 10:08 am

Beth wrote:I'm checking on whether we need to take a look at your database or not. In the meantime, I did see a few errors in the log that state:
"Exception of type System.OutOfMemoryException was thrown."

Have you watched the memory usage while trying to perform the import?

I was watching for a bit as the pre-scan neared it's completion. The memory usage of the vssimport program was nearly 200mb, but never went much beyond that. Even with SQL Server eating lots of memory as it likes to do, there was still over 150mb free at all times. After I acknowledged the pre-scan, I went home, so I was unable to watch the memory after that.

Could you also send a Vault Server Log and an Event Viewer Log that details what's going on during the import?

I'm getting an error sending those to you as your space is full.

If your Vault logging has not been increased, you mean wish to set that to debug mode. To place the Vault Server into Debug mode, open the Vault Admin Tool and go to the Server Options tab. The second half of the window covers logging. Set the Log Level to Debug.

The logging is not at debug mode; I can increase it, but I can't retry this import during the week. We need a live SCC system of one type or the other during the week.

One thing I noticed: I've got someone setting up draco.net to watch for changes and invoke builds and that was running at the same time. She pointing to a different repository, so she was reacting to the import, but there are a lot of repeated login/logout events. Would this have caused the issue?

Beth · Post by **Beth** » Mon Oct 09, 2006 10:36 am

I'm getting an error sending those to you as your space is full.

I think we allow up to 2mb attachments, so you may have to compress it and make sure to name it something unique. So that it's not public, if you want, you can send it to me using the private function, or send it in an email to support at sourcegear.com and reference this thread.

Mike Mestemaker · Post by **Mike Mestemaker** » Mon Oct 09, 2006 10:41 am

Beth wrote:I think we allow up to 2mb attachments, so you may have to compress it and make sure to name it something unique. So that it's not public, if you want, you can send it to me using the private function, or send it in an email to support at sourcegear.com and reference this thread.

It was under 1mb and I was sending it via the PM function on this page. No worries; I'll send it via email and we'll try that.

Beth · Post by **Beth** » Wed Oct 11, 2006 8:59 am

The logins/logouts shouldn't make a difference here.

The import log confirms the import is failing on large files -- specifically "data2.cab". How big is that file?

When you checked memory, you checked both the sending and the receiving side, right?

Could you check the standard settings to optimize large file uploads:

If the Vault Server OS is Windows 2003 Server, check in IIS on the Vault server machine for the properties of the web site that hosts VaultService. Make sure Connection Timeout is set to 1800.

If that still doesn't help, you could try to adjust your SQL Server timeout. This is in the
Vault.config file in Inetpub\wwwroot\VaultService. Look for the line
<SQLCommandTimeout>360</SQLCommandTimeout>.
Change the number to something bigger, like 3600.

Also check to see if there is a firewall, router, or other device that could be closing the connection after a certain amount of time or data has passed.

Mike Mestemaker · Post by **Mike Mestemaker** » Wed Oct 11, 2006 9:18 am

Beth wrote:The import log confirms the import is failing on large files -- specifically "data2.cab". How big is that file?

It's in a project I wasn't familiar with, so I had to check. It's big. Over 500mb. I'm going to exclude that project from the import. It appears to be a project someone loaded into SourceSafe solely to serve as a backup of those files.

When you checked memory, you checked both the sending and the receiving side, right?

The sourcesafe db, the import process and the sourcegear service/db are all on the same machine. So, yes.

If the Vault Server OS is Windows 2003 Server, check in IIS on the Vault server machine for the properties of the web site that hosts VaultService. Make sure Connection Timeout is set to 1800.

Checked

If that still doesn't help, you could try to adjust your SQL Server timeout. This is in the
Vault.config file in Inetpub\wwwroot\VaultService. Look for the line
<SQLCommandTimeout>360</SQLCommandTimeout>.
Change the number to something bigger, like 3600.

Was 360; changed to 3600 per your suggestion.

Also check to see if there is a firewall, router, or other device that could be closing the connection after a certain amount of time or data has passed.

Since only one machine is involved, this shouldn't be applicable, should it? We are using hardware firewalls, not software, so there's nothing running on the server that would do that.

The data2.cab find helps me tremendously. I now have confidence that I can do something and trying it again will have a chance at success.

I'm going to schedule another import run without that whole project in a couple weeks and hopefully it should work out better. Thanks for your time/help. If I have any further problems, I'll let you know.