Unable to check in large chunk of files

[gsmalter]

We recently moved our Vault server off-site, and a large check-in I am trying to do is failing. It is very similar to a check-in I successfully performed while Vault was on the LAN, so I'm confident it's a networking issue. However, the check-in in question is only 100MB, which is much smaller than the single 600MB zip file that I successfully uploaded to the Vault server using http (but not using Vault).

The largest single file in the check-in is around 20 megabytes. The Vault Admin tool's max file size is not binding here. When Vault was on the LAN, we successfully checked in single files in excess of 150MB.

The files I am checking in are being newly added to the repository. They are not updates of existing files.

The Vault client is set to use Chunked encoding.

I've attached a text file containing Vault's log file and two of the errors I'm seeing in the Windows Application Event Log (logged by ASP).

[lbauer]

The errors in the server log do point toward a network error in file transfer. The CRC of the file is incorrect indicating the file transfer suffered some errors and the data is corrupt.

To confirm that this is a network problem, can you upload the file from a client on the Vault Server machine itself, using "localhost" for the Vault Server name? This will bypass most network issues.

[gsmalter]

Yes, that works fine. Even external computers on the same LAN can do it successfully, as I mentioned. Basically, if it can do something fast, it works, and if it can't, it doesn't.

It's a network issue in that local and LAN operations work and WAN ones don't, but it's not a network issue in that, as I said, I can upload a 600MB file to the Vault server using FTP or some other non-Vault method.

So, it has to be something about IIS, Vault's IIS configuration, app pool recycling or some similar issue. What would prevent extended transactions from succeeding?

Thank you.

[lbauer]

Something on the network is causing the problem, so start by looking at what might be between the client and the server. Is there a personal Norton firewall or some other Web Proxy server?

If there is a proxy, can you bypass the proxy?

Since you're using chunked encoding, try turning it off. Some some proxies or other devices do not handle chunked encoding.

[gsmalter]

I have some more information on the problem. I zipped up the folder that I was trying to check in and checked it in as a single file using the Vault GUI Client. This worked. So, I was able to perform a checkin of that size/duration without network error.

So, the problem is only occurring when I try to check in the folder (containing multiple files) in Visual Studio 2005. VS2005 gives the following dialog box errors after the server logs the exceptions:

"Files could not be mapped to the Vault project $folder1/folder2/etc...."

Immediately after clicking okay, it says the following:

"Failed to check in one or more files. Some files remain checked out or not added." A quick look shows that no files were successfully checked in.

Any ideas on why one approach works and the other does not?

Thanks.

[lbauer]

Any ideas on why one approach works and the other does not?

Does the sgvault log file report the same types of errors as previously(FailFileInvalidChecksum, exception while calculating the CRC), or does this appear to be a different upload problem?

We've had cases where anti-virus software was interfering with large uploads. Do you have anti-virus software on the client or server? Can you temporarily disable it to see if that makes a difference? If it does, then perhaps you can reconfigure the A/V to not scan Vault's working folders (%TEMP% on the client and the working directory in vault.config on the server).

In the meantime, if you have a large project to upload, perhaps you can upload it from the Server machine so you can at least get started with your development.

Then we can continue to investigate the network issues.

[Don Thimsen]

gsmalter,

Anytime I hear "data corruption" and "network" in the same sentence, the first thing that comes to mind is MTU problem. As packets move through the network, they get chopped up and reassembled. The default MTU value used by most equipment MOSTLY works okay, but not always...

Our company requires that all clients connecting to our network lower their MTU value to 1002 (nothing magic about this value - it's just our standard). This has solved many headaches like you're describing. And it's easy to try... Google on "DrTCP MTU" and download the application that can set the MTU value for the client's network card.

It takes about five minutes to find out if you're working on an MTU problem...

Don

PS. Not affliliated with sourcegear

[gsmalter]

There is no anti-virus software on the client or the server.

I successfully checked in the group of files (without zipping them) using the Vault GUI Client. And I tried checking in the single zip file using Visual Studio 2005, and it failed. The problem is clearly checking into Vault from Visual Studio. I don't think there is a general network problem.

Interestingly, when I check in the single zip from within Visual Studio, it fails (with the same error as before) much more quickly (a minute versus an hour).

The errors in the event log and vault log are the same as previously noted.

Development isn't being held up by this, but I'd like to understand why it's failing so it doesn't become a bigger problem later.

Thanks,

Greg

[lbauer]

There may be something different about the way Visual Studio sends the files.

You could enable IDE logging to see if there's any useful information:

http://support.sourcegear.com/viewtopic.php?t=2898

Another thing to try is to do a checkin of the exact same files (all unzipped) from the IDE client and then again from the GUI Client. Have logging enabled for the GUI Client and then compare the logs for each upload to see how they differ when one succeeds and one fails.

http://support.sourcegear.com/viewtopic.php?p=5375

Log "all" classes. Be sure to stop logging after you've captured the upload, as the file can get very large.