Adding very large files?

[ismangil]

What settings should we tweak if we want to upload very large (up to 350MB each)? IIS seems to give up too soon. The client and server is already on the same LAN, so network wise as fast as it can go.

This is using 3.0.6.

[jclausius]

Within the Admin Tool, there is a Server option which controls file upload size. Try setting this to 409600.

Note, when IIS uploads a file the entire file takes up memory on the server, so be weary of memory related issues on the Vault server.

[lbauer]

If you have Window 2003 Server and IIS 6, you might also check the Connection Timeout under the website properties->Connections.

The default for this setting is 120 seconds, which is too low when committing large files.

[ismangil]

Modified the setting as instructed.

Adding a file 115MB, message says transaction started, adding file, progress bar goes up towards 100%, and then it is stuck at 100%, with the status saying "Uploading file...". I've left it for five minutes and it is still stuck.

Any ideas?

[ismangil]

If you have Window 2003 Server and IIS 6, you might also check the Connection Timeout under the website properties->Connections.

The default for this setting is 120 seconds, which is too low when committing large files.

Connection timeout is set at 1800 seconds.

[ismangil]

Modified the setting as instructed.

Adding a file 115MB, message says transaction started, adding file, progress bar goes up towards 100%, and then it is stuck at 100%, with the status saying "Uploading file...". I've left it for five minutes and it is still stuck.

Any ideas?

It's not stuck anymore, but it threw up an error, cannot upload.

[jclausius]

Can you look at your Vault server log and the Windows Event Viewer (System / Application)? Do you see any errors?

Also, you may want to have a look at this KB article - Troubleshooting add/checkin upload failures

[ismangil]

Can you look at your Vault server log and the Windows Event Viewer (System / Application)? Do you see any errors?

Not that I can see. I've located in the log the point where we set the upload size as directed:

System Started
Version 3.0.6.2856
Cache Level = 1
DataBase Buffer Size (KB) = 256
LogFile Path = C:\WINDOWS\Temp\sgvault
Log Level = Quiet
Archive Log = Weekly
ReverseDNS Lookup = True
Maximum HTTP Request Length = 409600
Overwrite Log on Startup = False
Session Timeout = 4320

No errors in all Event Log (App,Security,System).

Also, you may want to have a look at this KB article - Troubleshooting add/checkin upload failures

I will take a look now.

[ismangil]

Also, you may want to have a look at this KB article - Troubleshooting add/checkin upload failures

I will take a look now.

Everything seems to check out in the KB article. Anyway we have successfully added thousands of other files with hundreds of folder.

I have tried the upload again and got some new error messages (see below). It may be worth noting that the file is a binary file. The upload seems to work fine at first, it progresses to 100% and then it failed. This time I look closely at the performance graphs for the server (which has 4GB of RAM and is dedicated to host both the Vault app server and its db server) and everything seems OK (no 100% CPU usage, lots of memory left etc)

[26-04-2005 09:18:41] Preparing data to begin transaction
[26-04-2005 09:18:46] Beginning transaction
[26-04-2005 09:18:46] Add [filename masked]
[26-04-2005 09:19:49] Upload for item [filename masked] failed, retrying...
[26-04-2005 09:20:51] Upload for item [filename masked] failed, retrying...
[26-04-2005 09:21:52] Upload for item [filename masked] failed, retrying...
[26-04-2005 09:21:52] Upload for item [filename masked] failed too many times, aborting transaction.
Please verify your network settings using the Options dialog under the Tools menu in the Vault GUI Client.
The specific error was "The underlying connection was closed: An unexpected error occurred on a send."
An exception was encountered during the transaction. Exception: The underlying connection was closed: An unexpected error occurred on a send. at System.Net.HttpWebRequest.CheckFinalStatus()
at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
at System.Net.HttpWebRequest.GetResponse()
at VaultClientOperationsLib.ClientInstance.UploadItem(ChangeSetItem item, String txID, Int64 totalBytes, Int64& bytesWritten, Byte[]& streamBuffer, Int32& bytesWrittenThisFile, Boolean bIsImport)
at VaultClientOperationsLib.ClientInstance.Commit(ChangeSetItemColl givenItems, Boolean keepCheckedOut, Boolean removeLocalCopy, Boolean bIsImport, DateTime dateImport, Int32 nUserIDImport, Int64& nRevID)
[26-04-2005 09:21:53] Ending the transaction
[26-04-2005 09:21:53] Transaction failed

[ismangil]

Also, you may want to have a look at this KB article - Troubleshooting add/checkin upload failures

Re-reading the article, it mentions ChunkedEncoding. I have already verified that it is still disabled (as it is the default), as the KB suggesteted.

On a hunch I enabled it, because I am on a local network without any proxies.

What do you know, the upload went OK!

Thanks for all the support!

[jclausius]

There are so many variables that apply to file uploads, it can sometimes be a challenge to figure out which one is the culprit.

I'm glad you were able to isolate the issue.