Archiving options and performance of a large repository

[Igor.Samoylenko]

Hi,

Our source database is getting quite large - the database backup file for sgvault is around 15Gb. The general performance of Vault is now starting to suffer with many developers complaining about slow start up times for the client, very sluggish Visual Studio (when using Vault) etc.

At the moment, we neither archive nor obliterate anything in the repository. I would like to explore what we can do speed things up.

A few things to note about our source structure: we make extensive use of branching (and merging) and labelling (as a part of our internal automated build service). The vast majority of files under source control are text files (as opposed to binary). Typically, a source tree is branched for a development project and then merged back into trunk on release.

As far as I can see we have several options:

1. Archive old(er) source trees by moving them into a separate repository or into a separate database altogether. We don't really have a requirement to necessarily have immediate access to old source. As long as there is a way to get to it, it will do (compliance etc).

The problem here is that I cannot see any practical way to do it. There is a utility to export/import folders but it looks extremely time consuming to actually do it from what I can see.

2. If 1. is not simple, we can consider just taking a copy of the database on the tape and then simply deleting/obliterating older versions (older branches) from prod.

Sounds simple enough but will make it quite difficult to get access to archived folders (though I am sure we can think about something if this is the only option). Obliteration is also very slow as I remember from the last time I used it.

Can you let me know what you think please? Are we missing something here? Is there anything else we can try? Is 15Gb a lot for the source database (it is quite large to be honest and I don't really understand why I have to say)? Is there anything else we can do to improve performance without archiving source?

Thanks a lot for any help you can offer.

Regards,

Igor.

[lbauer]

Yes, 15 GB, is large, but we have customers with larger databases. The database will grow over time as you add files and history and as branching makes the database more complex.

If you have performance issues, the first thing to do is make sure your current hardware is adequate and that you run database maintenance, and that the drive with the SQL Server database is defragmented.

Suggestions for optimal performance here: http://support.sourcegear.com/viewtopic ... 163#p16163

If you think you might want to archive using Vault's Folder Export\Import, we caution against using Obliterate first, as it removes history, and the Export\Import Tool needs to recreate history to function properly.

You can delete unneeded labels to reduce database size.

You can delete (but not obliterate) unused branches to improve performance. The branches can always be undeleted if you need them later.

[Igor.Samoylenko]

Thanks Linda.

I've enabled client logging to see where the performance issues are. I have noticed several things:

1. There is a large section to do with pending server namespace changes:

Code: Select all

11/02/2009 16:58:49 <mrd>: [GUIClientWorkerThread:4600] Pending server namespace changes on startup:
11/02/2009 16:58:49 <mrd>: [GUIClientWorkerThread:4600] 	Rename: $/<file_path_to_rename> to $/<file_path_to_rename_to>
11/02/2009 16:58:49 <mrd>: [GUIClientWorkerThread:4600] 	Move: $/<folder_path__to_move> to $/<folder_path_to_move_to>
[...516 of these....]

which contains a large number of Rename and Move operations. Is this the full list of all pending rename and move requests currently on the server? Why is my client requesting them every time?

Are these orphaned requests? is there a way to find this out on the server? And how do I get rid of them if they are indeed orphaned?

2. There are several places where the client paused for a few seconds and I cannot see why from the log. For example:

Code: Select all

11/02/2009 16:58:57 <emailview>: [Main:4888] Active repository changed to Source
11/02/2009 16:59:03 <mutex>: [GUIClientWorkerThread:4600] Took mutex 1316

Why is there a 6 sec pause here?

3. Is there a utility to check a repository?

Regards,

Igor.

[lbauer]

Is this the full list of all pending rename and move requests currently on the server? Why is my client requesting them every time?

Are these orphaned requests? is there a way to find this out on the server? And how do I get rid of them if they are indeed orphaned?

It looks as though there were changes made on the server/database side, but the client has not yet retrieved these through a Get Latest.
If you think there are orphaned items (items stuck in the pending change set), you can undo the transactions or better yet, reset the client-side cache:

http://support.sourcegear.com/viewtopic.php?t=6

Regarding the 6-second pause in the client log, we don't have enough info to go on. You can send us more of the log plus info about what the user was doing at the time. Send the log zipped up to support at sourcegear.com, Attn: Linda. Please include a link to this forum post.