Splitting up a huge repository into smaller ones

If you are having a problem using Vault, post a message here.

Moderator: SourceGear

Post Reply
mscheuner
Posts: 16
Joined: Tue Nov 21, 2006 4:55 am
Location: Berne, Switzerland
Contact:

Splitting up a huge repository into smaller ones

Post by mscheuner » Mon May 31, 2010 2:51 am

Folks,

We're at a point where our repository is just too big as a single repository. We need to split it up into several, independent smaller repositories.

But that seems to be next to impossible in Vault.... can this be?? I mean - it's all SQL Server tables, right? Isn't there any way to do an "admin cleanup" and extract certain paths in our main repository into their own, separate, new repository?

Repository size is one of the major performance killers.... with our 40'544 folders and 307'047 files, speed is definitely becoming a major issue (and a source of aggravation for a lot of the devs in our company). Even just launching the GUI client can take several minutes - not to mention checking out a given revision for build on our build servers.

The smaller repositories still feel quite snappy in comparison to our huge main repository....

Beth
Posts: 8550
Joined: Wed Jun 21, 2006 8:24 pm
Location: SourceGear
Contact:

Re: Splitting up a huge repository into smaller ones

Post by Beth » Tue Jun 01, 2010 5:20 pm

I would recommend you try our Vault Export/Import tool. That installs automatically with the Vault client. It will export out to a file, and then you import it into either a new server or repository. Then delete the item you exported if you wish.

One thing to not is that Export/Import and Obliterate are not compatible, so I would recommend not using Obliterate. If you need to reclaim space in that repository, then only perform it after all the exporting you need to do is complete. Obliterate takes away history that Export/Import needs to work, so once it's done, there's a very high likelihood of not being able to Export from that repository again.
Beth Kieler
SourceGear Technical Support

mscheuner
Posts: 16
Joined: Tue Nov 21, 2006 4:55 am
Location: Berne, Switzerland
Contact:

Re: Splitting up a huge repository into smaller ones

Post by mscheuner » Wed Jun 02, 2010 3:22 am

Beth wrote:I would recommend you try our Vault Export/Import tool.
We had tried a few things with this - but without much luck - it never fully worked. It's a very annoying and complicated and overly involved process, in my opinion. A hack at best - not a solution - for this problem.

I still believe there ought to be a "in-database" process to do this - doesn't anyone else end up with large repositories and feel the urge to split them up??

Marc

Beth
Posts: 8550
Joined: Wed Jun 21, 2006 8:24 pm
Location: SourceGear
Contact:

Re: Splitting up a huge repository into smaller ones

Post by Beth » Wed Jun 02, 2010 12:47 pm

Right now this is the tool for the job. I can take a feature request for something else.

What kind of problems were you having during the Export/Import? Did you have obliterates getting in the way, or was it just taking too long?
Beth Kieler
SourceGear Technical Support

pogaz
Posts: 11
Joined: Wed Apr 28, 2010 9:39 am

Re: Splitting up a huge repository into smaller ones

Post by pogaz » Fri Apr 22, 2011 3:43 pm

Marc and/or Beth, was this issue ever resolved and, if so, how?

I am having the same issue: I want to split up a large repository by exporting out diretories to new repositories. I have tried the Import/Export tool and it takes a very long time (so long that I had to cancel the process...and I was only processing a just-created single directory with a single 1KB text file in it).

I don't know if Obliterate was ever run on this repository (I'm the "admin" right now and haven't done it so hopefully previous person(s) didn't do it - is there a way to check?), but I sure do hope that, if it was run, there is another way to export out the files with history.

Paul

Beth
Posts: 8550
Joined: Wed Jun 21, 2006 8:24 pm
Location: SourceGear
Contact:

Re: Splitting up a huge repository into smaller ones

Post by Beth » Mon Apr 25, 2011 8:29 am

I am replying to your other post over in this thread: http://support.sourcegear.com/viewtopic.php?f=5&t=13540
Beth Kieler
SourceGear Technical Support

Post Reply