VSS Import Very Slow and Resource-Intensive

[JohnWSaundersIII]

I have used VSS Import to import most of my projects. I'm now working on the final, and largest project. It contains 499Mb in 2,985 files and 115 folders.

The import is taking place on a dual 1.8Ghz XEON system with 1Gb of RAM. The C:\ drive starts off with 21Gb free.

The import takes all of the physical memory, frequently pegs the CPU, and uses at least 21Gb of disk space (I had to stop it after that). Before I aborted it, the import tool was telling me that the import would take 1 1/2 days to complete.

So, is there some way to have the import put its files on some other disk? We have a disk with 34Gb which we could fill up if necessary.

I assume there's no way to modify the CPU and memory usage, but please correct me if I'm wrong.

I might want to import this project in pieces. But that would involve importing subdirectories, which would miss out on files in the root of the project. Is there a way to tell the importer to only import the files in a directory, but not to recurse?

If I had to import subdirectories, I would have over 20 to import. Is there a way to use the import tool from the command line?

I'm a new, but desperate, Vault user, so please forgive me if I'm asking silly questions. SourceSafe has failed us for what we hope is the last time, so we're rushing to get everyrhing imported ASAP.

Thanks,

[ericsink]

You are correct, VSS Import can be extremely resource intensive.

All I can suggest is that you get more RAM and more disk.

On the bright side, your estimate of 1.5 days is one of the shortest I've seen. We've had people facing imports of over three weeks. I think I heard of at least one group that actually waited that long for it to complete.

[JohnWSaundersIII]

Thanks for the response.

Is there nothing I can do about the disk space? No way to move it off of drive C? Would the import tool mind if I made %appdata%\SourceGear be a junction point leading to a different disk volume?

Also, being new here, and having just used Source unSafe, I've been terrified of interrupting the import. How safe is that? Also, if I interrupt it and then start over, would it start from the beginning? If I deleted the cache files, would it recreate all of them?

Finally, if I were forced to do the import one subfolder at a time, is there any way for me to import files from the root of the project? $/Project/readme.txt, for instance?

Thanks again for the rapid response.

[ericsink]

Thanks for the response.

Is there nothing I can do about the disk space? No way to move it off of drive C? Would the import tool mind if I made %appdata%\SourceGear be a junction point leading to a different disk volume?

Also, being new here, and having just used Source unSafe, I've been terrified of interrupting the import. How safe is that? Also, if I interrupt it and then start over, would it start from the beginning? If I deleted the cache files, would it recreate all of them?

Finally, if I were forced to do the import one subfolder at a time, is there any way for me to import files from the root of the project? $/Project/readme.txt, for instance?

Thanks again for the rapid response.

I can try to answer a couple of these:

I know of no way to change the disk which is being used, sorry.

I would be quite suspicious of interrupting the import. It *might* work, but it's probably best to assume that it's unsafe.

[heath]

The VSSImport Tool puts all of its files in the folder defined by %TEMP%. If you change where this points, then you can change where the files go.

However, the only time it gets large amounts of data at a time is when it imports labels. The label verification step will get the label from both VSS and from Vault to compare the end result. Obviously, if your labels are large then the space required to verify them will be even greater.

In all other cases, the VSSImport Tool will get a file, upload it to Vault and then delete it, so you shouldn't see a large number of temp files hanging around.

The only other file writing that is done is the log file. It is placed in the folder where the import tool is started from. By default, this is the application's installation folder. If this is a problem, just change the shortcut's start here path to somewhere with more space.


To answer your question about importing the files in the root project, no, the import is only recursive. However, you could try archiving the project to a separate database and then import that. If you go this route then I would do that first, followed up by all of the subprojects.

[JohnWSaundersIII]

Thanks for the replies.

When it asks for the mapping of labels to virtual folders, if I don't give it a mapping for a label, will it import the label? If not, I'll try again, but without the older labels.

If necessary, I'll take your advice about archiving just the project.

BTW, when I came in this morning, it had 24hrs and 1.64 Mb left...

[Rick G]

When I went to import our 1957 files in 85 projects we had 4 years worth of check ins. I too had a bleak projected import time.

I copied my VSS database to my local hard drive, ran Archive in the Admin utility and archived everything older than 3 months using the Dmm/dd/yy version parameter.

I also noticed that my compiled EXEs that I check in after the nightly build script were taking a very long time. So I deleted those out of my local copy of the VSS database. I deleted a few other build script status message checkins and I was able to get my import time down to 2 hours.

The moral of the story is the frequency of different file check ins and the size of the files checked in have a big impact on the import time.

Also, we haven't really lost our historical information, we are keeping our original VSS database around for that. We can still do a diff using the old VSS tools. So if we really need to compare to a really old version of a file we still can.

I hope this helps.

-Rick

[Chris Leslie]

Hi
We are doing an evaluation of Vault at the moment, and are attempting to import an ~10Gig VSS store.

PS We could break this into smaller chunks

Is there a way to do the import into vault and ignore the file History and Labels ( We would probably do a final label on VSS and only import that label.

I suppose the real question is what is the performance impact of Importing file History.
Or should we investigate doing a VSS archive first.

We would probably be happy just to get the latest code streams into Vault.
We still want to use the Import Tool so that we can keep all our file shares etc.

Thanks in advance
Chris

[jeremy_sg]

If you have lots of folders that have all of their items shared (a fairly common use of sharing), then you probably would want to set up folder share anyway. If that is the case, I would recommend the get-from-VSS/add-to-Vault approach. After the add, you can go through and manually establish the folder shares.

You could also try archiving off any history, choosing to delete from the repository. Import the resulting VSS DB, and see if that will work for you. Let us know which method you choose and if you're having success.

[Chris Leslie]

Hi Jeremy
Thanks For the Reply,

If you have lots of folders that have all of their items shared (a fairly common use of sharing), then you probably would want to set up folder share anyway.

They are all file shares(DLL's, ASPX) between dev folders and release folders ( I.E. we don't share the codebehind files to release) There are 100's of shares, and would be to painful to do manually.

We are tesing an import of only 1 label at the moment.
Took VSS labeled it, and will attempt to import that latest label only.

Chris

[jeremy_sg]

Our import tool doesn't have the ability to import a label without importing the full VSS database. I would recommend you investigate ways to archive off the hiistory of the 10 gig DB and import the resulting (hopefully way smaller than 10 gig DB). For information on archiving off portions of the VSS DB, see

http://msdn.microsoft.com/library/defau ... abases.asp