Executing the SQL command directly took 16 hours and 33 minutes. I just took a look at my full DB backup file and it shrunk from 5.1GB to 3.2GB.
So for people in my situation where I had a decent sized VSS import where some of them had failures and had to be deleted and reimported (I'm sure there's some other situation this might arise), is there any way you can either take a look at the performance of that stored proc or publish an easier way to go about manually obliterating some files?
Obliterate failing
Moderator: SourceGear
I just installed 3.0.6 (both the server and the client) which has a note "Improved performance of Obliterate", yet it's still incredibly slow for me.
We have hundreds of those stupid VS.NET temp files, which I selected in the obliterate and it continually times out. I even tried to select only about 50 of them and obliterate them all at once and it times out before it shows the "are you sure" dialog.
Would adding a "would you like a confirmation" yes/no box be feasible so I at least wouldn't have to wait for the confirmation box to appear before starting the obliterate?
Do you guys still have improving the performance of obliterate in your scopes?
I also had one developer trying to get around the "can't branch on an imported label because the label doesn't branch the correct files" bug/feature (due to label promotion) and he started sharing, pinning, then branching one folder to see if he could get the correct set of files (similar to some of the weirdness you can do with VSS) and I have 6 branched folders (mind you, they had no history other than the share/pin/branch) that Vault won't let me obliterate. Some message about not being able to obliterate because all branches aren't deleted (and I can't get the message at the moment because Vault's failed obliterate has my SQL Server at 25% and any further requests while it's running always fail).
We have hundreds of those stupid VS.NET temp files, which I selected in the obliterate and it continually times out. I even tried to select only about 50 of them and obliterate them all at once and it times out before it shows the "are you sure" dialog.
Would adding a "would you like a confirmation" yes/no box be feasible so I at least wouldn't have to wait for the confirmation box to appear before starting the obliterate?
Do you guys still have improving the performance of obliterate in your scopes?
I also had one developer trying to get around the "can't branch on an imported label because the label doesn't branch the correct files" bug/feature (due to label promotion) and he started sharing, pinning, then branching one folder to see if he could get the correct set of files (similar to some of the weirdness you can do with VSS) and I have 6 branched folders (mind you, they had no history other than the share/pin/branch) that Vault won't let me obliterate. Some message about not being able to obliterate because all branches aren't deleted (and I can't get the message at the moment because Vault's failed obliterate has my SQL Server at 25% and any further requests while it's running always fail).
Server log entry for trying to obliterate the share/pin/branched folder:
----4/9/2005 3:22:55 PM admin--s03cms01.globeranger.com.pvt(127.0.0.1)--SSL Disabled VaultLib.VaultRequestObliterate returned: FailObliterateBranchExists
(which is odd, since there are 6 that need obliterated and we had deleted all of the failed branches)
----4/9/2005 3:22:55 PM admin--s03cms01.globeranger.com.pvt(127.0.0.1)--SSL Disabled VaultLib.VaultRequestObliterate returned: FailObliterateBranchExists
(which is odd, since there are 6 that need obliterated and we had deleted all of the failed branches)
There's a lot going on here, and I'll try to address it all. If I misunderstand, or leave something out, call me on it.
1. Obliterate performance was sped up for everything after the confirmation dialog. If your requests take a long time before the confirmation dialog comes up, they will take a long time after you hit ok.
2. Did you notice that we added the obliterate command to the command line client? This will let you obliterate without having to click the ok button in the confirmation dialog. Just a convenience, really.
3. Selecting 50 files to obliterate means 50 different calls into the stored procedure to obliterate. Is it fast enough when you select one or two of the files?
4. If you have a folder which you can't obliterate, because it claims to be branched, try undeleting it, and do a show history on the folder looking for branch operations. It's possible that you've come across an instance where obliterate is overly cautious.
5. In general, obliterate is a low priority for us. It's not a common operation, and because we go to so much trouble to store compressed data, it's usually not a big win. The one exception is when you've deleted a failed import and need to oblterate it, which as you saw gave a big size decrease. The cases which don't give large benefits are the stupid VS.Net temp files (which are 0 bytes long and only contain one version) and branches (which don't create any new file data in the tree).
1. Obliterate performance was sped up for everything after the confirmation dialog. If your requests take a long time before the confirmation dialog comes up, they will take a long time after you hit ok.
2. Did you notice that we added the obliterate command to the command line client? This will let you obliterate without having to click the ok button in the confirmation dialog. Just a convenience, really.
3. Selecting 50 files to obliterate means 50 different calls into the stored procedure to obliterate. Is it fast enough when you select one or two of the files?
4. If you have a folder which you can't obliterate, because it claims to be branched, try undeleting it, and do a show history on the folder looking for branch operations. It's possible that you've come across an instance where obliterate is overly cautious.
5. In general, obliterate is a low priority for us. It's not a common operation, and because we go to so much trouble to store compressed data, it's usually not a big win. The one exception is when you've deleted a failed import and need to oblterate it, which as you saw gave a big size decrease. The cases which don't give large benefits are the stupid VS.Net temp files (which are 0 bytes long and only contain one version) and branches (which don't create any new file data in the tree).
Thanks for the reply. The main concern I have with this is that Vault becomes and stays permanetly "dirty". I don't like my recycle bin to stay full for very long and I repeatedly empty my Outlook inbox because that crap is just taking up wasted space.
I see deleted files in the same regard. Vault forces me to not be able to permanently delete something from the main UI. It creates a large list of waste in my database (granted, it's not a lot of waste compared to files, but I do have numerous deleted labels, folders, branches, etc) and I simply cannot find the things that would be appropriate to obliterate. There are roughly 1204 items in my obliterate list for one repository (calculated by 14 per page times the number of pages when I click the scrollbar).
Whether or not my 1204 deleted items are taking up any space isn't really relevant to me as Vault is basically just trashing some stuff and basically refuses to empty its own recycle bin without me spending 1204 clicks (since I can't do them too many at a time because whenever I try to get too many it freaks out) to get rid of them (taking a cursory glance, at least 1/2 of everything in there are deleted labels like so: $/EdgeServices/Source/...../...DiscoveryAgent.cs_VSSImport0).
Also, what happens when I create a folder named "X" then delete it, then add a folder named "X" again and try to undelete the previous folder? I don't see that in the documentation and that's where I'm at currently with the 4 folders that won't obliterate.
I see deleted files in the same regard. Vault forces me to not be able to permanently delete something from the main UI. It creates a large list of waste in my database (granted, it's not a lot of waste compared to files, but I do have numerous deleted labels, folders, branches, etc) and I simply cannot find the things that would be appropriate to obliterate. There are roughly 1204 items in my obliterate list for one repository (calculated by 14 per page times the number of pages when I click the scrollbar).
Whether or not my 1204 deleted items are taking up any space isn't really relevant to me as Vault is basically just trashing some stuff and basically refuses to empty its own recycle bin without me spending 1204 clicks (since I can't do them too many at a time because whenever I try to get too many it freaks out) to get rid of them (taking a cursory glance, at least 1/2 of everything in there are deleted labels like so: $/EdgeServices/Source/...../...DiscoveryAgent.cs_VSSImport0).
Also, what happens when I create a folder named "X" then delete it, then add a folder named "X" again and try to undelete the previous folder? I don't see that in the documentation and that's where I'm at currently with the 4 folders that won't obliterate.
Once again, there is a lot going on here.
1. "Deleted Labels" These are files that are needed for labels imported from VSS. You should not oblterate these files unless you are sure that you don't need the labels that they are in. Obliterating these files will permanently alter the labels that were imported. I strongly advise against obliterating them.
2. Undelete will not work if there's already an item with that name in the folder. The current state of the tree should not affect obliterate at all. Can I assume that the folders that are failing are the pin/share/branch folders that your coworker was working with?
1. "Deleted Labels" These are files that are needed for labels imported from VSS. You should not oblterate these files unless you are sure that you don't need the labels that they are in. Obliterating these files will permanently alter the labels that were imported. I strongly advise against obliterating them.
2. Undelete will not work if there's already an item with that name in the folder. The current state of the tree should not affect obliterate at all. Can I assume that the folders that are failing are the pin/share/branch folders that your coworker was working with?
1. This is how the import tricked the VSS import to use the label promotion, etc? It adds a file, labels, then deletes the file right afterwards?
2. Yes, the folders that are problematic are the ones that had all the weird-ass stuff done to them in an attempt to branch, however none of them were shared/branched from another one of them. The process always went:
* share from the main trunk
* pin the share
* branch the share
* didn't work, delete
2. Yes, the folders that are problematic are the ones that had all the weird-ass stuff done to them in an attempt to branch, however none of them were shared/branched from another one of them. The process always went:
* share from the main trunk
* pin the share
* branch the share
* didn't work, delete
1. Yes, this is how Vault includes files in a label that were not imported (if they've been moved or deleted in VSS).
2. I'm not sure what's going on with the folders that you can't obliterate. There are a couple of known cases where obliterate reports that an item has an existing branch, but that branch would be obliterated in the same action. It's possible that you are seeing one of those cases.
2. I'm not sure what's going on with the folders that you can't obliterate. There are a couple of known cases where obliterate reports that an item has an existing branch, but that branch would be obliterated in the same action. It's possible that you are seeing one of those cases.
-
SteveMarriott
- Posts: 6
- Joined: Thu Oct 21, 2004 11:33 am
Obliterate stored procedure running for 15 days straight.
I too am trying to obliterate a large number of files. I have been following the instructions here and now have a SQL stored procedure that has been executing for over 15 days (360 hours) now.
I am wondering if it will ever finish (successfully), and when. I have stopped the backup schedule for my Vault db while this is running so as not to interfere with the execution of the stored procedure. My database size is around 9 GB and the repository that I am trying to delete is probably about 20% of the db content. Both CPU's in my Vault SQL server are running about 25% ever since the stored procedure was started.
Not sure if I should kill the process...
Any suggestions? Words of encouragement?
I am wondering if it will ever finish (successfully), and when. I have stopped the backup schedule for my Vault db while this is running so as not to interfere with the execution of the stored procedure. My database size is around 9 GB and the repository that I am trying to delete is probably about 20% of the db content. Both CPU's in my Vault SQL server are running about 25% ever since the stored procedure was started.
Not sure if I should kill the process...
Any suggestions? Words of encouragement?
It's probably not a good idea to stop the process. It might leave the database in an uncertain state.
For future reference -- if a an import into a new repository isn't successful, deleting the entire repository is the fastest way to remove data from the SQL database.
For future reference -- if a an import into a new repository isn't successful, deleting the entire repository is the fastest way to remove data from the SQL database.
Linda Bauer
SourceGear
Technical Support Manager
SourceGear
Technical Support Manager