Obliterate via running stored procedure directly?

[CCS]

In the Vault GUI Client, folder properties (select a folder, right click and select Properties) will show you the number of files in a folder, plus the number of any deleted files. Does that help?

not at all - it does not since
a) it can only show there properties for folders that are NOT deleted, and I need this info for DELETED folders!
b) it only shows a number of files right in the folder I'm checking properties for - but not the TOTAL number of files in it including subfolders.

[ian_sg]

It would be difficult to come up with a script that would provide the progress information you're looking for. As you've seen, obliterate spends the vast majority of it's time doing SELECTs to determine that the DELETE is safe. So you'd see what looks like no progress for hours and hours, and this would be "normal". I'll take a look at it, but I can't promise I'll produce anything useful.

[CCS]

Ok - we have a new problem now
we had to kill the query yesterday after almost 90 hours with 0 progress so today we restored the DB from the backup but surprisingly all the folders & files that were added AFTER the backup had been taken are still showing up in Vault GUI
We thought it's because of clients caching the repository tree, but it turned out not to be the case since we cleared out local Vault's cache, received a message 'retreining repository structure from the server' and yet again the files/folders that aren't supposed to be there show up!
when we check their history it is blank.
seems like something is also cached on web server? How can we refresh all that state?

[ian_sg]

You need to restart IIS on the Vault server for it to refresh its cache. When that's done, you'll want to clear out the client caches again as well.

[CCS]

Well - the saga continues...
After we've killed the process & restored the DB from the backup we decided to break the process into smaller chunks by restoring deleted folders, then obliterating their sub-folders one-by-one (first in QA, then after timing it in production)
everything was going as planned (some sub-folders were taking 90 hours to delete though!) until we ended up with an empty top folder that we restored at the beginning. Properties for this folder were showing no deleted sub-folders, and admin tool was showing no folders to obliterate.
we also modified SP obliterateobject to include logging in it
when we proceeded to obliterate the top folder all of a sudden we started seeing that some folders that had been MOVED from this folder to a different location in the tree were popping up in the log.
we let the query run for 166 hours before killing it - and it was still cycling through sub-folders that were NOT even supposed to be there.
something like this:
$/foo
$/bar
$/bar/test

we had moved test to foo first, then deleted bar & tried obliterating it (it was showing as empty) - then we saw 'test' folder in spobliterateobject logs.

How come?! what's happening with folders that got moved?

[ian_sg]

Vault tracks all versions of files and folders that were ever checked in. That means when you obliterate something, obliterate scrutinizes the entire history of that item. I don't know the details of what the logging you've added is doing, but the fact that you're seeing obliterate consider items that were once under that folder, but are not now, is not at all surprising.

something like this:
$/foo
$/bar
$/bar/test

we had moved test to foo first, then deleted bar & tried obliterating it (it was showing as empty) - then we saw 'test' folder in spobliterateobject logs.

Let's say you undeleted $/bar, and then somebody wanted to retrieve a historical version of it that included $/bar/test. Vault lets you do that, so obliterate has to consider that possibility before it deletes relevant data. As it is, Vault is looking to see if the data from $/bar/test now exists elsewhere, so it doesn't delete the file data that's still needed, as it's referenced now under $/foo/test.

Make sense?

[CCS]

not really - since there is no $/bar/test anymore - BEFORE bar was deleted test had been moved, and so had nothing to do with bar at the deletion time
so it's impossible to restore $/bar/test since move operation appears to be irreversible - at least there's no trace left in a form of deleted folders or anything like that.

we have already obliterated all sub folders, but the folders that were moved
supposedly have nothing to do with the folder of interest, and so should not even be considered shouldn't it?

how else can I break it down any further?

[ian_sg]

we have already obliterated all sub folders, but the folders that were moved
supposedly have nothing to do with the folder of interest, and so should not even be considered shouldn't it?

Vault, like any good source control system, tracks all versions of files and folders that are ever checked in. When you obliterate a folder, you're obliterating every version of that folder that ever existed. Anything that ever existed under a folder you're obliterating is relevant to the obliterate.

[CCS]

also can move operation be reversed? if not then what's the point of it to even be "considered"?
This doesn't help much so I have to repeat the original question:

"how else can I break it down any further?"

[ian_sg]

The GUI doesn't allow rollbacks of moves, because you can't roll back a folder at all. Luckily, the Vault server tracks all versions of all files and folders ever checked in, so we could implement this and the databases of existing installations out in the world would contain the data we need to make it work retroactively. You can retrieve historical trees via the GUI, as an example of how that data is used today. (Using your example, you can retrieve the historical version of $/bar that still contains foo.)

In terms of answering your question, "how can I break it down further," I'm not sure what you're asking. Are you asking how can you make the obliterate smaller? I don't know that you can. Actually, by moving things in or out you're making it "bigger" because you're adding more versions of the folder for obliterate to consider.

[CCS]

what are the following SPs for?

spbubblefolders
spsavefolderentrydeltas
spsavecurrenttreefolders

admin tool calls them after the call to spobliterateobject, but we didn't when we were obliterating manually

[CCS]

isn't the turnaround on this forum is supposed to be a day or so?
could you please provide the info since these SPs are called by the admin tool among with obliterate so we're not sure if we need to do it as well when running SPs directly.

[ian_sg]

These are used internally by Vault to update the tree and its history after a transaction.

Edit: You should be okay not running them.