File remains "Renegade" despite of "Get Lates

[TeamWiSE]

This effect only happens with one specific file. Nothing special about it, a simple plain old C include file. After a VSS import into Vault the file is shown as renegade. In order to avoid timezone problems the CRC-option for determining differences is active. In a folder difference the file is also shown as different. Using the file comparison, the file is considered identical! A binary compare also reveals no differences whatsoever. Binary comparison of the VSS-version with the Vault-version show no differences either. With a Get latest version and specifying overwrite the file is still renegade though identical. Deleting the file and getting it again still shows renegade.

My guess is, that the bytes stored in SQL Server after the VSS import are different from the bytes produced in getting the file, thus producing different CRC's. Note that EOL conversion is explicitly turned off. Any ideas?

[Beth]

Is your file size larger than the limit you may have placed on files to use the CRC for? Right under where one checks the CRC option (in Tools - Options - Local Files) is the option to only perform the CRC checks on files under a certain size.

If your file size is fine or you didn't place any file size limites, then what should clear this up is setting a fresh working folder on your disk and performing a Get Latest.

[TeamWiSE]

no, it's not beyond file limits (limit is 10MB, filesize is 27KB). we've also tried with a fresh working folder - made no difference. in the meantime we've edited the file in question and checked it in again. now everything seems to be fine. we even did a rollback to the previous version - everything fine. but then again, rollback in vault doesn't really rollback but creates a new version. if we get the old (identical) version, the status changes to "needs merge", although no differences are shown and a merge is also a no-op. very mysterious and this doesn't really feel good in such a sensitive development component.

[jclausius]

In order to avoid timezone problems the CRC-option for determining differences is active.

You shouldn't have to change this option to appease Vault. All date/time info is sent in UTC, so any timestamps on the client are the localized timestamps of the server.

Any ideas?

Over the five year history of Vault, there have been zero reports of any problems with CRCs or detecting changed files. So it would be surprising if there is some problem with the code itself.

When you diffed, what file was used for the comparison? Another idea would be to do a get on a totally different machine, and diff the two files.

You could try to delete the file out of the working folder, and re-try the get operation.

Also, if there is a small amount of history of the file, I wonder what would happen if you re-add the file w/ all the changes?

[TeamWiSE]

You shouldn't have to change this option to appease Vault.

We use the CRC-option since we had one other case of renegade though identical since apparently the rounding of the filetime in vss and vault was in this case slightly different. I could try to reproduce this one as well.

Over the five year history of Vault, there have been zero reports of any problems with CRCs or detecting changed files. So it would be surprising if there is some problem with the code itself.

We're (surprise!) in the business of developing software ourselves and never cease to wonder, what errors we detect in our software that have been in the product since the earliest releases in 1992 but never emerged to the surface until a certain specific (not necessarily exotic) combination of factors occured. And in this case I more or less expect the error to have occured in the VSS Import process - a piece of code certainly less heavily used and tested than Vault. I suspect that the import tool has somehow clobbered up some data. We do have another topic in the VSS Import forum that might be related. And we also suspect (but have not yet been able to reproduce) that certain PDF-files get garbled by VSS Import as well.

When you diffed, what file was used for the comparison? Another idea would be to do a get on a totally different machine, and diff the two files.
You could try to delete the file out of the working folder, and re-try the get operation.

We really tried out everything meticulously with two pairs of eyes on every step, since we really tried to understand what was going on here.

Also, if there is a small amount of history of the file, I wonder what would happen if you re-add the file w/ all the changes?

That would be quite tedious, since this file has 142 revisions. Perhaps you can give me some SQL with which I could export the relevant data from the SQL database so that you might be able to look into the decoded binary data of the deltas?

[jclausius]

We use the CRC-option since we had one other case of renegade though identical since apparently the rounding of the filetime in vss and vault was in this case slightly different.

What is the target file system of the GET operation? Vault is designed to handle a wide scale of time rounding issues. We've tested against FAT-32, NTFS, and ext3.

One thing we could try is to see what is happening on the file system and your working folder itself. We can use the Vault 3.5 Diagnostic tool to compare time stamps on the system.

And in this case I more or less expect the error to have occured in the VSS Import process - a piece of code certainly less heavily used and tested than Vault. I suspect that the import tool has somehow clobbered up some data.

So is it one of these .ctx or .frx files that is causing the problem?

One thing you could do is export the folder containing the file using the Vault Export, and upload it here. We could then check the database for that file.

And we also suspect (but have not yet been able to reproduce) that certain PDF-files get garbled by VSS Import as well.

Are the .pdf file properties set to Binary within VSS? There might be problems in retrieving them or the files stored within VSS itself. There have been a lot of new issues about placing .pdf files in VSS 2005. See http://groups.google.com/groups?lnk=hps ... df+corrupt for more information.

Perhaps you can give me some SQL with which I could export the relevant data from the SQL database so that you might be able to look into the decoded binary data of the deltas?

The Vault Folder export tool I mentioned above would be the best bet.

[TeamWiSE]

In the meantime, we're talking about three issues here:

• 1. a file being shown as renegade although identical using CRC compare
  2. a file being shown as renegade although identical using timestamp compare
  3. pdf-files that are corrupt after vss import (a topic we did not want to raise until further investigation)

Issue 1

So is it one of these .ctx or .frx files that is causing the problem?

Nope. As I mentioned before, it's a plain old-fashioned C-include file.

One thing you could do is export the folder containing the file using the Vault Export, and upload it here. We could then check the database for that file.

We were going to send it through private e-Mail, but then again, the export was some 52MB in size (the folder is not so large, but it does contain some 12 years of history). So we tried to reduce the export by sharing the include file in question in a new project. As we tried getting the revision that showed the symptoms described in order to offer you a proper scenario in reproducing the problem, the file/revision did not show up as renegade in the new folder... We studied the various test-runs again and the one suggestion that we did not try out was to do a get on another machine. Furthermore, the folder in question had been used in the early phases of evaluating vault, where minor subsets of the VSS-stored data had been imported (and deleted) in Vault. Apparently the problem is caused by the caching mechanism that vault stores in the application data of the logged-on user. So clearing your local project folder does not set you up completely afresh from scratch. What more does it take to make sure that a client is properly in sync with the vault database data? Is there such a command as "clear your memory" or the like? Note that in the evaluation process a vault database with the same name had been created and deleted over and over again, each time with new vss-imported data, but with the same databasename. We've dropped this topic from our checklist, but a new client management/setup topic has been entered...

Issue 2

What is the target file system of the GET operation?

The target fs is NTFS.

One thing we could try is to see what is happening on the file system and your working folder itself. We can use the Vault 3.5 Diagnostic tool to compare time stamps on the system.

We'll try to reproduce that problem - this will take some time. Ofcourse we'll also have a closer look to check if this problem might be related to vaults caching mechanism. The file in question here had only been involved in the final evaluation step (total import of all vss-data) and as such could not have left any caching traces in earlier steps. We'll study this again to see if the CRC-compare is really necessary or the timestamp compare is reliable enough.

Issue 3

Are the .pdf file properties set to Binary within VSS? There might be problems in retrieving them or the files stored within VSS itself.

I suspect and assume that the cause of the PDF-Problem lies here. As I mentioned, we weren't able to reproduce that problem yet and the explanations in the various articles fit exactly to the symptoms we noticed. So we'll consider this issue closed.

I really appreciate the time you take to support us on this. That's already a huge plus in the evaluation of Vault.

[jclausius]

Nope. As I mentioned before, it's a plain old-fashioned C-include file.

Sorry, I was mixing up the messages. It was the header file you were talking about.

We were going to send it through private e-Mail, but then again, the export was some 52MB in size (the folder is not so large, but it does contain some 12 years of history).

Not a problem. For future reference, you could also use anonymous ftp site.

So clearing your local project folder does not set you up completely afresh from scratch. What more does it take to make sure that a client is properly in sync with the vault database data? Is there such a command as "clear your memory" or the like?

Unfortunately not at this time, but this is an enhancement we will be adding to a future version of Vault. One work-around is to store the working folder state files within the working folders themselves. This does require special consideration, as some people have accidentally over-write / change the read-only state files.

We'll try to reproduce that problem - this will take some time. Ofcourse we'll also have a closer look to check if this problem might be related to vaults caching mechanism. The file in question here had only been involved in the final evaluation step (total import of all vss-data) and as such could not have left any caching traces in earlier steps. We'll study this again to see if the CRC-compare is really necessary or the timestamp compare is reliable enough.

I'm hopeful that is the case. Note, you can use the Client Diagnostic tool to open up the directory where the client side state files exist. You can then delete them from there. Again, this is a work-around until this functionality is built into the Vault client itself.

I really appreciate the time you take to support us on this. That's already a huge plus in the evaluation of Vault.

We're here to help. Anytime you have a question, please feel free to post.