Shadow folder won't populate?

[Figbashing]

I am implementing Vault on an existing website of about 13,000 files. I want to use a shadow folder to push to my web root, but I can't get my shadow folder to populate. I started off setting the shadow folder to the existing webroot that still had all the files in it that I'd used to populate the repository to begin with after creating a copy of that structure to be my client working folder, plus setting it for a complete synch, not speed-optimized. Then after poking around the forum some I thought those existing files were maybe confusing Vault so I also tried deleting that Shadow Folder setting in the admin and starting over again with either an empty webroot, or a deleted webroot directory to see if Vault would want to create it itself. None of these options worked and I am stumped. Please advise.

(Related: I am dearly hoping that shadow folders will also mirror my repository's subfolders—and subsubsubsubfolders—without each having to be set up individually. Can you confirm?)

[lbauer]

What version of Vault are you using?

Is the shadow folder on the same machine as Vault Server, or a different machine on the same LAN?

We'd like to see a copy of the Vault Server log and the shadow folder log. The Vault log is called sgvault.log and is in %windir%\temp\sgvault on the server machine. The shadow folder log is also in Windows\Temp, in SgShadowfolder.

Send the logs zipped up to support at sourcegear.com, Attn: Linda. Please include a link to this forum post.

[Figbashing]

Version 5.1.2.19281.

Same machine.

Will send logs.

[Figbashing]

Linda found my problem and offered a good fix, via email, quoted below, and then I have another question on Shadow Folders...

6/12/2012 12:41:44 AM <generic>: [<No Name>:5572] Vault's Shadow Folders encountered an exception attempting to get the files for [repository name]. The Shadow Folders for the repository may not be synchronized with the Vault Server.Access to the path 'C:\inetpub\[...]png' is denied. :::: Time elapsed from Get() start to exception was 0.0473200833333333

The account running the Shadow Folder service may not have write access to the 'C:\inetpub\[...] and subfolders. Based on your Vault Server log, the Vault server and Shadow Folder service is being run under the [...] account.

Try giving that account write access or maybe Full Control of C:\inetpub\[...] and subfolders. You could also try sharing that folder.

I did, and it worked, but now I'm having a hard time really grasping the docs' description of Speed Optimized=True. (For example, what constitutes a "monitored transaction?") So let me describe my use case and see what seems to fit...

The site I'm implementing Vault on has an existing custom-built CMS, plus a WordPress installation. So there are several ways that content authors are able to arbitrarily create web pages or upload certain types of binaries like images and PDFs via these admin tools.

Since, again, our Shadow Folder is our webroot, then under Vault, what will happen to files that appear in the Shadow Folder due to being uploaded via any of the above-described means? The docs make it sound like Speed Optimized=False would maybe delete them? But Speed Optimized = True sounds like it would allow all manner of changes directly to the Shadow Folder outside of the context of version control, and I may simply not be understanding it, but I can't even envision the use case for which I'd want to have that turned on. I am, after all, installing Vault so that I never have anyone making unauthorized or unnoticed changes, and mostly I'm thinking they'd be done via filesystem methods like FTP or even direct editing via remote desktop login. I'd be grateful for a better explanation of this feature.

[lbauer]

Here are the two options:

Synchronized with Repository - The Shadow Folder service maintains an exact copy of the repository folder. If changes are made to any of the files within the file system of the Shadow Folder's target, these changes are overwritten on the next Shadow Folder update.

Optimized for Speed - After the Shadow Folder target has been retrieved to the file system, only changes within monitored transactions are reflected on disk. The Shadow Folder target will not be fully synchronized with the Shadow Folder's repository folder if changes are made to items on the file system. Only the change set items committed to the repository will be synchronized in the Shadow Folder target. This is the default setting for new shadow folders.

Vault controls what goes into the source control database through user login, repository access rights and folder security. However, Windows operations still have an impact on what's outside of Vault, on the file system.

[Figbashing]

OK, thanks, I appreciate this, but I'm not sure it brought much more clarity.

I still couldn't tell if a file created in the Shadow Folder via filesystem (with Synchronized with Repository setting) would be automatically deleted by a process that recognizes it as not having gotten there via repository shadowing. So I tested. It doesn't appear any deletion occurs. So that's not truly an exact copy of the repository folder. Is this correct/working as designed?

It also seems that (with Synchronized with Repository setting) direct filesystem edits to files in shadow folders which did get there via correct shadowing from repository are only detected and automatically reverted when some action (maybe check-ins only?) is taken on the repository. So there is no process that runs periodically to do checks for just this scenario, and reversions depend then on a certain frequency of user interaction with the repository. Is this correct/working as designed?

Lastly, I remain unclear on what constitutes a "monitored transaction," but if the "Optimized for Speed" setting to which it applies is even less strict about directory shadowing than the "Synchronized with Repository" scenarios described above, I don't guess it'll fit my needs.

My intention is not to nitpick, but to very thoroughly understand, as my client really wants to minimize or even eliminate the ability for anyone to make website changes other than through a logged and traceable mechanism, and I need to know what the risks are with Vault. It sounds like anyone who can log into my server and gain filesystem access is free to bypass Vault and create new files in the shadow directory, even with "Synchronized with Repository." If this is incorrect, due perhaps to misconfiguration on my part or any possible bug on the part of my Vault install, I hope you'll let me know so I have the opportunity to tighten things up.

Either way, thanks for your help.

[lbauer]

Optimize allows the contents of the shadow folder to be modified. Files added manually are not deleted. A monitored transaction is one where the file in the shadow folder is in the repository and is being retrieved and updated in the ShadowFolder,

Synchronize is supposed to keep the shadow folder in sync with the repository folder. That means any items added should be deleted when the shadow folder is updated.

Unfortunately, we have discovered a bug in the Synchronize function in Vault 5.1.2, and Synchronize behaves in the same way as Optimize. We have bug 15376 logged to fix this, and hope to fix it in the next release of Vault.

NOTE: See Clarification on Synchronize, below.
http://support.sourcegear.com/viewtopic ... 036#p69252

[Figbashing]

Thanks so much for clarifying this. Very, very helpful. I must push my luck though and ask another question (sorry for the long delay; I was traveling).

After the synchronize bug is fixed and that feature is working as designed, is there a way to designate certain directories within the shadow folder as being exempt from the synchronize rule? Or do you have any other suggestions for making it work so that (in a nutshell) my whole wwwroot is synchronize EXCEPT wwwroot\photouploads?

[lbauer]

The Shadow Folder retrieval is recursive, so that if you are shadowing $/Mywebsite, then every subfolder will also be shadowed and subject to the same optimization setting.

A workaround might be to move the photouploads folder in the repository so that it is not under the root of the website. Then you can create a separate shadow folder for it, within the same shadow folder directory

I tried something like this and it worked, though your mileage may vary.

Repository tree:

$/mywebsite shadowed to C:\WebsiteShadow, synchronized

$/mywebsitephotos also shadowed to C:\Website Shadow, optimized

The shadow directory would look like this:

C:\mywebsite
-- \subfolders of mywebsite
-- \more subfolders of mywebsite
-- \mywebsitephotos

[lbauer]

A clarification on the Synchronize option for Shadow Folders:

With the Synchronize option, the Shadow Folder contents should match the repository contents. Any files modified directly on disc will be re-sychronized so the contents match the version in the repository.

Any files added to the repository or deleted from the repository will be added to or deleted from the Shadow folder directory. However the Shadow Folder service will not delete files in the Shadow Folder directory that were added directly, and not by the Shadow Folder service.