Version control theory and best practicces?

[gmagana]

Hello all,

I am new to source control and to Vault (in fact I've asked a couple questions about Vault already), and am relatively lost when it comes to the whole version control area.

In the past my source control was limited to "ok, it's a release so let's ZIP the source code at this point." Now I am trying to manage many projects via Vault and some stuff just does not translate, such as this:

Before I had a multi-platform application in C++ that I develop and debug in Windows. The true release is done to DOS in an embedded application. Before, I had a single copy of the source code, both the Windows version and the DOS version would compile off of it, any platform-specific code is included or excluded via preprocessor directives.

The file were laid out like this:

Main Directory
|-- Source Code
|-- DOS build and link scripts
|-- Windows project, temp, and output directories

Now with Vault, I cannot arrange it like this because Vault does not like the source code to be in a directory different than the solution/project files. So I grudgingly resorted to shared files within Vault, but I don;t like it because normally I am compiling constantly with one compiler and then making sure the other compiler (which runs on another machine) likes the code. Under this scenario, I have to check out, make a change, check in, then do a GET with the other machine, check out, make changes, check in, then repeat on the other machine.

I don't think this is the best way to do it, there are too many checking in and out... Maybe if I could check out to a different source code directory on a network drive and then use the source from both compilers until a check in made sense, then it would be better. I cannot do this because the source I change from Windows is now in a separate directory than the DOS source code.

So I am asking two questions: 1) What is a better way to do this? and 2) Where could I get more background on version control best practices? It seems to me like I don;t have a good picture of how all this is supposed to work...

Thanks.

[ericsink]

First of all, why do you say the following?

Vault does not like the source code to be in a directory different than the solution/project files

Vault doesn't care where you place things. Visual Studio gets fussy sometimes, but you can route around that particular outage by fiddling with Visual Studio. Anyway, your tree layout is very reasonable. I would probably do it the same way, and there is no reason Vault will object.

We get a lot of requests for "best practices" or "common practices" or "background in version control". There frankly isn't much material out there. Most people learn version control by doing it badly and learning from their mistakes.

Part of the reason there is no general material on version control is that there is no standard terminology across the various tools. We see words like like branch, merge, resolve, tag, label, and checkout, but these words mean different things in different systems.

Stephen Berczuk and Brad Appleton have done a reasonably nice job trying to catalog the various patterns in their book "Software Configuration Management Patterns". I often recommend this book for people looking for version control background information.

[gmagana]

First of all, why do you say the following?

Vault does not like the source code to be in a directory different than the solution/project files

I'm sure I screwed up on where I got the message from, it must have been VS.NET telling me this. I'll try to recreate the project, maybe I can hand-edit the project file to reference source code in a separate directory.

Sorry about getting mixed up.

We get a lot of requests for "best practices" or "common practices" or "background in version control". There frankly isn't much material out there. Most people learn version control by doing it badly and learning from their mistakes.

Well, I just don't want to lose source code, but I would like to keep my life easy as much as I can.

I am unsure about some fundamentals, for example, when is it time to do a check-in? When a feature is finished, or at the end of the day when your source compiles but does not yet work right? Or am I totally off-base and I should check in every half hour? In a couple weeks we'll introduce more programmers to Vault and I would like to get an idea of how we should be working together...

Part of the reason there is no general material on version control is that there is no standard terminology across the various tools. We see words like like branch, merge, resolve, tag, label, and checkout, but these words mean different things in different systems.

That's also part of my problem: When should I "label" and when should I "branch"? Do I wait for the first bug reported in a prior version before I branch a labeled version? As you mention I have also found that the terms and exact meaning and implications vary between version control products.

Stephen Berczuk and Brad Appleton have done a reasonably nice job trying to catalog the various patterns in their book "Software Configuration Management Patterns". I often recommend this book for people looking for version control background information.

Ahh, I just found it on sale. I will read it up. I really feel lost and need to educate myself as much as possible! Thanks for the reference!

[ericsink]

I am unsure about some fundamentals, for example, when is it time to do a check-in? When a feature is finished, or at the end of the day when your source compiles but does not yet work right? Or am I totally off-base and I should check in every half hour? In a couple weeks we'll introduce more programmers to Vault and I would like to get an idea of how we should be working together...

That's also part of my problem: When should I "label" and when should I "branch"? Do I wait for the first bug reported in a prior version before I branch a labeled version? As you mention I have also found that the terms and exact meaning and implications vary between version control products.

Ah, yes.

There are a lot of different ways to do this stuff. I didn't realize just how many ways until we got into the version control market and started hearing from customers.

Personally, I am a big believer in holding checkins until they work properly. The tree is sacred. Don't check in anything which breaks the tree, because that prevents other people on the team from working. For years, I have operated with a rule that anybody who breaks the tree has to buy donuts for the team. Unfortunately, it turns out that everybody's eating habits when we have donuts three times a week, but that's another problem.

Sometimes this means you will have stuff sitting on your private machine for several days before it is checked in. This is normal.

However, if you find yourself in a situation where you have to wait 1, 2 or 3 weeks or more before doing a checkin, that's a signal that you should probably be doing a private branch.

A branch is like a private area in the source control system. It allows us to keep doing checkins without breaking the tree. It's our branch, and we can break it if we want to, with no donut penalties.

The downside of a branch is that it is divergent. It is a separate, parallel line of development from the main trunk. Eventually, we want these lines to merge again. Vault offers a feature called Merge Branches which assists with the process of migrating changes from the branch back to its trunk, or in the other direction.

A label is a simpler beast. Usually a label is used to simply mark or tag a specific moment in the history of your tree. Branches are fairly lightweight, but labels are even lighter. We apply a label every time we do a build, which is basically every day. A label allows us to grab the tree as it existed at the moment the label was applied.

You also asked:

Do I wait for the first bug reported in a prior version before I branch a labeled version?

Yes, IMHO. I never branch until I need to.

[Guest]

Here are some things we do

- Label after every build. This lets developers get a stable build on their local. Since there is no gurantee that any commits after a build will actually compile.

- Build often. We build 3 times a day and if the build was faster we would do it more often.

- Branch releases. When you need to release a version say 1.0 create a branch for it. That way if you find you need to fix a bug / issue a patch you will have all the files in their original state.

- Make sure you are carefull when issuing patches. Most often you will need to merge in the fix / patch into the main tree so that the bug is fixed in both spots. Prepare your merge before checking in the fix.

- Check in files that only compile. This should be a rule that is never broken; use shame to keep developers from taking short cuts.

- Keep your source tree "organized". Don't let developers create new sub directories willy-nilly, make sure they follow a pattern that is well estiablished supported.

- Don't check in binaries. Anything that can be created from a build shouldn't go into source control.

cheers/

[gmagana]

Thanks for the great suggestions. I'll see which options fit us better.

As for the labeling, checking in/out, etc... What happens with Vault a year or two from now when we have a bunch of labels/branches/log entries, etc...? Is there a way to "trim" smoe of the activity? For example, Is there a way to just keep the branches and labels intact, so that between label "ver1.0" and label "ver1.01" there might be 10 different version of a file (ie, 10 check-ins, from #203 to #212), but I want to get rid of the middle check ins, and merge all of those check-ins into #212 (so that the database is slimmed down a bit as far as log entries)?

I don't know if this makes sense, but I am thinking of what happens when the database gets to a "huge" size. How do we deal with that situation in Rainbow?

[ericsink]

Thanks for the great suggestions. I'll see which options fit us better.

As for the labeling, checking in/out, etc... What happens with Vault a year or two from now when we have a bunch of labels/branches/log entries, etc...? Is there a way to "trim" smoe of the activity? For example, Is there a way to just keep the branches and labels intact, so that between label "ver1.0" and label "ver1.01" there might be 10 different version of a file (ie, 10 check-ins, from #203 to #212), but I want to get rid of the middle check ins, and merge all of those check-ins into #212 (so that the database is slimmed down a bit as far as log entries)?

I don't know if this makes sense, but I am thinking of what happens when the database gets to a "huge" size. How do we deal with that situation in Rainbow?

A version control purist (like me) would say that nothing should ever be trimmed or pruned from the history of a repository.

That's one of the beautiful things about version control: The Delete command doesn't really do anything. Nothing is ever really gone. The history of the repository is complete. Every state which has ever existed is still there to be found.

When we designed Vault, we did make the compromising decision to include an Obliterate command. This feature allows the user to really delete something, permanently. In a utopian world, this feature wouldn't exist. However, SourceGear isn't the first company to learn that there is money to be made by selling people things they can use to do themselves harm.

[gmagana]

I see where you're coming from. I like building databases which store a history that's a long as possible. This is why we're an Oracle shop, and the only exception to this rule is Vault (by the way, when is an Oracle version coming? ). By the way, you can see just how much we like Vault, we broke our sacred "only Oracle, no exceptions" rule so we could use Vault! But we definitely would switch to an Oracle version as soon as it came out.

Anyway, I am wondering about the time when the Vault database is just too large? Just by adding my projects to Vault (about 8 years' worth of frantic programming) the SQL Server backups are already up to 50 MB zipped. Now I figure that as we add the other programmers' projects to Vault, it should be about 50 additional MBs, and the it will keep growing, so eventually we will start to see that the Vault database is getting too big. What will our options be then?

If it happened right now, what I would do is GET the latest versions of everything and start a new database on another server and "freeze" the current database.

Is there a standard thing to do in Vault in this case?

[ericsink]

I see where you're coming from. I like building databases which store a history that's a long as possible. This is why we're an Oracle shop, and the only exception to this rule is Vault (by the way, when is an Oracle version coming? ). By the way, you can see just how much we like Vault, we broke our sacred "only Oracle, no exceptions" rule so we could use Vault! But we definitely would switch to an Oracle version as soon as it came out.

Anyway, I am wondering about the time when the Vault database is just too large? Just by adding my projects to Vault (about 8 years' worth of frantic programming) the SQL Server backups are already up to 50 MB zipped. Now I figure that as we add the other programmers' projects to Vault, it should be about 50 additional MBs, and the it will keep growing, so eventually we will start to see that the Vault database is getting too big. What will our options be then?

If it happened right now, what I would do is GET the latest versions of everything and start a new database on another server and "freeze" the current database.

Is there a standard thing to do in Vault in this case?

There is not "a standard thing to do in Vault in this case".

Besides, you shouldn't have to worry about this for a very, very long time. I'm not sure what you mean by "too big", but 50 MB doesn't seem even close. I'd be inclined to start wondering at around 50 GB.