A version control system (or revision control system) is a combination of technologies and practices for tracking and controlling changes to a project's files, in particular to source code, documentation, and web pages. If you have never used version control before, the first thing you should do is go find someone who has, and get them to join your project. These days, everyone will expect at least your project's source code to be under version control, and probably will not take the project seriously if it doesn't use version control with at least minimal competence.
The reason version control is so universal is that it helps with virtually every aspect of running a project: interdeveloper communications, release management, bug management, code stability and experimental development efforts, and attribution and authorization of changes by particular developers. The version control system provides a central coordinating force among all of these areas. The core of version control is change management: identifying each discrete change made to the project's files, annotating each change with metadata like the change's date and author, and then replaying these facts to whoever asks, in whatever way they ask. It is a communications mechanism where a change is the basic unit of information.
This section does not discuss all aspects of using a version control system. It's so all-encompassing that it must be addressed topically throughout the book. Here, we will concentrate on choosing and setting up a version control system in a way that will foster cooperative development down the road.
This book cannot teach you how to use version control if you've never used it before, but it would be impossible to discuss the subject without a few key terms. These terms are useful independently of any particular version control system: they are the basic nouns and verbs of networked collaboration, and will be used generically throughout the rest of this book. Even if there were no version control systems in the world, the problem of change management would remain, and these words give us a language for talking about that problem concisely.
To make a change to the project; more formally, to store a change in the version control database in such a way that it can be incorporated into future releases of the project. "Commit" can be used as a verb or a noun. As a noun, it is essentially synonymous with "change." For example: "I just committed a fix for the server crash bug people have been reporting on Mac OS X. Jay, could you please review the commit and check that I'm not misusing the allocator there?"
A bit of commentary attached to each commit, describing the nature and purpose of the commit. Log messages are among the most important documents in any project: they are the bridge between the highly technical language of individual code changes and the more user-oriented language of features, bug fixes, and project progress. Later in this section, we'll look at ways to distribute log messages to the appropriate audiences; also, Section 6.4.2 in Chapter 6 discusses ways to encourage contributors to write concise and useful log messages.
To ask that others' changes (commits) be incorporated into your local copy of the project; that is, to bring your copy "up-to-date." This is a very common operation; most developers update their code several times a day, so that they know they're running roughly the same thing the other developers are running, and so that if they see a bug, they can be pretty sure it hasn't been fixed already. For example: "Hey, I noticed the indexing code is always dropping the last byte. Is this a new bug?" "Yes, but it was fixed last week—try updating, it should go away."
A database in which changes are stored. Some version control systems are centralized: there is a single, master repository, which stores all changes to the project. Others are decentralized: each developer has her own repository, and changes can be swapped back and forth between repositories arbitrarily. The version control system keeps track of dependencies between changes, and when it's time to make a release, a particular set of changes is approved for that release. The question of whether centralized or decentralized is better is one of the enduring holy wars of software development; try not to fall into the trap of arguing about it on your project lists.
The process of obtaining a copy of the project from a repository. A checkout usually produces a directory tree called a "working copy" (see the next entry), from which changes may be committed back to the original repository. In some decentralized version control systems, each working copy is itself a repository, and changes can be pushed out to (or pulled into) any repository that's willing to accept them.
A developer's private directory tree containing the project's source code files, and possibly its web pages or other documents. A working copy also contains a little bit of metadata managed by the version control system, telling the working copy what repository it comes from, what "revisions" (see the next entry) of the files are present, etc. Generally, each developer has his own working copy, in which he makes and tests changes, and from which he commits.
A revision is usually one specific incarnation of a particular file or directory. For example, if the project starts out with revision 6 of file F, and then someone commits a change to F, this produces revision 7 of F. Some systems also use the terms "revision," "change," or "changeset" to refer to a set of changes committed together as one conceptual unit.
These terms occasionally have distinct technical meanings in different version control systems, but the general idea is always the same: they give a way to speak precisely about exact points in time in the history of a file or a set of files (say, immediately before and after a bug is fixed). For example: "Oh yes, she fixed that in revision 10" or "She fixed that in revision 10 of foo.c."
When one talks about a file or collection of files without specifying a particular revision, it is generally assumed that one means the most recent revision(s) available.
A textual representation of a change. A diff shows which lines were changed and how, plus a few lines of surrounding context on either side. A developer who is already familiar with some code can usually read a diff against that code and understand what the change did, and even spot bugs.
A label for a particular collection of files at
specified revisions. Tags are usually used to preserve
interesting snapshots of the project. For example, a tag is
usually made for each public release, so that one can obtain,
directly from the version control system, the exact set of
files/revisions comprising that release. Common tag names are
things like Release_1_0,
Delivery_00456, etc.
A copy of the project kept under version control but isolated, so that changes made to the branch don't affect the rest of the project and vice versa, except when changes are deliberately merged from one side to the other (see the next entry). Branches are also known as "lines of development." Even when a project has no explicit branches, development is still considered to be happening on the "main branch" also known as the "main line" or "trunk."
Branches offer a way to isolate different lines of development from each other. For example, a branch can be used for experimental development that would be too destabilizing for the main trunk. Or conversely, a branch can be used as a place to stabilize a new release. During the release process, regular development would continue uninterrupted in the main branch of the repository; meanwhile, on the release branch, no changes are allowed except those approved by the release managers. This way, making a release needn't interfere with ongoing development work. See Section 3.3.3.4 later in this chapter for a more detailed discussion of branching.
To move a change from one branch to another. This includes merging from the main trunk to some other branch, or vice versa. In fact, those are the most common kinds of merges; it is rare to port a change between two non-main branches. See Section 3.3.3.5 for more about this kind of merging.
Merge has a second, related meaning: it is what the version control system does when it sees that two people have changed the same file but in non-overlapping ways. Since the two changes don't interfere with each other, when one of the people updates his copy of the file (already containing his own changes), the other person's changes will be automatically merged in. This is very common, especially on projects where multiple people are hacking on the same code. When two different changes do overlap, the result is a "conflict"; see the next entry.
What happens when two people try to make different changes to the same place in the code. All version control systems automatically detect conflicts, and notify at least one of the humans involved that their changes conflict with someone else's. It is then up to that human to resolve the conflict, and to communicate that resolution to the version control system.
A way to declare an exclusive intent to change a particular file or directory. For example, "I can't commit any changes to the web pages right now. It seems Alfred has them all locked while he fixes their background images." Not all version control systems even offer the ability to lock, and of those that do, not all require the locking feature to be used. This is because parallel, simultaneous development is the norm, and locking people out of files is (usually) contrary to this ideal.
Version control systems that require locking to make commits are said to use the lock-modify-unlock model. Those that do not are said to use the copy-modify-merge model. An excellent in-depth explanation and comparison of the two models may be found at http://svnbook.red-bean.com/svnbook-1.0/ch02s02.html. In general, the copy-modify-merge model is better for open source development, and all the version control systems discussed in this book support that model.
As of this writing, the version control system of choice in the free software world is the Concurrent Versions System, or CVS (http://www.nongnu.org/cvs). CVS has been around for a long time. Most experienced developers are already familiar with it, it does more or less what you need, and since it's the default, you won't end up in any long debates about whether or not it was the right choice. CVS has some disadvantages, however. It doesn't provide an easy way to refer to multi-file changes; it doesn't allow you to rename or copy files under version control (so if you need to reorganize your code tree after starting the project, it can be a real pain); it has poor merging support; it doesn't handle large files or binary files very well; and some operations are slow when large numbers of files are involved.
None of CVS's flaws is fatal, and it is still quite popular. However, in the last few years a number of new version control systems have appeared, and free software projects are beginning to try them out. Appendix A lists all the ones I know of. As that list makes clear, deciding on a version control system could easily become a lifelong research project. Possibly you will be spared the decision because it will be made for you by your hosting site. But if you must choose, consult with your other developers, ask around to see what people have experience with, then pick one and run with it. Any stable, production-ready version control system will do; you don't have to worry too much about making a drastically wrong decision. If you simply can't make up your mind, then go with CVS. It's still the standard, and will probably continue to be so for a few years. Also, many of the other systems support one-way conversion from CVS, so you can change your mind later anyway.
The recommendations in this section are not targeted toward a particular version control system, and should be simple to implement in any of them. Consult your specific system's documentation for details.
Keep not only your project's source code under version control, but also its web pages, documentation, FAQ, design notes, and anything else that people might want to edit. Keep them right next to the source code, in the same repository tree. Any piece of information worth writing down is worth versioning—that is, any piece of information that could change. Things that don't change should be archived, not versioned. For example, an email, once posted, does not change; therefore, versioning it wouldn't make sense (unless it becomes part of some larger, evolving document).
The reason versioning everything together in one place is important is so people have to learn only one mechanism for submitting changes. Often a contributor will start out making edits to the web pages or documentation, and move to small code contributions later, for example. When the project uses the same system for all kinds of submissions, people have to learn the ropes only once. Versioning everything together also means that new features can be committed together with their documentation updates, that branching the code will branch the documentation too, etc.
Don't keep generated files under version control. They are not truly editable data, since they are produced programmatically from other files. For example, some build systems create configure based on the template configure.in. To make a change to the configure, one would edit configure.in and then regenerate; thus, only the template configure.in is an "editable file." Always version only the templates. If you version the result files as well, people will inevitably forget to regenerate when they commit a change to a template, and the resulting inconsistencies will cause no end of confusion.
The rule that all editable data should be kept under version control has one unfortunate exception: the bug tracker. Bug databases hold plenty of editable data, but for technical reasons generally cannot store that data in the main version control system. (Some trackers have primitive versioning features of their own, however, independent of the project's main repository.)
The project's repository should be browseable on the Web. This means not only the ability to see the latest revisions of the project's files, but to go back in time and look at earlier revisions, view the differences between revisions, read log messages for selected changes, etc.
Browseability is important because it is a lightweight portal to project data. If the repository cannot be viewed through a web browser, then someone wanting to inspect a particular file (say, to see if a certain bug fix had made it into the code) would first have to install version control client software locally, which could turn her simple query from a two-minute task into a half-hour or longer task.
Browseability also implies canonical URLs for viewing specific revisions of files, and for viewing the latest revision at any given time. This can be very useful in technical discussions or when pointing people to documentation. For example, instead of saying "For tips on debugging the server, see the HACKING file in the top of your working copy," one can say "For tips on debugging the server, see http://svn.collab.net/repos/svn/trunk/HACKING," giving a URL that always points to the latest revision of the HACKING file. The URL is better because it is completely unambiguous, and avoids the question of whether the addressee has an up-to-date working copy.
Some version control systems come with built-in repository-browsing mechanisms, while others rely on third-party tools to do it. Three such tools are ViewCVS (http://viewcvs.sourceforge.net/), CVSWeb (http://www.freebsd.org/projects/cvsweb.html), and WebSVN (http://websvn.tigris.org/). The first works with both CVS and Subversion, the second with CVS only, and the third with Subversion only.
Every commit to the repository should generate an email showing who made the change, when they made it, what files and directories changed, and how they changed. The email should go to a special mailing list devoted to commit emails, separate from the mailing lists to which humans post. Developers and other interested parties should be encouraged to subscribe to the commits list, as it is the most effective way to keep up with what's happening in the project at the code level. Aside from the obvious technical benefits of peer review (see Section 2.4.3 in Chapter 2), commit emails help create a sense of community, because they establish a shared environment in which people can react to events (commits) that they know are visible to others as well.
The specifics of setting up commit emails will vary depending on your version control system, but usually there's a script or other packaged facility for doing it. If you're having trouble finding it, try looking for documentation on hooks, specifically a post-commit hook, also called the loginfo hook in CVS. Post-commit hooks are a general means of launching automated tasks in response to commits. The hook is triggered by an individual commit, is fed all the information about that commit, and is then free to use that information to do anything—for example, to send out an email.
With prepackaged commit email systems, you may want to modify some of the default behaviors:
Some commit mailers don't include the actual diffs in the email, but instead provide a URL to view the change on the web using the repository browsing system. While it's good to provide the URL, so the change can be referred to later, it is also very important that the commit email include the diffs themselves. Reading email is already part of people's routine, so if the content of the change is visible right there in the commit email, developers will review the commit on the spot, without leaving their mail reader. If they have to click on a URL to review the change, most won't do it, because that requires a new action instead of a continuation of what they were already doing. Furthermore, if the reviewer wants to ask something about the change, it's vastly easier to hit reply-with-text and simply annotate the quoted diff than it is to visit a web page and laboriously cut-and-paste parts of the diff from web browser to email client.
(Of course, if the diff is huge, such as when a large body of new code has been added to the repository, then it makes sense to omit the diff and offer only the URL. Most commit mailers can do this kind of limiting automatically. If yours can't, then it's still better to include diffs, and live with the occasional huge email, than to leave the diffs off entirely. Convenient reviewing and commenting is a cornerstone of cooperative development, much too important to do without.)
The commit emails should set their Reply-to header to the regular development list, not the commit email list. That is, when someone reviews a commit and writes a response, their response should be automatically directed toward the human development list, where technical issues are normally discussed. There are a few reasons for this. First, you want to keep all technical discussion on one list, because that's where people expect it to happen, and because that way there's only one archive to search. Second, there might be interested parties not subscribed to the commit email list. Third, the commit email list advertises itself as a service for watching commits, not for watching commits and occasional technical discussions. Those who subscribed to the commit email list did not sign up for anything but commit emails; sending them other material via that list would violate an implicit contract. Fourth, people often write programs that read the commit email list and process the results (for display on a web page, for example). Those programs are prepared to handle consistently-formatted commit emails, but not inconsistent human-written mails.
Note that this advice to set Reply-to does not contradict the recommendations in Section 3.2.3 earlier in this chapter. It's always okay for the sender of a message to set Reply-to. In this case, the sender is the version control system itself, and it sets Reply-to in order to indicate that the appropriate place for replies is the development mailing list, not the commit list.
Non-expert version control users are sometimes a bit afraid of branching and merging. This is probably a side effect of CVS's popularity: Its interface for branching and merging is somewhat counterintuitive, so many people have learned to avoid those operations entirely.
If you are among those people, resolve right now to conquer any fears you may have and take the time to learn how to do branching and merging. They are not difficult operations, once you get used to them, and they become increasingly important as a project acquires more developers.
Branches are valuable because they turn a scarce resource—working room in the project's code—into an abundant one. Normally, all developers work together in the same sandbox, constructing the same castle. When someone wants to add a new drawbridge, but can't convince everyone else that it would be an improvement, branching makes it possible for her to go to an isolated corner and try it out. If the effort succeeds, she can invite the other developers to examine the result. If everyone agrees that the result is good, they can tell the version control system to move ("merge") the drawbridge from the branch castle over to the main castle.
It's easy to see how this ability helps collaborative development. People need the freedom to try new things without feeling like they're interfering with others' work. Equally important, there are times when code needs to be isolated from the usual development churn, in order to get a bug fixed or a release stabilized (see Section 7.3 and Section 7.6 in Chapter 7) without worrying about tracking a moving target.
Use branches liberally, and encourage others to use them. But also make sure that a given branch is only active for exactly as long as needed. Every active branch is a slight drain on the community's attention. Even those who are not working in a branch still maintain a peripheral awareness of what's going on in it. Such awareness is desirable, of course, and commit emails should be sent out for branch commits just as for any other commit. But branches should not become a mechanism for dividing the development community. With rare exceptions, the eventual goal of most branches should be to merge their changes back into the main line and disappear.
Merging has an important corollary: never commit the same change twice. That is, a given change should enter the version control system exactly once. The revision (or set of revisions) in which the change entered is its unique identifier from then on. If it needs to be applied to branches other than the one on which it entered, then it should be merged from its original entry point to those other destinations—as opposed to committing a textually identical change, which would have the same effect in the code, but would make accurate bookkeeping and release management impossible.
The practical effects of this advice differ from one version control system to another. In some systems, merges are special events, fundamentally distinct from commits, and carry their own metadata with them. In others, the results of merges are committed the same way other changes are committed, so the primary means of distinguishing a "merge commit" from a "new change commit" is in the log message. In a merge's log message, don't repeat the log message of the original change. Instead, just indicate that this is a merge, and give the identifying revision of the original change, with at most a one-sentence summary of its effect. If someone wants to see the full log message, he should consult the original revision.
The reason it's important to avoid repeating the log message is that log messages are sometimes edited after they've been committed. If a change's log message were repeated at each merge destination, then even if someone edited the original message, he'd still leave all the repeats uncorrected—which would only cause confusion down the road.
The same principle applies to reverting a change. If a change is withdrawn from the code, then the log message for the reversion should merely state that some specific revision(s) is being reverted, not describe the actual code change that results from the reversion, since the semantics of the change can be derived by reading the original log message and change. Of course, the reversion's log message should also state the reason why the change is being reverted, but it should not duplicate anything from the original change's log message. If possible, go back and edit the original change's log message to point out that it was reverted.
All of the above implies that you should use a consistent
syntax for referring to revisions. This is helpful not only in log
messages, but in emails, the bug tracker, and elsewhere. If you're
using CVS, I suggest path/to/file/in/project/tree:REV, where
REV is a CVS revision number such
as 1.76. If you're using
Subversion, the standard syntax for revision 1729 is r1729 (file paths are not needed because
Subversion uses global revision numbers). In other systems, there is
usually a standard syntax for expressing the changeset name.
Whatever the appropriate syntax is for your system, encourage people
to use it when referring to changes. Consistent expression of change
names makes project bookkeeping much easier (as we will see in Chapter 6 and Chapter 7), and since a lot of
the bookkeeping will be done by volunteers, it needs to be as easy
as possible.
See also Section 7.7 in Chapter 7.
Most version control systems offer a feature whereby certain people can be allowed or disallowed from committing in specific sub-areas of the repository. Following the principle that when handed a hammer, people start looking around for nails, many projects use this feature with abandon, carefully granting people access to just those areas where they have been approved to commit, and making sure they can't commit anywhere else. (See Section 8.4 in Chapter 8 for how projects decide who can commit where.)
There is probably little harm done by exercising such tight control, but a more relaxed policy is fine too. Some projects simply use an honor system: when a person is granted commit access, even for a sub-area of the repository, what he actually receives is a password that allows him to commit anywhere in the project. He's just asked to keep his commits in his area. Remember that there is no real risk here: in an active project, all commits are reviewed anyway. If someone commits where he's not supposed to, others will notice it and say something. If a change needs to be undone, that's simple enough—everything's under version control anyway, so just revert.
There are several advantages to the relaxed approach. First, as developers expand into other areas (which they usually will if they stay with the project), there is no administrative overhead to granting them wider privileges. Once the decision is made, the person can just start committing in the new area right away.
Second, expansion can be done in a more fine-grained manner. Generally, a committer in area X who wants to expand to area Y will start posting patches against Y and asking for review. If someone who already has commit access to area Y sees such a patch and approves of it, he can just tell the submitter to commit the change directly (mentioning the reviewer/approver's name in the log message, of course). That way, the commit will come from the person who actually wrote the change, which is preferable from both an information management standpoint and from a crediting standpoint.
Last, and perhaps most important, using the honor system encourages an atmosphere of trust and mutual respect. Giving someone commit access to a subdomain is a statement about his technical preparedness—it says: "We see you have expertise to make commits in a certain domain, so go for it." But imposing strict authorization controls says: "Not only are we asserting a limit on your expertise, we're also a bit suspicious about your intentions." That's not the sort of statement you want to make if you can avoid it. Bringing someone into the project as a committer is an opportunity to initiate him into a circle of mutual trust. A good way to do that is to give him more power than he's supposed to use, then inform him that it's up to him to stay within the stated limits.
The Subversion project has operated on the honor system way for more than four years, with 33 full and 43 partial committers as of this writing. The only distinction the system actually enforces is between committers and non-committers; further subdivisions are maintained solely by humans. Yet we've never had a problem with someone deliberately committing outside their domain. Once or twice there's been an innocent misunderstanding about the extent of someone's commit privileges, but it's always been resolved quickly and amiably.
Obviously, in situations where self-policing is impractical, you must rely on hard authorization controls. But such situations are rare. Even when there are millions of lines of code and hundreds or thousands of developers, a commit to any given code module should still be reviewed by those who work on that module, and they can recognize if someone committed there who wasn't supposed to. If regular commit review isn't happening, then the project has bigger problems to deal with than the authorization system anyway.
In summary, don't spend too much time fiddling with the version control authorization system, unless you have a specific reason to. It usually won't bring much tangible benefit, and there are advantages to relying on human controls instead.
None of this should be taken to mean that the restrictions themselves are unimportant, of course. It would be bad for a project to encourage people to commit in areas where they're not qualified. Furthermore, in many projects, full (unrestricted) commit access has a special status: it implies voting rights on project-wide questions. This political aspect of commit access is discussed more in Section 4.3.4 in Chapter 4.