This chapter introduces the technology, terminology, and concepts at the core of Perforce operation. A basic knowledge of these core factors will make it easier to relate interface activities to the results you're trying to achieve. These high-level introductions provide the basic framework for their specific application later in this book.
In this chapter we will cover:
Version control and Software Configuration Management (SCM)
The roles of Perforce servers and clients
What repositories and depots are and how they relate to workspaces
The core Perforce concept of a changelist
Practical issues when using Perforce
Tips for understanding the Perforce mindset
What is version control and what does a version control tool such as Perforce do for us? To many people version control is self-evident. They use it and talk about it without giving it much thought. Yet, like other self-evident activities, a little knowledge and experience can help you avoid problems and become a significantly more successful user.
When most people are asked to define version control they typically start by saying something along the lines of "tracking changes to files". This is followed by a long, disjointed list of features and buzzwords. Others attempt to abstract the details and talk about concepts or tasks. However, we have found that it is usually easier to describe what version control provides by imagining life without a version control tool such as Perforce.
The basic usage of any computer includes sets of files. These files might represent the source code of a program, or a website, or be the test data for an integrated circuit, and many other collections of files. Without version control, you have to manage the files using just the operating system tools. Version control makes the management of these collections both easier and less risky.
If you have a set of files you might want to change them. This might mean adding new files, modifying existing files, and deleting files that are no longer required. But you might also want to track why you made these changes in some form of a change log or even a document or spreadsheet. That introduces potential sources for ambiguity and error as you attempt to transfer information between the different tools. Don't forget the complexities associated with files and directories that are renamed or moved. Version control tracks all of these changes, without error, using a single tool.
At some point you realize that you might want to recover older versions of files in your collection. So you start to make copies of individual files or even the entire collection, for example, when you make a release. Another reason for copying files is when you might want to isolate changes to your files because you're not sure if what you want to do is possible or appropriate. Deciding what to copy, individual files or entire collections, isn't always particularly obvious, especially when you try to account for renames and moves. Version control manages storage space concerns and gives you access to older file versions quickly and easily. It also helps you integrate the changes you made in a copy with the primary versions of the files.
All of this was just considering the needs of a single user. What would happen if more than one user wanted to make changes to the same set of files? How would they avoid or resolve conflicts when working concurrently? How would they share work? Without a version control tool the answers to these questions get complicated very quickly.
So whatever your problem, version control is the solution, well perhaps that's a minor exaggeration, but it is incredibly useful!
The terms version control and software configuration management have been in use for many years. Within the software development community they are often used interchangeably. Outside the realm of software development people often refer to SCM as a specifically software related activity. All of which can lead to reader confusion.
To reduce this, we avoid using the phrase SCM in this book. When necessary, the phrase version control will refer to Perforce features dealing with the versions of a single file; and the phrase configuration management will refer to Perforce features dealing with a collection of files.
Centralized version control and distributed version control are phrases commonly used to differentiate version control systems. Centralized systems (including Subversion, Perforce, and many other commercial tools) have a central, shared master repository of version information. Distributed systems (such as Git and Mercurial) provide each individual user with their own private copy of a repository.
Each technology has strengths and weaknesses relating to performance, communication, and security. Centralized systems often provide a way to operate even when they cannot access the central repository, and distributed systems still need a master repository if more than one user is doing work on the same code base. Perforce further blurs this distinction by being a centralized system with full support for distributed users. It now includes a bidirectional interface called Git Fusion in its recent versions. This allows developers to use Git if they wish, and then use Perforce as the master repository for sharing with other people and the rest of the organization.
The basic Perforce architecture is client and server. There is a single server program (P4D) managing a repository and typically many client programs (such as P4V or P4) which communicate with that server as shown in the following diagram:
The server is responsible for maintaining the history of the contents of files, and for all of the information that describes the events that created that content (the metadata). Each client program is responsible for managing files and their contents within a particular workspace, and for communicating status changes to the server. In order to manage files a client program must connect to a server. Connecting to a server is covered in Chapter 2, The P4V GUI.
Users and client programs have no direct access to the repository managed by the server, so it is best to view the server as a black box. It is still useful to understand some of the services that the server provides.
The obvious services are to track the historic content of all the files managed by the server. The most visible part of this tracking is the actual content of all the versions of every file. An equally important part of this tracking is the metadata that the server maintains about file versions. Metadata describes details such as user, workspace, time, date, and check in comments, these are all part of the context in which the historic content was created.
Another service provided by the server is to track the state of files within each workspace, which it does in coordination with the client program. The state includes things such as whether files are present in the workspace, which version is present, whether the user has checked out particular files or versions, and so on.
The most important service provided by the server is to control and coordinate change, so having the server store information, rather than the client program, makes it available to everyone. This supports good engineering practice in the modern dynamic development environment. For example, when you check out a file the server tells you if the same file is checked out in other workspaces. A file checked out in multiple workspaces at the same time may or may not be a problem. Only a human can make that decision.
Client programs manage workspaces. At any one time, a client program manages a single workspace. It is a straightforward and fast operation for a client program to manage another workspace as necessary. It is common for users to have multiple workspaces on their local workstations such as their PC or laptop.
Note
Terminology warning – the word "client" has three meanings!
The word client is used in three different ways when talking about Perforce: to mean a client program, a client machine, or a client workspace. The first two meanings are fairly standard, but you will see the "client workspace" meaning in some of the Perforce documentation and in warning or error messages within P4V. The context usually makes it clear which meaning is intended. Unless otherwise specified, we use client to mean a client program.
Clients and their workspaces are isolated and independent. Clients must communicate with the server to determine what is going on in other workspaces (whichever machine those workspaces might be on). This is true even if there is more than one client running on a single machine. If you tell the server about changes that you intend to make within a workspace such as an edit, add, delete, or rename, this information can be communicated to other clients and thus to other users.
Clients can operate even if they are not connected with the server. This can be useful at customer sites, in airplanes, and at times when communication with the server is either not available or unstable. Because of their specific roles, the client and the server can always determine how to bring the client context up-to-date when they next communicate.
This section introduces the core Perforce concepts and the contexts they apply to.
Depot is a word that you will frequently see within this book, Perforce documentation, and in the P4V interface. In general, it refers to a directory structure that the server uses to store the historic content of files.
The repository that the server manages consists of one or more depots. Each depot has a unique name. A depot acts like a hierarchical file system containing folders and files. It is very common for a repository to have multiple depots. Creating depots is an administrator function.
Workspace is a common word within the source control community. Unfortunately, there seem to be more definitions for workspace than there are tools that support them. For Perforce, the term workspace refers to a collection of files, usually on a user's local machine, that the user wants Perforce to manage. Implicit to the Perforce concept of workspace is the relationship between repository files and the files in that workspace.
Workspaces usually contain a copy of a small subset of all the files in the repository, for example, only the files relevant to a particular project. Users operate on the copies of the files in their workspace, and submit their changes back to the repository. If another user has their workspace configured to see the same project, they can update their workspace to receive the latest copies of any updated files whenever they are ready to do so. In the following diagram, two developers (Fred and Wilma) exchange changes between their workspaces via the repository:
There are many aspects of workspaces that a user may want to manage. However, for basic operations a single workspace on your workstation may be all that you need. Managing workspaces is covered in Chapter 6, Managing Workspaces.
Changelist is a core Perforce concept. It is the fundamental unit of change to the repository, you don't check in files one-by-one, you check in changelists one-by-one. Each changelist contains one or many files. Changelists with thousands, tens of thousands, and even more files are not uncommon. In each workspace, it is possible to have multiple active changelists that are not yet checked in.
Changelist refer to a specific set of content and metadata changes made to the files managed by the server. Changelists have a unique numerical identifier and other attributes such as status, description, date and time of submission, and the user who submitted it. More than just a set of changes, changelists are also atomic. That is, changelists don't overlap with each other. Two changelists might be submitted at the same time, but the server will ensure that processing is completed for one changelist entirely before it starts processing the second changelist; changelists will never be intermingled by the server. Furthermore, once the server has successfully completed processing a changelist that changelist becomes immutable. That is, users can't add, remove, or modify any of the file changes represented by the changelist.
A reasonable analogy is to think of a changelist as a transaction. The Perforce equivalent is that if your changelist contains ten files, then either all ten will be submitted, or none of them. You will never get only a part of the total files submitted.
Many Perforce customers have servers handling thousands of users, and millions or more changelists. Changelists are ideal reference points for build systems, release processes, validation audits, and myriad other uses where a reproducible and verifiable set of managed sources is required.
In the following diagram, two files from the workspace are submitted as a single changelist to the repository. The changelist will automatically be assigned a unique number from an increasing sequence by the server. We cover this in more detail in Chapter 4, Changelists, including changelist states such as pending and submitted.
Perforce is a very straight forward system to install on various operating systems. It is not deeply embedded and thus it is relatively independent of different versions of operating systems.
Unlike many version control systems, Perforce does not require that clients and servers be at the exact same version. When a client connects with a server, they both determine the set of features they have in common and act accordingly. Some clients require a minimum server version. The 2013.2 version of P4V requires a server that is version 2008.1 or later.
The examples and images in this book are based on the 2013.2 version of P4V and the 2013.2 version of the Perforce server. If your P4V or server versions are not the same, you may see slightly different choices and options. The information you'll find in this book is still applicable.
Perforce servers and clients are available for a wide range of operating systems. The 2013.2 release of P4V is available for Windows, Linux, and Macintosh. You can download any of these from the Perforce website. However, some organizations have standardized on specific (older) releases. Check with your system administrator as appropriate.
Clients and servers can be running on different operating systems and still be able to work together. The servers take care of the differences between client and server host machines. The clients take care of issues that are specific to the host machine on which they operate such as text file line endings and directory separator characters.
As a user you may need to deal with character case sensitivity in directory paths and filenames. This typically comes up when a Windows-based client connects with a Perforce server that is not Windows-based. We discuss this in more detail in Chapter 6, Managing Workspaces, which focuses on workspaces. That chapter also addresses settings to handle different line endings between Windows and Linux/Unix operating systems.
Another issue that may come up is internationalization. The server is responsible for the character set used for the names of files or any part of their path. This allows you to store files with Unicode names, such as クライアントサーバモデル
.doc which is the Japanese name for a file which would be called client-server model.doc in English. Consult with your Perforce administrator for further details. The client is responsible for dealing with the character set used within the contents of files.
This book uses P4V as the client interface. P4V is a good choice as it has many presentation and graphical features. These features help to manage the large amount of information that typically describes files in a project.
P4 is the command-line client: p4.exe on Windows or just p4 on Linux/Unix. P4 is available for all of the operating systems supported by Perforce. The p4 client is ideal for automation and scripting. To assist with automation, there are language-specific interfaces providing optimized ways to execute p4 commands and process their results. These include interfaces for Perl, Ruby, Python, .NET, and Java. There is a logging option in P4V that shows the basic p4 commands being used to provide the P4V functionality. This can be very useful for individuals learning to how to automate actions or processes.
Appendix B, Command Line helps to relate the p4 commands to the P4V operations described in the various chapters.
Perforce integrations with IDEs such as Visual Studio and Eclipse are also common. All of the concepts and command features described within this book apply to the IDE integrations. It is not uncommon for users to use both P4V and an IDE integration. P4V provides access to the advanced information management features that don't always fit within the context of an IDE.
There are many more interfaces and integrations to Perforce. See the Perforce website (www.perforce.com) for the current list.
Unlike some other version control tools, the Perforce clients and the server do not perform actions unless they are told explicitly to do them. So your workspace files will not be updated unless you select an action or run a command to do so. Your files will not be submitted to the server unless you, the user, select that action.
It is also expected that users inform the server about changes they intend to make to workspace files by running specific client commands such as add or edit. This allows the server to communicate those status changes to other users. It is a slightly different way of working to tools such as Git or Subversion, where users just start editing files locally and then resolve the status later at the time of commit.
You can get a lot of valuable information by just reading this book. You don't have to touch a keyboard, move a mouse, or click on a button. However, the authors have found that trying the described actions, experimenting with what ifs, and making a few mistakes, is a far better way to learn and will help you retain the information. Unfortunately, even if there weren't licensing and resource considerations, experimenting with source files on a production server is not usually encouraged. So it can be hard to get the hands-on experience that makes you a more effective tool user.
Fortunately, a Perforce server doesn't need extraordinary resources or licensing to operate. A server and client can easily be run on the same host machine including laptops. Appendix A, A Demo Server of this book describes how to obtain and install your own Perforce server and a test repository. All of the examples within this book were created using such a configuration.