Data Manipulation with R - Second Edition

To download a package from CRAN and install it, follow these steps:

To install a package from within the R console, we can use the install.packages() command; this command will prompt us to select the appropriate server CRAN. Note that to install packages using this approach, the computer must have active Internet connection.

For example, to install the plyr package, we can use the following command:

install.packages("plyr")

The previous command will prompt us to select a regional server and, after selecting the server from the available list, the package will be installed on the local computer.

Whatever we do in R, is stored as objects. An R object is anything that can be assigned to a variable of interest. This could be a single number or a set of numbers, characters, and special characters, for example, TRUE, FALSE, NA, NaN, and Inf. Also, these can be already defined in R as functions, such as seq (to generate a sequence of numbers with a specified increment), names (to extract names such as variable names from a dataset), row.names (to extract the row names of the data, if any), or col.names (this is equivalent to names, and it extracts column names from a matrix or data frame).

Some examples of R objects are as shown in the following code:

# Constant
> 2
[1] 2
> "July"
[1] "July"
> NULL
NULL
> NA
[1] NA
> NaN
[1] NaN
> Inf
[1] Inf
# Object can be created from existing object
# to make the result reproducible means every time we run the 
# following code we will get the same results # we need to set # a seed value
> set.seed(123)  
> rnorm(9)+runif(9)
[1] -0.2325549  0.7243262  2.4482476  0.7633118  0.7697945  2.7093348  1.1166220 -0.5565308 -0.1427868

One important thing about objects in R is that, if we do not assign an object to any variable, we will not be able to reuse it, and it does not store the object internally. In the preceding example, all are different objects, but they are not assigned to any variable. So, they are not stored, and we cannot use them later, until we enter the object's value itself. Thus, whenever we deal with an object, we will assign it to an appropriate variable; interestingly, the assigned variable is also an object in R!

To assign an object in R to a variable, we can define the variable name in various ways, such as lowercase, uppercase, a combination of uppercase and lowercase, or even a combination of uppercase, lowercase, a number, and/or a dot. However, there are some rules to define variable names. For example, the name cannot start with numbers; it must start with a character or an underscore. There is no special character allowed in variable names, such as @, #, $, and *. Though R does not have a standard guideline for naming conventions, according to Bååth (in the paper The State of Naming Conventions in R, which can be found at http://journal.r-project.org/archive/2012-2/RJournal_2012-2_Baaaath.pdf), the most popular naming convention for functions is lowerCamelCase, while the most popular naming convention for arguments separates them by a period. For a variable name, we can use the same naming convention as that of arguments, but again, there is no strict rule for naming conventions in R.

The following table is constructed from the same article by Bååth to give you an idea of the different naming conventions used in R and their popularity:

Once we store the R object into a variable, it is still treated as an R object. Each and every object in R has some attributes to describe the nature of the information contained in it. The mode and class are the most important attributes of an R object. Commonly encountered modes of an individual R object are numeric, character, and logical. When we work with data in R, problems may arise due to incorrect operations in incorrect object modes. So, before working with data, we should study the mode; we need to know what type of operation is applicable.

The mode function returns the mode of R objects.

The following example code describes how we can investigate the mode of an R object:

# Storing R object into a variable and then viewing the mode

> num.obj <- seq(from=1,to=10,by=2)
mode(num.obj)
[1] "numeric"

> logical.obj<-c(TRUE,TRUE,FALSE,TRUE,FALSE)
> mode(logical.obj)
[1] "logical"
> character.obj <- c("a","b","c")
> mode(character.obj)
[1] "character"

For the numeric mode, R stores all numeric objects into either a 32-bit integer or a double-precision floating point.

If an R object contains both numeric and logical elements, the mode of that object will be numeric and, in this case, the logical element automatically gets converted to a numeric element. The logical element TRUE converts to 1 and FALSE converts to 0. On the other hand, if any R object contains a character element, along with both numeric and logical elements, it automatically converts to the character mode.

Let's have a look at the following code:

# R object containing both numeric and logical element
> xz <- c(1, 3, TRUE, 5, FALSE, 9)
> xz
[1] 1 3 1 5 0 9
> mode(xz)
[1] "numeric"

# R object containing character, numeric, and logical elements
> xw <- c(1,2,TRUE,FALSE,"a")
> xw
[1] "1"     "2"     "TRUE"  "FALSE" "a"    
> mode(xw)
[1] "character"

The mode() function is not the only way to test R object modes. There are alternative ways too: is.numeric(), is.charater(), and is.logical(), as shown in the following code. The output of these functions is always logical:

> num.obj <- seq(from=1,to=10,by=2)
> logical.obj<-c(TRUE,TRUE,FALSE,TRUE,FALSE)
> character.obj <- c("a","b","c")

> is.numeric(num.obj)
[1] TRUE
> is.logical(num.obj)
[1] FALSE
> is.character(num.obj)
[1] FALSE

Other than these three modes (numeric, logical, and character) of objects, another frequently encountered mode is function. Here is an example:

> mode(mean)
[1] "function"
# Also we can test whether "mean" is function or not as follows
> is.function(mean)
[1] TRUE

The class() function provides the class information of an R object. The primary purpose of the class() function is to know how different functions, including generic functions, work. For example, with the class information, the generic function print or plot knows what to do with a particular R object. To assess the class information of the object created earlier, we can use the class() function. Let's have a look at the following code:

> num.obj <- seq(from=1,to=10,by=2)
> logical.obj<-c(TRUE,TRUE,FALSE,TRUE,FALSE)
> character.obj <- c("a","b","c")

> class(num.obj)
[1] "numeric"

> class(logical.obj)
[1] "logical"

> class(character.obj)
[1] "character"

Although we can easily assess the mode and class of an R object through mode() and class(), there is another collection of R commands that is also used to assess whether a particular object belongs to a certain class. These functions start with is.; for example, is.numeric(), is.logical(), is.character(), is.list(), is.factor(), and is.data.frame(). As R is an object-oriented programming language, there are many functions (collectively known as generic functions) that will behave differently depending on the class of that particular object.

The mode of an object tells us how it's stored. It could happen that two different objects are stored in the same mode with different classes. How the two objects are printed using the print command is determined by its class. Here is an example:

# Output omitted due to space limitation
> num.obj <- seq(from=1,to=10,by=2)
> set.seed(1234) # To make the matrix reproducible
> mat.obj <- matrix(runif(9),ncol=3,nrow=3)
> mode(num.obj)
> mode(mat.obj)
> class(num.obj)
> class(mat.obj)
# prints a numeric object
> print(num.obj) 
# prints a matrix object
> print(mat.obj)

Like character and numeric, there is another method you can use to store data when the data is categorical in nature. In categorical data, we usually have some unique values and their corresponding labels. To store this type of object in R, we use the factor class. This class allows less storage location, because it is required to store unique levels only once.

Interestingly, once we try to see the mode of a factor object, it always shows as numeric, even if it displays character data. Here is an example:

> character.obj <- c("a","b","c")
> character.obj
[1] "a" "b" "c"

> is.factor(character.obj)
[1] FALSE

# Converting character object into factor object using as.factor()
> factor.obj <- as.factor(character.obj)
> factor.obj
[1] a b c
Levels: a b c 

> is.factor(factor.obj)
[1] TRUE

> mode(factor.obj)
[1] "numeric"

> class(factor.obj)
[1] "factor"

We have to be careful when dealing with the factor class data in R. The important thing to remember is that, for vectors (we will discuss vectors in the Vector section in this chapter), the class and mode will always be numeric, logical, or character. On the other hand, for matrices and arrays (we will discuss matrices and arrays in the Factor and its types section in this chapter), a class is always a matrix or array, but its mode can be numeric, character, or logical.

R is a domain-specific programming language, specially designed to perform statistical analysis on data. In statistics, when we analyze data, the first thing that comes to mind is a variable with hundreds of observations in it. This reminds us of the picture of a vector. Probably, this is the main reason why, in R, the most elementary data type is a vector. A vector is a contiguous cell that contains data, where each cell can be accessed by an index:

> age <- c(10,20,30,40)

This is an example of a vector. The age of five individuals is stored in the age vector. Pay attention to how the vector was formed and stored under the age variable. Here, c() is a function used to create a vector, but this does not store all the data in the system. <- is called an assignment operator that is used to store a vector under a variable.

Now, in the console, let's type the following line and press Enter:

> age
 [1] 10 20 30 40

We successfully stored all the ages under the age variable, but what is [1]? This means that the index of the value 10 is 1.

If you want to see the first values of the vector, type the following command:

> age[3]
[1] 30

Why did R only show the index of the first value and not the other values? This is only to keep the output clean and informative. Every time R writes a new line, it first gives the index number of the next value. Pretty soon, you will be familiar with this convention. We can store a single value under a variable, but it will be a vector with one element:

> height<- 175

To show you that height is not a scalar but a vector with one element, we will store one additional value in it:

> height[2]<- 180

Pay attention to how we added another value inside an existing vector. Here, we put 180 in the second cell of the vector. Can you recall how we accessed the value in the second cell for the age variable? Using age[2], right? Similarly, we can assign a value to the second cell of the vector using the same syntax. Let's try to put another value inside the height variable:

> height[3] <- 165

Now, we can see all the values stored inside the height variable:

> height
[1] 175 180 165   

Although the basic data structure in R is vectors, there can be different types of vector. We use a numeric vector to store numeric data such as age, height, weight, and so on. Character vectors are used to store string data such as name, address, and so on. The way we can define a character vector in R is simple:

> name<- c("Rob", "Bob", "Jude","Monica")

When we want to store a character in R, we need to use double quotes, as used in the previous example. This tells R that this is a string input. We can put numeric values using double quotes but, if we use a character without double quotes, then it will return an error message.

Another special type of vector is the logical vector. There are two ways we could define a logical vector; first, we will show you the more formal way and, second, we will show you the quick way. There can be two possible elements in a logical vector: TRUE and FALSE. This logical vector is used in logical operations in R. It can be used to select specific rows from a dataset.

We can define a logical vector in the following way:

> logical<- c(TRUE, FALSE, TRUE, FALSE)

This logical vector can be used as a row selector of the age vector in the following way:

> age[logical]
[1] 10 30

Look closely to find out what we just did. We have seen how we can extract age from a vector using indexing. A logical vector can be thought of as a vector of an index. The first element of the logical vector is TRUE, which means that the first element of the age vector will be selected. The second element of the logical vector is FALSE. This means that the second element of the age vector will not be selected. So, the logical vector will select only the elements of the age vector for which the logical vector is TRUE. So, finally, two elements of the age vector will be selected, and a vector of two elements will be returned. A question that may come to your mind is, What can we do with this feature? The answer will be clearer in the Data frame section.

A data frame is a rectangular arrangement of rows and columns of vectors and/or factors, such as a spreadsheet in MS Excel. The columns represent variables in the data, and the rows represent observations or records. In other software, such as a database package, each column represents a field, and each row represents a record. Dealing with data does not mean dealing with only one vector or factor variable; it is rather a collection of variables. Each column represents only one type of data: numeric, character, or logical. Each row represents case information across all columns. One important thing to remember about R data frames is that all vectors should be of the same length. In an R data frame, we can store different types of variables, such as numeric, logical, factor, and character. To create a data frame, we can use the data.frame() command.

The following example shows us how to create a data frame using different vectors and factors:

#creating vector of different variables and then creating data frame
> var1 <- c(101,102,103,104,105)
> var2 <- c(25,22,29,34,33)
> var3 <- c("Non-Diabetic", "Diabetic", "Non-Diabetic", "Non-Diabetic", "Diabetic")
> var4 <- factor(c("male","male","female","female","male"))
# now we will create data frame using two numeric vectors one 
# character vector and one factor
> diab.dat <- data.frame(var1,var2,var3,var4)
> diab.dat
   var1 var2         var3   var4
1  101   25 Non-Diabetic   male
2  102   22     Diabetic   male
3  103   29 Non-Diabetic female
4  104   34 Non-Diabetic female
5  105   33     Diabetic   male

Now, if we look at the class of individual columns of the newly created data frame, we will see that the first two columns' classes are numeric, and the last two columns' classes are factor, though, initially, the class of var3 was character. One thing is obvious here—when we create data frames and any one of the column's classes is character, it automatically gets converted to factor, which is a default R operation. However, there is one argument, stringsAsFactors=FALSE, that allows us to prevent the automatic conversion of character to factor during data frame creation.

In the following example, we will see this:

#class of each column before creating data frame 
> class(var1)
[1] "numeric"
> class(var2)
[1] "numeric"
> class(var3)
[1] "character"
> class(var4)
[1] "factor"
# class of each column after creating data frame
> class(diab.dat$var1)
[1] "numeric"
> class(diab.dat$var2)
[1] "numeric"
> class(diab.dat$var3)
[1] "factor"
> class(diab.dat$var4)
[1] "factor"
# now create the data frame specifying as.is=TRUE
> diab.dat.2 <- data.frame(var1,var2,var3,var4,stringsAsFactors=FALSE)
> diab.dat.2
var1 var2         var3   var4
1  101   25 Non-Diabetic   male
2  102   22     Diabetic   male
3  103   29 Non-Diabetic female
4  104   34 Non-Diabetic female
5  105   33     Diabetic   male

> class(diab.dat.2$var3)
[1] "character"

To access individual columns (variables) from a data frame, we can use a dollar ($) sign, along with the data frame name–for example, diab.dat$var1.

There are some other ways to access variables from a data frame, such as the following:

Besides these, there is one other way that allows us to access each of the individual variables as separate objects. The R attach() function allows us to access individual variables as separate R objects. When we use the attach() command, we need to use detach() to remove individual variables from the working environment.

Let's have a look at the following code:

# To run the folloing code snipped, 
# the code block 16 need to run.
# Especially var1 var2 var3 and var4. 
# After that, from code block 17 "diab.dat.2" object should run
 
# The following line will remove var1 to var4 
# object from the workspace
> rm(var1);rm(var2);rm(var3);rm(var4)
# The following command will allow 
# us to access individual variables 
> attach(diab.dat.2)
# Printing valuse of var1
> var1
# checking calss of var3
> class(var3)
# Now to detach the data frame from the workspace
> detach(diab.dat.2)
# Now if we try to print individual varaiable it will give error
> var1

A matrix is also a two-dimensional arrangement of data, but it can take only one class. To perform any mathematical operations, all columns of a matrix should be numeric. However, in data frames, we can store numeric, character, or factor columns. To perform any mathematical operation, especially a matrix operation, we can use matrix objects. However, in data frames, we are unable to perform certain types of mathematical operation, such as matrix multiplication. To create a matrix, we can use the matrix() command or convert a numeric data frame to a matrix using as.matrix().

We can convert the data frame that we created earlier as diab.dat to a matrix using as.matrix(). However, this is not suitable for performing mathematical operations, as shown in the following example:

# data frame to matrix conversion
> mat.diab <- as.matrix(diab.dat)
> mat.diab
     var1  var2 var3           var4    
[1,] "101" "25" "Non-Diabetic" "male"  
[2,] "102" "22" "Diabetic"     "male"  
[3,] "103" "29" "Non-Diabetic" "female"
[4,] "104" "34" "Non-Diabetic" "female"
[5,] "105" "33" "Diabetic"     "male"

> class(mat.diab)
[1] "matrix"
> mode(mat.diab)
[1] "character"

# matrix multiplication is not possible with this newly created matrix

> t(mat.diab) %*% mat.diab
Error in t(mat.diab) %*% mat.diab : 
requires numeric/complex matrix/vector arguments

# creating a matrix with numeric elements only
# To produce the same matrix over time we set a seed value
> set.seed(12345) 
> num.mat <- matrix(rnorm(9),nrow=3,ncol=3)
> num.mat
           [,1]       [,2]       [,3]
[1,]  0.5855288 -0.4534972  0.6300986
[2,]  0.7094660  0.6058875 -0.2761841
[3,] -0.1093033 -1.8179560 -0.2841597

> class(num.mat)
[1] "matrix"
> mode(num.mat)
[1] "numeric"

# matrix multiplication
> t(num.mat) %*% num.mat
          [,1]       [,2]       [,3]
[1,] 0.8581332 0.36302951 0.20405722
[2,] 0.3630295 3.87772320 0.06350551
[3,] 0.2040572 0.06350551 0.55404860

An array is a multiply subscripted data entry that allows the storing of data frames, matrices, or vectors of different types. Data frames and matrices are of two dimensions only, but an array can be of any number of dimensions. Sometimes, we need to store multiple matrices or data frames into a single object; in this case, we can use arrays to store this data.

Here is a simple example to store three matrices of order 2 x 2 in a single array object:

> mat.array=array(dim=c(2,2,3))

# To produce the same results over time we set a seed value
> set.seed(12345)

> mat.array[,,1]<-rnorm(4)
> mat.array[,,2]<-rnorm(4)
> mat.array[,,3]<-rnorm(4)

> mat.array
, , 1

          [,1]       [,2]
[1,] 0.5855288 -0.1093033
[2,] 0.7094660 -0.4534972

, , 2

           [,1]       [,2]
[1,]  0.6058875  0.6300986
[2,] -1.8179560 -0.2761841

, , 3

           [,1]       [,2]
[1,] -0.2841597 -0.1162478
[2,] -0.9193220  1.8173120

A list object is a generic R object that can store other objects of any type. In a list object, we can store single constants, vectors of numeric values, factors, data frames, matrices, and even arrays.

Recalling the var1, var2, var3, and var4 vectors, the data frame created using these vectors, and also recalling the array created in the Arrays section, we will create a list object in the following example:

> var1 <- c(101,102,103,104,105)
> var2 <- c(25,22,29,34,33)
> var3 <- c("Non-Diabetic", "Diabetic", "Non-Diabetic", "Non-Diabetic", "Diabetic")
> var4 <- factor(c("male","male","female","female","male"))
> diab.dat <- data.frame(var1,var2,var3,var4)

> mat.array<-array(dim=c(2,2,3))

> set.seed(12345)

> mat.array[,,1]<-rnorm(4)
> mat.array[,,2]<-rnorm(4)
> mat.array[,,3]<-rnorm(4)

# creating list
> obj.list <- list(elem1=var1,elem2=var2,elem3=var3,elem4=var4,elem5=diab.dat,elem6=mat.array) 


> obj.list
$elem1
[1] 101 102 103 104 105

$elem2
[1] 25 22 29 34 33

$elem3
[1] "Non-Diabetic" "Diabetic"     "Non-Diabetic" "Non-Diabetic" "Diabetic"    

$elem4
[1] male   male   female female male  
Levels: female male

$elem5
  var1 var2         var3   var4
1  101   25 Non-Diabetic   male
2  102   22     Diabetic   male
3  103   29 Non-Diabetic female
4  104   34 Non-Diabetic female
5  105   33     Diabetic   male

$elem6
, , 1

          [,1]       [,2]
[1,] 0.5855288 -0.1093033
[2,] 0.7094660 -0.4534972

, , 2

           [,1]       [,2]
[1,]  0.6058875  0.6300986
[2,] -1.8179560 -0.2761841

, , 3

           [,1]       [,2]
[1,] -0.2841597 -0.1162478
[2,] -0.9193220  1.8173120

To access individual elements from a list object, we can use the name of that element or use double square brackets with the index of those elements. For example, obj.list[[1]] will give the first element of the newly created list object.