R Data Science Essentials: R Data Science Essentials

Importing data to R is quite simple and can be done from multiple sources. The most common method of importing data to R is through the comma-separated values (CSV) format. The CSV data can be accessed through the read.csv function. This is the simplest way to read the data as it requires just a single line command and the data is ready. Depending on the quality of the data, it may or may not require processing.

data <- read.csv("c:/local-data.csv")

The other function similar to read.csv is read.csv2. This function is also used to read the CSV files but the difference is that read.csv2 is mostly used in the European countries, where comma is used as decimal point and semicolon is used as a separator. Also, the data can be read from R using a few more parameters, such as read.table and read.delim. By default, read.delim is used to read tab-delimited files, and the read.table function can be used to read any file by supplying suitable parameters as the input:

data  <- read.delim("local-data.txt", header=TRUE, sep="\t")
data  <- read.table("local-data.txt", header=TRUE, sep="\t")

All the preceding functions can take multiple parameters that would explain the data source's format at best. Some of these parameters are as follows:

These are some of the common parameters used along with the functions to read the data from a file.

We have so far explored reading data from a delimited file. In addition to this, we can read data in Excel formats as well. This can be achieved using the xlsx or XLConnect packages. We will see how to use one of these packages in order to read a worksheet from a workbook:

install.packages("xlsx")
library(xlsx)
mydata <- read.xlsx("DTH AnalysisV1.xlsx", 1)
head(mydata)

In the preceding code, we first installed the xlsx package that is required to read the Excel files. We loaded the package using the library function, then used the read.xlsx function to read the excel file, and passed an additional parameter, 1, that specifies which sheet to read from the excel file.

Apart from reading the data from a local file, R allows us to read the data from different sources and different formats. If we consider any enterprise setup, the data would mostly be present in a database. It will be complicated if we import the data from the database to a local file and then perform the analysis; we should be able to access the data directly from the source. This can be achieved using R.

First, let's see how to access data from a database. In order to read data from a database, we need to establish a connection with the database, which could reside in the local system or a remote server. We can establish the connection to the database using either ODBC or JDBC, which are R packages.

We will have a detailed look at accessing the data from the database using the JDBC method. In order to perform this operation, we need to install the RJDBC and sqldf packages. The RJDBC package is used to establish a connection with the database and the sqldf package is used to write the SQL queries:

install.packages("RJDBC")
library(RJDBC)
install.packages("sqldf")
library(sqldf)

We will now learn to establish a connection with the DB. We need to set up a few things in order to connect with the DB. To use the JDBC connection, we need to download the driver. The downloaded file will depend on the database to which we are going to connect, such as SQL Server, Oracle, or PostgreSQL.

In the following case, we will connect to a SQL server database. The JDBC driver can be downloaded from http://www.microsoft.com/en-in/download/details.aspx?id=11774 in order to provide connectivity. In the following code, we will pass the driver name as well as the location of the JAR file that comes with the download to the JDBC function. The JDBC function creates a new DBI driver that can be used to start the JDBC connection.

drv <- JDBC("com.microsoft.sqlserver.jdbc.SQLServerDriver", "C:/Users/Downloads/Microsoft SQL Server JDBC Driver 3.0/sqljdbc_3.0/enu/sqljdbc4.jar")

By using the dbConnect function, we establish the actual connection with the database. We need to pass the location of the database, username, and password to this function. On the successful execution of the following code, the connection will be established and we can check the connectivity using the dbGetQuery function:

conn <- dbConnect(drv, "jdbc:sqlserver://localhost;database=SAMPLE_DB", "admin", "test")
bin <- dbGetQuery(conn, "select count(*) from  sampletable")

In addition to the relational databases, we can also connect and access the non-relational databases, such as Cassandra, Hadoop, MongoDB, and so on. We will now see how to access data from a Cassandra database. In order to access the data from a Cassandra database, we need to install the RCassandra package. The connection to the database can be made using the RC.connect function. We need to pass the host IP as well as the port number to establish the connection. Then finally, we need to specify the username and password as follows to establish the connection successfully:

library(RCassandra)
conn <- RC.connect(host = "localhost", port = 9160)
RC.login(conn, username = "user", password = "password123")

In Cassandra, the container for the application data is keyspace, which is similar to schema in the relational database. We can use the RC.describe.keyspaces function to get an understanding about the data, and then using the RC.use function, we select keyspace to be used for all the subsequent operations:

RC.describe.keyspaces(conn)
RC.use(conn, keyspace = "sampleDB", cache.def = TRUE)

We can read the data using the following code and once all the readings are done, we can close the connection using RC.close:

a<-RC.read.table(conn, c.family = "Users", convert = TRUE, na.strings = "NA",
              as.is = FALSE, dec = ".")
RC.close(conn)

We can discuss the RMongo package as well as it is quite popular.

Similarly, R has the ability to read data from a table in a website using the XML package. We can also read the data of SAS, SPSS, Stata, and Systat using the Hmisc package for SPSS and SAS and foreign for Stata and Systat.

For more details about the methodology of extracting the data from these sources, find the reference at the following URLs:

While establishing connectivity with a remote system, you could face a few issues related to security and others specific to the R version and package version. Most likely, the common issues would have been discussed in the forum, stackoverflow.

We explored the various ways that we can read the data from R in the previous session. Let's have a look at the various data types that are supported by R. Before going into the details of the data type, we will first explore the variable data types in R.

a <- 10
class(a)
[1] "numeric"

a <- as.integer(a)
class(a)
[1] "integer"

name <- "Sharan"
class(name)
[1] "character"

# Logical Type
flag <- TRUE
class(flag)
[1] "logical"

v1 <- c(12, 34, -21, 34.5, 100) # numeric vector
class(v1)
 [1] "numeric"
v2 <- c("sam", "paul", "steve",  "mark") # character vector
class(v2)
[1] "character"
v3 <- c(TRUE, FALSE, TRUE, FALSE, TRUE, FALSE) #logical vector
class(v3)
[1] "logical"

newV <- c(v1,v2)
class(newV)
[1] "character"

newV
 [1] "12"    "34"    "-21"   "34.5"  "100"   "sam"   "paul"  "steve" "mark" 

rnames <- c("R1", "R2", "R3", "R4", "R5")
cnames <- c("C1", "C2", "C3", "C4", "C5")
matdata <-matrix(1:25, nrow=5,ncol=5, dimnames=list(rnames, cnames))
class(matdata)
[1] "matrix"
typeof(matdata)
[1] "integer"
Matdata
C1 C2 C3 C4 C5
R1  1  6 11 16 21
R2  2  7 12 17 22
R3  3  8 13 18 23
R4  4  9 14 19 24
R5  5 10 15 20 25

typeof(l1)
> l1
[[1]]
[1]  12.0  34.0 -21.0  34.5 100.0

[[2]]
[1] "sam"   "paul"  "steve" "mark" 

[[3]]
[1]  TRUE FALSE  TRUE FALSE  TRUE FALSE

The first step after loading the data to R would be to check for possible issues such as missing data, outliers, and so on, and, depending on the analysis, the preprocessing operation will be decided. Usually, in any dataset, the missing values have to be dealt with either by not considering them for the analysis or replacing them with a suitable value.

To make this clearer, let's use a sample dataset to perform the various operations. We will use a dataset about India named IndiaData. You can find the dataset at https://github.com/rsharankumar/R_Data_Science_Essentials. We will perform preprocessing on the dataset:

data <- read.csv("IndiaData.csv", header = TRUE)
#1. To check the number of Null
sum(is.na(data))
[1] 6831

After reading the dataset, we use the is.na function to identify the presence of NA in the dataset, and then using sum, we get the total number of NAs present in the dataset. In our case, we can see that a large number of rows has NA in it. We can replace the NA with the mean value or we can remove these NA rows.

The following function can be used to replace the NA with the column mean for all the numeric columns. The numeric columns are identified by the sapply(data, is.numeric) function. We will check for the cells that have the NA value, then we identify the mean of these columns using the mean function with the na.rm=TRUE parameter, where the NA values are excluded while computing the mean function:

for (i in which(sapply(data, is.numeric))) {
  data[is.na(data[, i]), i] <- mean(data[, i],  na.rm = TRUE)
}

Alternatively, we can also remove all the NA rows from the dataset using the following code:

newdata <- na.omit(data)

The next major preprocessing activity is to identify the outliers package and deal with it. We can identify the presence of outliers in R by making use of the outliers function. We can use the function outliers only on the numeric columns, hence let's consider the preceding dataset, where the NAs were replaced by the mean values, and we will identify the presence of an outlier using the outliers function. Then, we get the location of all the outliers using the which function and finally, we remove the rows that had outlier values:

install.packages("outliers")
library(outliers)

We identify the outliers in the X2012 column, which can be subsetted using the data$X2012 command:

outlier_tf = outlier(data$X2012,logical=TRUE)
sum(outlier_tf)
[1] 1

#What were the outliers
find_outlier = which(outlier_tf==TRUE,arr.ind=TRUE)
#Removing the outliers
newdata = data[-find_outlier,]
nrow(newdata)

The column from the preceding dataset that was considered in the outlier example had only one outlier and hence we can remove this row from the dataset.

The following are the different data operations available in R:

a1 <- c(1,2,3,4,5)
b1 <- c(6,7,8,9,10)
c1 <- a1+b1
[1]  7  9 11 13 15
c1 <- b1-a1
 [1] 5 5 5 5 5
c1 <- b1*a1
 [1]  6 14 24 36 50
c1 <- b1/a1
 [1] 6.000000 3.500000 2.666667 2.250000 2.000000

c1 <- b1/a1
c1 <- b1 %% a1

x <- c(1:10)
x[(x>=8) | (x<=5)]

matdata1 <-matrix(1:25, nrow=5,ncol=5, dimnames=list(rnames, cnames))
matdata2 <-matrix(1:25, nrow=5,ncol=5, dimnames=list(rnames, cnames))
newmat <- matdata1 * matdata2
newmat

x <- "The Shawshank Redemption" 
substr(x, 6, 14)
[1] "Shawshank"

grep("Shawshank", c("The","Shawshank","Redemption"), fixed=TRUE)
 [1] 2

sub("\\s",",","Hello There")
 [1] "Hello,There"

strsplit("Redemption", "")
 [1] "R" "e" "d" "e" "m" "p" "t" "i" "o" "n"

paste("Today is", date())
[1] "Today is Fri Jun 26 01:39:26 2015"

data <- mtcars
mean(data$mpg) 
[1] 20.09062

med <- median(data$mpg)
paste("Median MPG:", med)
[1] "Median MPG: 19.2"

hp <- sum(data$hp)
paste("Total HP:", hp)
[1] "Total HP: 4694"

max <- max(data$mpg)
min <- min(data$mpg)
paste("Maximum MPG:", max, "and Minimum MPG:", min)
[1] "Maximum MPG: 33.9 and Minimum MPG: 10.4"

sd <- sd(data$mpg)
paste("Std Deviation of MPG:", sd)
[1] "Std Deviation of MPG: 6.0269480520891"

We have covered the different operations that are available in R. Now, let's look at the control structures used in R. Control structures are the key elements of any programming language.

The control structures commonly used in R are as follows:

if(<condition>) { 
## do something 
} else { 
## do something else 
}

for(i in 1:10) { 
print(i) 
}

count<-0 
while(count<10) { 
print(count) 
count<-count+1 
}

> sum <- 1
> repeat
{
sum <- sum + 2;
print(sum);
if (sum > 11)
break;
}
3
5
7
9
11
13

for(i in 1:5) { 
if(i<=3) { 
next 
} 
print(i) 
}
[1] 4
[1] 5

We covered reading the data in R, understanding the data types, and performing various operations on the data. Now, we will see a few concepts that will be used just before an analysis or building a model. While performing an analysis, we might not need to study the entire dataset and we can just focus a subset of it, or, on the other hand, we might have to combine data from multiple data sources. These are the various concepts that will be covered in this chapter.

The most commonly used functionality will be to select the desired column from the dataset. While building the model, we will not be using all the columns in the dataset but just some of them that are more relevant. In order to select the column, we can either specify the column name or number, or simply delete the columns that are not required.

newdata <- data[c(1,5:10)]
head(newdata)
# excluding column 
newdata <- data[c(-2, -3, -4, -11)]
head(newdata) 
                   mpg drat    wt  qsec vs am gear
Mazda RX4         21.0 3.90 2.620 16.46  0  1    4
Mazda RX4 Wag     21.0 3.90 2.875 17.02  0  1    4
Datsun 710        22.8 3.85 2.320 18.61  1  1    4
Hornet 4 Drive    21.4 3.08 3.215 19.44  1  0    3
Hornet Sportabout 18.7 3.15 3.440 17.02  0  0    3
Valiant           18.1 2.76 3.460 20.22  1  0    3

In the preceding code, we first selected the column by its position. The first line of the code will select the first column and then the 5^th to 10^th column from the dataset, whereas, in the last line, the specified two columns are removed from the dataset. Both the preceding commands will yield the same result.

We can also arrive at a situation where we need to filter the data based on a condition. While building the model, we cannot create a single model for the whole of the population but we should create multiple models based on the behavior present in the population. This can be achieved by subsetting the dataset. In the following code, we will get the data of cars that have an mpg more than 25 alone:

newdata <- data[ which(data$mpg > 25), ]
                mpg cyl  disp  hp drat    wt  qsec vs am gear carb
Fiat 128       32.4   4  78.7  66 4.08 2.200 19.47  1  1    4    1
Honda Civic    30.4   4  75.7  52 4.93 1.615 18.52  1  1    4    2
Toyota Corolla 33.9   4  71.1  65 4.22 1.835 19.90  1  1    4    1
Fiat X1-9      27.3   4  79.0  66 4.08 1.935 18.90  1  1    4    1
Porsche 914-2  26.0   4 120.3  91 4.43 2.140 16.70  0  1    5    2
Lotus Europa   30.4   4  95.1 113 3.77 1.513 16.90  1  1    5    2

We might also need to consider a sample of the dataset. For example, while building a regression or logistic model, we need to have two datasets—one for the training and the other for the testing. In these cases, we need to choose a random sample. This can be done using the following code:

sample <- data[sample(1:nrow(data), 10, replace=FALSE),]
sample
                  mpg cyl  disp  hp drat    wt  qsec vs am gear carb
Honda Civic      30.4   4  75.7  52 4.93 1.615 18.52  1  1    4    2
Porsche 914-2    26.0   4 120.3  91 4.43 2.140 16.70  0  1    5    2
Merc 450SLC      15.2   8 275.8 180 3.07 3.780 18.00  0  0    3    3
Dodge Challenger 15.5   8 318.0 150 2.76 3.520 16.87  0  0    3    2
Duster 360       14.3   8 360.0 245 3.21 3.570 15.84  0  0    3    4
Fiat X1-9        27.3   4  79.0  66 4.08 1.935 18.90  1  1    4    1
Fiat 128         32.4   4  78.7  66 4.08 2.200 19.47  1  1    4    1
Lotus Europa     30.4   4  95.1 113 3.77 1.513 16.90  1  1    5    2
Toyota Corona    21.5   4 120.1  97 3.70 2.465 20.01  1  0    3    1
Mazda RX4 Wag    21.0   6 160.0 110 3.90 2.875 17.02  0  1    4    4

We considered a random sample of 10 rows from the dataset. Along with these, we might have to merge two different datasets. Let's see how this can be achieved. We can combine the data both row-wise as well as column-wise as follows:

sample1 <- data[sample(1:nrow(data), 10, replace=FALSE),]
sample2 <- data[sample(1:nrow(data), 5, replace=FALSE),]
newdata <- rbind(sample1, sample2)

The preceding code is used to combine two datasets that share the same column format. Then we can combine them using the rbind function. Alternatively, if the two datasets have the same length of data but different columns, then we can combine them using the cind or merge functions:

newdata1 <- data[c(1,5:7)]
newdata2 <- data[c(8:11)]
newdata <- cbind(newdata1, newdata2)

When we have two different datasets with a common column, then we can use the merge function to combine them. On using merge, the dataset will be merged based on the common columns.

These are the essential concepts necessary to prepare the dataset for the analysis, which will be discussed in the next few chapters.

In this chapter, you learned to import and read data from different sources such as CSV, TXT, XLSX, and relational data sources and the different data types available in R such as numeric, integer, character, and logical data types. We covered the basic data preprocessing techniques used to handle outliers, missing data, and inconsistencies in order to facilitate analysis.

You learned to perform different arithmetic operations that can be performed on the data using R, such as addition, subtraction, multiplication, division, exponentiation, and modulus, and also learned the string operations that can be performed on the data using R, such as subsetting a string, replacing a string, changing the case, and splitting the string into characters, which helps in data manipulation. Finally, you learned about the different control structures in R, such as if, else, for, while, repeat, break, next, and return, which facilitate a recursive or logical execution. We also covered bringing data to a usable format for analysis and building a model. In the next chapter, we will see how to perform exploratory data analysis using R. It will include a few statistical techniques and also variable analyses, such as univariate, bivariate, and multivariate analyses.