R Builtin Data Structures¶

Feng Li

School of Statistics and Mathematics

Central University of Finance and Economics

feng.li@cufe.edu.cn

https://feng.li/statcomp

>>> Link to Python version 1, 2, 3

Generate a sequence¶

Sequences¶

• Generate a sequence: seq()

• Repeat a vector: rep()

Vectors¶

• Numerical vectors

• Logical vectors

• Characters

• Length of a vector

• Vector calculations

Mathematical functions¶

• sqrt(), log()

• sin(),cos(), tan()

Vectors and matrices¶

Matrices¶

• Create a matrix: matrix()

• Dimension of a matrix: dim()

• How many elements in a matrix: length()

• Extract elements from a matrix.

• Replace elements with new entries.

• Create special matrices: diagonal matrix, identity matrix, zero matrix...

• Matrix multiplications: %*%

• Matrix inverse: solve()

• Transpose of a matrix: t()

• Element-wise operation with a matrix.

• Combine two or more matrices: rbind(), cbind()

Array¶

• An array is a high dimensional matrix.

• A matrix is a special case of an array when the dimension is two.

• A vector is a special array when their is no dimension (In R the dimension is usually dropped in this situation)

List¶

• Special data structure that matrix could not handle.

  • Data length are not the same.

  • Data type are not the same.

  • Nested data structure within a list.

• Create a list: list()

• Extract elements of a list: [[]] or $

• Delete an element within a list: set NULL to that element.

Data frame¶

• data.frame(): tightly coupled collections of variables which share many of the properties of matrices and of lists, used as the fundamental data structure by most of R's modeling software.

• In most cases, the operation with a data frame is similar to matrix operation.

• See also dplyr package.

  • written by Hadley Wickham of RStudio

  • everything dplyr does could already be done with base R, but it greatly simplifies existing functionality in R.

  • it provides a \"grammar\" (in particular, verbs) for data manipulation and for operating on data frames.

  • the dplyr functions are very fast, as many key operations are coded in C++.

Discussion¶

What type of data structure would you choose when you meet the following situations.

• Data are of the same length but different types.

• Data are not of the same length.

• Hierarchical structure of the data.

Managing data frames with the dplyr package¶

dplyr Grammar¶

Some of the key "verbs" provided by the dplyr package are

• select: return a subset of the columns of a data frame, using a flexible notation
• filter: extract a subset of rows from a data frame based on logical conditions
• arrange: reorder rows of a data frame
• rename: rename variables in a data frame
• mutate: add new variables/columns or transform existing variables
• summarise / summarize: generate summary statistics of different variables in the data frame, possibly within strata
• %>%: the "pipe" operator is used to connect multiple verb actions together into a pipeline

Common dplyr Function Properties¶

All of the functions have a few common characteristics. In particular,

• The first argument is a data frame.

• The subsequent arguments describe what to do with the data frame specified in the first argument, and you can refer to columns in the data frame directly without using the $ operator (just use the column names).

• The return result of a function is a new data frame.

• Data frames must be properly formatted and annotated for this to all be useful. In particular, the data must be tidy. In short, there should be one observation per row, and each column should represent a feature or characteristic of that observation.

• Installing the dplyr package

    install.packages("dplyr")

• After installing the package it is important that you load it into your R session with the library() function.

select()¶

• We will use a dataset containing air pollution and temperature data for the city of Chicago in the U.S.

• Sometimes you may want to use only a couple of variables out of many.

• Sometimes you may want to drop some variables that are not useful.

• If you wanted to keep every variable that ends with a "2", we could do

• Or if we wanted to keep every variable that starts with a "d", we could do

filter()¶

• The filter() function is used to extract subsets of rows from a data frame.

• We could for example extract the rows where PM2.5 is greater than 30 and temperature is greater than 80 degrees Fahrenheit.

arrange()¶

The arrange() function is used to reorder rows of a data frame according to one of the variables/columns.

• Here we can order the rows of the data frame by date, so that the first row is the earliest (oldest) observation and the last row is the latest (most recent) observation.

• Columns can be arranged in descending order too by useing the special desc() operator. Looking at the first three and last three rows shows the dates in descending order.

rename()¶

Renaming a variable in a data frame in R is surprisingly hard to do! The rename() function is designed to make this process easier.

• Here you can see the names of the first five variables in the chicago data frame. Now we rename the awkward variable names.

mutate()¶

The mutate() function exists to compute transformations of variables in a data frame.

• For example, with air pollution data, we often want to detrend the data by subtracting the mean from the data.

group_by()¶

The group_by() function is used to generate summary statistics from the data frame within strata defined by a variable. For example, in this air pollution dataset, you might want to know what the average annual level of PM2.5 is.

• First, we can create a year variable using as.POSIXlt().

• Now we can create a separate data frame that splits the original data frame by year.

In a slightly more complicated example, we might want to know what are the average levels of ozone (o3) and nitrogen dioxide (no2) within quintiles of pm25. A slicker way to do this would be through a regression model, but we can actually do this quickly with group_by() and summarize().

• First, we can create a categorical variable of pm25 divided into quintiles.

• Now we can group the data frame by the pm25.quint variable.

• Finally, we can compute the mean of o3 and no2 within quintiles of pm25.

Summary¶

The dplyr package provides a concise set of operations for managing data frames. With these functions we can do a number of complex operations in just a few lines of code. In particular, we can often conduct the beginnings of an exploratory analysis with the powerful combination of group_by() and summarize().

• dplyr can work with other data frame "backends" such as SQL databases. There is an SQL interface for relational databases via the DBI package

• dplyr can be integrated with the data.table package for large fast tables

• The dplyr package is handy way to both simplify and speed up your data frame management code. It's rare that you get such a combination at the same time!