Tidying Data for Comprehensive Analysis

Welcome back! Now that you've mastered reshaping data, separating and uniting columns, and handling missing values using tidyr, it's time to bring it all together for comprehensive data analysis. You'll get to practice all the concepts you've learned and prepare your data for in-depth analysis.

In this lesson, we'll focus on applying everything you've learned in the previous units, elevating your data tidying skills to the next level!

For example, we'll take a semi-tidy data frame and apply a series of transformations to make it fully tidy. Specifically, you will:

1. Separate Concatenated Columns: Break down columns containing multiple values into separate columns. This step ensures that each piece of information has its own column.
2. Convert Data Types: Make sure each column has the correct data type, such as converting text to numerical values where needed.
3. Unite Columns: Combine multiple columns into a single column where it makes sense, often creating more readable and informative labels.

Let's look at this advanced example:

# Suppress package startup messages for a cleaner output
suppressPackageStartupMessages(library(tidyr))
suppressPackageStartupMessages(library(dplyr))

# A semi-tidy tibble
semi_tidy_df <- tibble(
  Name = c("John", "Jane", "Alex", "Emily", "David"),
  Age_Height = c("28.180", "22.165", "35.175", "29.160", "40.170"),
  Weight = c(75, 60, 82, 55, 68),
  Address = c("123 Main St, Springfield", "456 Elm St, Springfield", 
              "789 Oak St, Metropolis", "321 Maple St, Gotham", 
              "654 Pine St, Star City")
)

# Transforming the semi-tidy data
tidy_df <- semi_tidy_df %>%
  separate(Age_Height, into = c("Age", "Height"), sep = "[.]") %>%
  mutate(
    Age = as.numeric(Age),
    Height = as.numeric(Height)
  ) %>%
  separate(Address, into = c("Street", "City"), sep = ", ") %>%
  unite(Name_Age, Name, Age, sep = ": ")

print("Tidied DataFrame:")
print(tidy_df)

With these combined transformations, data becomes more organized and easier to analyze.

Note: In the above example, square brackets [.] are used around the period in sep (for the first separate function) to indicate that the period is a literal character and not a regular expression special character. An alternative way to achieve the same result is to use double backslashes \\. to escape the period. If neither of these methods were used, the period would match any character, leading to incorrect splitting.