Practice 4: Key Operators, Tidy Data, and Transforming Rows and Columns

Let’s work through what you saw in Week 4 together. Rather than treating this as a separate assignment, think of it as picking up where the lessons left off: we’ll walk through the same ideas step by step, and you’ll try each one out for yourself. This week the questions start to look like real data work: you’ll keep, sort, and reshape the rows and columns of a data frame.

Due Sunday, 10/25 at 11pm.

Lesson Topic Video Slides What you’ll practice here
18 Key operators and functions Video Slides Key operators and functions, Step C
19 Tidy data Video Slides Tidy data, Step E
20 Intro to data transformations and summarizations Video Slides Intro to dplyr
21 Data transformation: rows Video Slides Transforming rows, Step D, Step E
22 Data transformation: columns Video Slides Transforming columns, Step B, Step E
TipHow this works

There are two kinds of questions on this page:

  • Code boxes are little sandboxes you can type R code into. Click Run Code to run and check your answer. A green box means you’ve got it; an orange box means it’s not quite right yet and will nudge you toward the fix. Stuck? Click Show Hint (and Next Hint if there’s more than one). Want a clean slate? Click Start Over. Format tidies up your spacing.
  • Multiple choice questions give you feedback as soon as you pick an answer. For select all that apply, tick every option you think fits, then click Check my answer. Each one has a Hint box you can open.

Your answers are saved in this browser as you go, so you can come back later. A few tasks are things you do on your own computer (in RStudio), and those are marked ungraded self-check.

Turning it in: fill in the AI Use Statement, then follow the steps in Your turn: download your answers as a .qmd, render it in RStudio, and submit both the .qmd and the .html on Brightspace.

Key operators and functions

Lesson 18 expanded the operators you met in Lesson 4. To combine TRUE/FALSE statements, use Boolean operators. Wrap each statement in parentheses so it’s easy to read:

Operation In R Example In plain language
And & (1 == 1) & (3 > 2) Are both TRUE?
Or \| (1 == 1) \| (1 > 2) Is at least one TRUE?
Not ! !(1 > 2) Is it not TRUE?
In %in% "a" %in% c("a", "b") Is the value anywhere in this set?

%in% is case sensitive. You also met many handy functions. A few to remember:

Purpose Function Notes
rounding round(x, digits = n) rounds .5 to the nearest even number, so round(4.5, digits = 0) is 4
rounding janitor::round_half_up(x, digits = n) the “round .5 up” rule from school
round up / down ceiling(x) / floor(x) always up / always down
statistics mean(), median(), sd(), sum(), min(), max(), range(), summary() add na.rm = TRUE to ignore missing values
sequences seq(from, to, by), rep(x, ntimes) rep(c("a", "b"), 2) gives "a" "b" "a" "b"
random sample sample(x, size) random, so run set.seed() first to get the same draw each time

A note on TRUE vs. T: the Lesson 18 table writes na.rm = T as a shortcut. In this class, always write out na.rm = TRUE.

Several questions on this page use a tiny, made-up sample called mini_hrs (ten “respondents” with HRS-style variables). It’s already built for you in each code box, so you don’t need to create it:

id age_yr age_mo sex degree srh cesd diab income
R01 55 665 Female Bachelor’s degree Good 2 No 35000
R02 62 748 Male High school diploma Fair 4 Yes 0
R03 71 856 Male Master’s degree Very Good 6 No 108400
R04 48 580 Female NA Excellent 0 No 52000
R05 67 806 Male Master’s degree Poor NA Yes 28000
R06 59 713 Female GED Good 3 No 61500
R07 100 1203 Female None Fair 5 Yes 15000
R08 74 893 Male High school diploma Good 1 No 80250
R09 52 628 Female Bachelor’s degree Very Good 2 No 43000
R10 66 795 Male NA Poor 7 Yes 22000

(NA means a missing value.)

  1. What does (5 > 3) & (2 > 4) give?

See the table of And / Or / Not on the “We can further expand the relational operators” slide in Lesson 18. Work out each side first, then combine them.

  1. Is 7 greater than 5 AND 7 greater than 10? Fill in the blank with the operator that means AND, so R answers the question.
NoteHint 1

The table of Boolean operators in Lesson 18 has one for AND. It is a single character.

NoteHint 2

(7 > 5) & (7 > 10)

  1. Is 4 one of the numbers in c(2, 4, 6)? Fill in the operator that checks whether something is in a set of values.
NoteHint 1

The operator starts and ends with a percent sign and has a short word in the middle. It’s on the “Is something in something else?” slide in Lesson 18.

NoteHint 2

4 %in% c(2, 4, 6)

  1. What does "A" %in% c("a", "b", "c", "d") give?

Look at the last example on the “Is something in something else?” slide in Lesson 18.

  1. Which function always rounds 3.1234 up to the next whole number (4)?

Look at the “Some good functions to know” table in Lesson 18.

  1. What does round(4.5, digits = 0) give in R?

Look at the two rounding rows in the “Some good functions to know” table in Lesson 18. Then run round(4.5, digits = 0) and round(3.5, digits = 0) in your Console.

  1. The vector ages has one missing value (NA). Fill in the blank so median() ignores it and finds the median of the four ages we do have.
NoteHint 1

Statistical functions like mean() and median() return NA if even one value is missing. The “Useful statistical functions” table in Lesson 18 shows the argument that removes them.

NoteHint 2

median(ages, na.rm = TRUE). In this class we write out TRUE (not T).

  1. Use one function to get the smallest and largest age_yr in mini_hrs at the same time. Fill in the function name.
NoteHint 1

Look at the “Useful statistical functions” table in Lesson 18. One function returns both ends at once.

NoteHint 2

range(mini_hrs$age_yr)

  1. rep(x, ntimes) repeats x, ntimes times. Fill in the blank so R repeats the letters a, b, c two times: "a" "b" "c" "a" "b" "c".
NoteHint 1

The second input says how many times to repeat the whole set.

NoteHint 2

rep(c("a", "b", "c"), 2)

  1. We run set.seed(523) right before sample(). Why?

Read the blue note on the last slide of the functions table in Lesson 18 (“Extra note: we will set a seed…”).

  1. Run the code below as it is and read the error. Then fix it so R draws 5 random numbers from 1 to 15. Because of set.seed(523), you should get 11 1 8 9 13.
NoteHint 1

The error says unexpected symbol. Look at where R stopped: what is missing between 1:15 and size? (This is the “unexpected symbol” error from Lesson 9.)

NoteHint 2

Arguments are separated by commas: sample(1:15, size = 5).

Tidy data

Lesson 19: tidy data follow three rules. Tidy data are easiest for R to transform, summarize, and plot.

  1. Each variable has its own column
  2. Each observation has its own row
  3. Each value has its own cell

Data in “human readable” form (like many Excel sheets) are often not tidy, and R can’t read them easily. Our HRS data are tidy: each row is one person, and each column is one piece of information about them.

  1. Which of these is not one of the three rules of tidy data?

See the three numbered rules on the “Tidy data” slide in Lesson 19.

  1. Here are the same data in two forms. Which table is tidy?

    Table 1

    id age_yr cesd
    R1 55 2
    R2 62 4

    Table 2

    id variable value
    R1 age_yr 55
    R1 cesd 2
    R2 age_yr 62
    R2 cesd 4

Check the three rules on the “Tidy data” slide in Lesson 19: does each variable have its own column, and does each observation (a person) have its own row?

  1. Why isn’t Table 2 (in the question above) tidy?

Which rule breaks when one person (one observation) takes up two rows and the variable names are inside a column?

  1. Is hrs_data tidy? What does one row of it represent?

Look at the “Our HRS data are in tidy form” slide in Lesson 19.

Intro to dplyr

Lesson 20: dplyr is a tidyverse package of data-transformation functions. Each function is a verb that describes an action on your data. Most take a data frame as the first argument (the pipe |> fills it in for you) and give a data frame back. The other arguments say which columns to use. For example:

mini_hrs |>
  select(age_yr, srh)

dplyr functions come in four groups, based on what they operate on: rows (filter()), columns (select(), mutate()), groups (group_by(), summarize()), and joins (not covered in this class).

  1. Most dplyr functions take one kind of object in and give one kind of object out. Which?

See the “dplyr basics” slide in Lesson 20: Input and Output.

  1. Use select() to keep only the age_yr and srh columns of mini_hrs. Fill in the blank with the two column names, separated by a comma. (Column names don’t need quotes.)
NoteHint 1

Type the names of the columns you want to keep, with a comma between them. No quotes needed.

NoteHint 2

select(age_yr, srh)

  1. Lesson 20 sorts dplyr functions into four groups. Which group will we use later to get summaries for each category (like the average cesd for each sex)?

Look at the “Types of dplyr functions” slide in Lesson 20.

Transforming rows

Lesson 21 covered three functions that transform rows. We use columns to decide which rows to keep, drop, or reorder.

Function What it does Example
filter() keeps rows that meet a condition filter(age_yr > 71)
arrange() sorts rows (add desc() for high to low) arrange(desc(cesd))
distinct() keeps unique rows distinct(srh)

A few things to remember about filter():

  • Comparison operators: >, <, >=, <=, == (equal), != (not equal). Use == (two equal signs!) to compare, never a single =.
  • For categories, put the value in quotes and spell it exactly (apostrophes and capitals too). A misspelled value does not give an error. You just get 0 rows, so always check your output!
  • NA is not equal to anything, so != also drops the NA rows. filter_out() is the newer companion that keeps them. drop_na() removes rows with missing values: in any column if you give it no arguments, or just in the columns you name.

arrange() sorts by the first column you list, then breaks ties with the next one. distinct() with column names shows the unique values; add .keep_all = TRUE to keep the other columns too.

  1. Which of these functions transform rows? Select all that apply.

Look at the “Data transformations on rows” slide in Lesson 21. The other two operate on columns.

  1. Use filter() to keep the respondents who are older than 71. Fill in the comparison operator.
NoteHint 1

“Older than” means strictly greater than, so 71-year-olds are not included. See the operator list on the filter() slide in Lesson 21.

NoteHint 2

filter(age_yr > 71)

  1. Use filter() to keep only the respondents whose degree is "Master's degree". Fill in the operator.
NoteHint 1

For categories, we check for an exact match. The operator is two characters.

NoteHint 2

filter(degree == "Master's degree")

  1. Run the code below. R doesn’t give an error, but look at the result: 0 rows. Fix the code so it keeps the two respondents with a Master’s degree.
NoteHint 1

R won’t warn you when filter() matches nothing, so you have to check your output! Compare your value to the one on the filter() slides in Lesson 21, letter by letter.

NoteHint 2

The value has an apostrophe: "Master's degree".

  1. In mini_hrs, 2 respondents have a Master’s degree and 2 have a missing (NA) degree. How many rows does filter(degree != "Master's degree") keep out of the 10?

Look at the “WAIT! Did those two operations have different rows?” slide in Lesson 21.

  1. How is filter_out(degree == "Master's degree") different from filter(degree != "Master's degree")?

Look at the filter_out() slide in Lesson 21. Think about what happens to NA rows.

  1. Use drop_na() to remove only the rows where degree is missing. Fill in the column name. (Rows with missing values in other columns, like cesd, should stay.)
NoteHint 1

With no column name, drop_na() removes rows with a missing value in any column. Put the column you care about inside the parentheses.

NoteHint 2

drop_na(degree)

  1. Sort mini_hrs from the highest cesd score to the lowest. Fill in the blank with the function that makes the order descending.
NoteHint 1

By default, arrange() sorts from smallest to largest. You wrap the column in a helper function to flip it. It’s on the “Example: sort data by descending…” slide in Lesson 21.

NoteHint 2

arrange(desc(cesd))

  1. Sort mini_hrs by sex first, and then, within each sex, by age_yr (youngest first). Fill in both blanks.
NoteHint 1

arrange() sorts by the first column you list, then uses the next column to break ties. The order matters!

NoteHint 2

arrange(sex, age_yr)

  1. Use distinct() to see the unique values of self-rated health, srh, in mini_hrs. Fill in the column name.
NoteHint 1

Name the column whose unique values you want to see.

NoteHint 2

distinct(srh)

  1. What does adding .keep_all = TRUE do in distinct(degree, .keep_all = TRUE)?

Look at the two distinct() slides with degree in Lesson 21. Compare which columns are in each output.

Transforming columns

Lesson 22 covered four functions that transform columns.

Function What it does Example
select() keeps (or drops) columns select(id, age_mo), select(-diab), select(age_mo:degree)
relocate() moves columns (front by default) relocate(income, .after = id)
mutate() creates or changes columns mutate(age_yrs = age_mo / 12)
rename() renames columns, as new_name = old_name rename(self_rated_health = srh)
  • filter() subsets rows; select() subsets columns.
  • Inside mutate() use a single = to name the new column (not == or <-).
  • For most of your work, save your output with <-. The lesson names saved data frames with a number for the order and a description of the action, like hrs_01_vars_subset.
  1. You only need the income and cesd variables for your analysis, and you want to drop all the other columns. Which function do you use?

Look at the “Distinguishing filter() vs select()” slide in Lesson 22.

  1. Keep every column except diab and srh. Fill in the two column names.
NoteHint 1

A minus sign in front of a column name means “drop this one”. The minus signs are already there, so just add the names.

NoteHint 2

select(-diab, -srh)

  1. The columns of mini_hrs are in this order: id, age_yr, age_mo, sex, degree, srh, cesd, diab, income. Use : to keep the block of columns from age_mo to degree. Fill in the last column name.
NoteHint 1

first_column:last_column keeps those two columns and every column between them. Where should the block end?

NoteHint 2

select(age_mo:degree)

  1. Move income so it sits right after the id column. Fill in the column it should come after.
NoteHint 1

.after = column_name places the moved column directly after that column.

NoteHint 2

relocate(income, .after = id)

  1. age_mo is age in months. Create a new column called age_yrs by dividing age_mo by 12. Fill in the operator.
NoteHint 1

Which math operator means divide? (Lesson 18 and Lesson 4 have the table.)

NoteHint 2

mutate(age_yrs = age_mo / 12)

  1. Inside one mutate(), you can use a column you just created. Fill in the blank so age_decades is age_yrs divided by 10.
NoteHint 1

Which column holds age in years? You created it on the line right above.

NoteHint 2

age_decades = age_yrs / 10

  1. Run the code below as it is and read the error. Then fix it so it creates the new column age_yrs.
NoteHint 1

The error says object 'age_yrs' not found. R thought you were comparing age_yrs to something, so it looked for an existing column with that name. What operator names a new column?

NoteHint 2

Inside mutate(), use a single =: mutate(age_yrs = age_mo / 12).

  1. Rename the srh column to self_rated_health. Fill in both blanks. Remember the order: new_name = old_name.
NoteHint 1

The new name goes on the left of the = and the old name on the right.

NoteHint 2

rename(self_rated_health = srh)

  1. You start from hrs_00, subset the columns, and save the result. Which name best follows the naming convention from Lesson 22?

See the orange “Naming conventions” box near the start of Lesson 22.

  1. Put it together! From mini_hrs, keep only respondents whose diab is "Yes", then keep only the id and age_yr columns. Fill in the operator and the column name.
NoteHint 1

Read the pipe as “then”: take mini_hrs, then filter the rows, then select the columns. The first blank is a comparison operator for a category.

NoteHint 2

filter(diab == "Yes") |> select(id, age_yr)

AI Use Statement

NoteAI Use Statement

Using AI is not required in this class. If you did use an AI tool (ChatGPT, Claude, Copilot, Gemini, an RStudio assistant, etc.) on this practice, add a brief AI Use Statement. Include:

  • Which tool(s) you used
  • What for (a sentence or two, with question numbers)
  • Where it contributed to any written answers (if at all)
  • You do NOT need to link your conversation.

For example:

AI Use Statement: Claude was used to explain an error message in Question 36. All reasoning and answers are my own.

OR

AI Use Statement: AI was not used in this assignment.

Before using AI, check the table of allowed uses in the Generative AI Policy. For practice, AI can help you understand (explain an error or a concept, explain what a function does, check an answer you already worked out), but it should not do the problem for you. A half-finished answer that shows your thinking is always better than a polished one from AI!

Fill in your statement below. It’s saved as you type, and it goes at the top of your downloaded .qmd, right where your submission needs it.

Your turn: working with real data

Now let’s put this week together in RStudio, with the real HRS data. These steps are ungraded self-check, but your downloaded .qmd (and the .html you render from it) is what you turn in. Do them in order, and open your R Project (double-click PUBH_523_26F.Rproj) before you start.

Step A: Download your answers, open them, and load packages

Click Download my answers as .qmd (the button in the sidebar, or at the bottom of the last section, Step F). Move the file into PUBH_523_26F/practice/, and rename it lastname_firstinitial_practice_04.qmd. In RStudio (with your project open), open it from the Files pane. Change author: in the YAML to your name.

Then find the Load packages chunk near the top of the file and type this inside it:

pacman::p_load(tidyverse, here, rio, janitor)

tidyverse is new in your Load packages chunk. It loads dplyr and tidyr (the packages for this week’s functions) plus several others, so the first time may take a minute or two to install. Run the chunk (Ctrl + Enter / Cmd + Enter on the line) to be sure it works.

Next, scroll to the Import the HRS data section. It already has the import code from Practice 3 filled in:

hrs_data <- import(here("data_practice", "hrs_data.rds")) |>
  clean_names()

Run it. You should see hrs_data in your Environment pane with 2728 observations of 32 variables. (If R can’t find the file, you may not have saved it in Practice 3. Download hrs_data.rds and save it in PUBH_523_26F/data_practice/.) Don’t render yet!

Step B: Your first render fails on purpose

(Lessons 9 and 22) In the Import the HRS data section, find the chunk labeled Step B. It has this code:

hrs_check <- hrs_data |>
  mutate(age_yrs == age_mo / 12)

Now click Render. It will stop with an error that looks like this:

Error in `mutate()`:
ℹ In argument: `age_yrs == age_mo/12`.
Caused by error:
! object 'age_yrs' not found

This is the same mistake as Question 36. Two equal signs compare things, so R went looking for an existing column called age_yrs. To fix it, inside that chunk, change == to a single = (it names the new column). Render again. It should work this time.

Step C: Operators and functions on the HRS data

(Lesson 18) Scroll to the Key operators and functions (Lesson 18) section of your file and find the chunk labeled Step C. Inside the chunk, replace each ______ with the missing value. Keep the pieces that are already written:

hrs_data$cesd |>
  mean(na.rm = ______) |>
  round(digits = 2)

set.seed(523)
hrs_data$id |>
  sample(size = ______)

If it works, the first line gives 1.79 (the mean cesd), and the sample is the five ids "559315_010" "555653_010" "559806_020" "553623_010" "553793_010".

Step D: Filter and find unique rows

(Lesson 21) In the Transforming rows (Lesson 21) section of your file, find the chunk labeled Step D. Inside the chunk, replace each ______:

hrs_01_masters <- hrs_data |>
  filter(degree ______ "Master's degree")
nrow(hrs_01_masters)

hrs_02_degrees <- hrs_data |>
  distinct(______)
nrow(hrs_02_degrees)

nrow() is a new little helper: it counts the rows of a data frame (you can also read the size at the top of a printed tibble). If it works, the first nrow() is 487 and the second is 8 (seven degree levels plus NA). Below the chunk (as regular text), write one sentence: if you typed "Masters degree" (no apostrophe) in the filter(), what would you see, and how would you catch the mistake?

Step E: Put rows and columns together in one pipe

(Lessons 19, 21, and 22) In the Transforming columns (Lesson 22) section of your file, find the chunk labeled Step E. Inside the chunk, replace each ______:

hrs_03_poor_health <- hrs_data |>
  filter(srh ______ "Poor") |>
  select(id, age_mo, cesd) |>
  mutate(age_yrs = age_mo ______ 12) |>
  rename(depression_score = ______)

nrow(hrs_03_poor_health)
names(hrs_03_poor_health)

If it works, nrow() gives 187 and names() gives "id" "age_mo" "depression_score" "age_yrs". Below the chunk (as regular text), write 1–2 sentences: read your pipe out loud using the word “then”, and say what one row of hrs_03_poor_health represents (is it still tidy?).

Step F: Render and submit

Render one last time. Check that your AI Use Statement shows at the top of the .html. Then submit both your .qmd and .html on Brightspace.

Didn’t finish every question? That’s okay! Practice is graded on completion: you get credit for answering at least 75% of the question parts, whether your answers are right or wrong. Anything you haven’t finished shows up as Not answered yet or Not finished yet in your download, and won’t stop your file from rendering.

Check your overall progress on this practice set on the My Progress page.