Problem set: data wrangling II

Overview

Inside your camp project, write a script called code/03_set.R that answers the questions below. It is your script: for any line in it, you should be able to say what it does. New today: for every case_when(), say which values matched no condition and what happened to them; for every join, say how many rows there were before and after, and what didn’t match.

For all of your questions, do the following:

  • Make the graphs look good (add labels, a theme)
  • Make your code look good (comment it, write it in chunks)
  • Save each of your plots in figures/ using ggsave() and here()
  • Save any cleaned data in data/clean/. Never overwrite anything in data/
  • When you think you’re done: restart R and run the script top to bottom, from a fresh session. If it works then, it works

Nothing is turned in. Your figures/ and data/clean/ folders are the record of what you did.

Questions

You’ll need the following packages:

library(tidyverse)
library(juanr)
library(here)
  1. Collapse a category. In bot, the religion column has 13 values. Run count(bot, religion) to see them. Make a new column, religion3, with three values: “Christian” (Protestant, Roman Catholic, Mormon, Eastern or Greek Orthodox), “None” (Atheist, Agnostic, Nothing in particular), and “Other religion” for everything else. Hint: several values can share one condition if you combine them with |, as on Day 2.

    1. Run count(religion, religion3). Did every value land where you wanted? What happened to the rows where religion was missing?

    2. Make a bar chart of religion3.

    3. Ask your agent to collapse educ into three levels of its choosing. Read the case_when() it wrote: where did it put “2-year” and “Some college”? Do you agree with its choices?

  2. Merge, then check the merge. euro_elections has one row per party per election. parties has one row per party, with its ideology (left_right, where higher means further right) and its family.

    1. Which column identifies a party in both datasets? A good identifier appears once per row in parties. Check: run parties |> count(party_id, sort = TRUE). Is the largest count 1?

    2. Make a small version of parties that keeps only party_id, family, and left_right. Use left_join() to add those columns to euro_elections. Count rows before and after. Did the count change?

    3. Use anti_join() to find the election results that have no matching party. How many are there? Use count() to see which countries they mostly come from.

    4. Make a scatterplot of parties’ left-right position (x-axis) against their vote share (y-axis).

    5. Use str_detect() inside mutate() to make a TRUE/FALSE column for whether the party’s name contains “Green”. Color the points by that column. Where do the Green parties sit?

    6. What happens if you join by country instead of party_id? Count the rows before you plot anything. In a comment, explain what went wrong.

  3. Reshape, with your agent. therm has one row per survey respondent. Sixteen columns starting with ft_ are “feeling thermometers”: how warmly the respondent feels toward a group, from 0 to 100.

    1. Before you write any code: if this data were long, with one row per respondent-group, how many rows would it have? Write the number in a comment.

    2. Ask your agent to make the data long, with one column called group (the old column names) and one called feeling (the values). Save the result under a new name. Check nrow() against your prediction.

    3. The group names still say “ft_black”, “ft_police”, and so on. Use str_remove() to drop the “ft_”. Check with count(group): are there 16 groups, and does each have the same number of rows?

    4. Use filter() to keep three groups you find interesting. Make a histogram of feeling for each group (use facet_wrap() or the fill aesthetic). What do you see?

    5. Save the long data in data/clean/ as a .csv, using here() for the path.

  4. Groups from numbers. In art, each row is an artwork sold at auction. Make a new column, age_of_art, that is the number of years between when the artwork was made (year_of_artwork) and when it sold (event_year). Then make a categorical column, older_than_10, with two values: “Older than 10 years” and “10 years or newer”.

    1. Run count() on older_than_10. Some rows are NA. Use filter() to look at a few of them: what is missing? Then use drop_na() and say how many rows it removed.

    2. Make a histogram of the real price (real_price_usd), with the two age groups separated by fill. Prices are extremely skewed, so add scale_x_log10() to your plot: it spreads the x-axis out so that 1,000 and 1,000,000 are the same distance apart as 1,000,000 and 1,000,000,000.

    3. Is newer art more or less expensive than older art?