library(tidyverse)
library(juanr)
library(here)Problem set: data wrangling II
Overview
Inside your camp project, write a script called code/03_set.R that answers the questions below. It is your script: for any line in it, you should be able to say what it does. New today: for every case_when(), say which values matched no condition and what happened to them; for every join, say how many rows there were before and after, and what didn’t match.
For all of your questions, do the following:
- Make the graphs look good (add labels, a theme)
- Make your code look good (comment it, write it in chunks)
- Save each of your plots in
figures/usingggsave()andhere() - Save any cleaned data in
data/clean/. Never overwrite anything indata/ - When you think you’re done: restart R and run the script top to bottom, from a fresh session. If it works then, it works
Nothing is turned in. Your figures/ and data/clean/ folders are the record of what you did.
Questions
You’ll need the following packages:
Collapse a category. In
bot, thereligioncolumn has 13 values. Runcount(bot, religion)to see them. Make a new column,religion3, with three values: “Christian” (Protestant, Roman Catholic, Mormon, Eastern or Greek Orthodox), “None” (Atheist, Agnostic, Nothing in particular), and “Other religion” for everything else. Hint: several values can share one condition if you combine them with|, as on Day 2.Run
count(religion, religion3). Did every value land where you wanted? What happened to the rows wherereligionwas missing?Make a bar chart of
religion3.Ask your agent to collapse
educinto three levels of its choosing. Read thecase_when()it wrote: where did it put “2-year” and “Some college”? Do you agree with its choices?
Merge, then check the merge.
euro_electionshas one row per party per election.partieshas one row per party, with its ideology (left_right, where higher means further right) and itsfamily.Which column identifies a party in both datasets? A good identifier appears once per row in
parties. Check: runparties |> count(party_id, sort = TRUE). Is the largest count 1?Make a small version of
partiesthat keeps onlyparty_id,family, andleft_right. Useleft_join()to add those columns toeuro_elections. Count rows before and after. Did the count change?Use
anti_join()to find the election results that have no matching party. How many are there? Usecount()to see which countries they mostly come from.Make a scatterplot of parties’ left-right position (x-axis) against their vote share (y-axis).
Use
str_detect()insidemutate()to make aTRUE/FALSEcolumn for whether the party’s name contains “Green”. Color the points by that column. Where do the Green parties sit?What happens if you join by
countryinstead ofparty_id? Count the rows before you plot anything. In a comment, explain what went wrong.
Reshape, with your agent.
thermhas one row per survey respondent. Sixteen columns starting withft_are “feeling thermometers”: how warmly the respondent feels toward a group, from 0 to 100.Before you write any code: if this data were long, with one row per respondent-group, how many rows would it have? Write the number in a comment.
Ask your agent to make the data long, with one column called
group(the old column names) and one calledfeeling(the values). Save the result under a new name. Checknrow()against your prediction.The group names still say “ft_black”, “ft_police”, and so on. Use
str_remove()to drop the “ft_”. Check withcount(group): are there 16 groups, and does each have the same number of rows?Use
filter()to keep three groups you find interesting. Make a histogram offeelingfor each group (usefacet_wrap()or thefillaesthetic). What do you see?Save the long data in
data/clean/as a.csv, usinghere()for the path.
Groups from numbers. In
art, each row is an artwork sold at auction. Make a new column,age_of_art, that is the number of years between when the artwork was made (year_of_artwork) and when it sold (event_year). Then make a categorical column,older_than_10, with two values: “Older than 10 years” and “10 years or newer”.Run
count()onolder_than_10. Some rows areNA. Usefilter()to look at a few of them: what is missing? Then usedrop_na()and say how many rows it removed.Make a histogram of the real price (
real_price_usd), with the two age groups separated byfill. Prices are extremely skewed, so addscale_x_log10()to your plot: it spreads the x-axis out so that 1,000 and 1,000,000 are the same distance apart as 1,000,000 and 1,000,000,000.Is newer art more or less expensive than older art?