Meetup 9: Hierarchical Data and Webscraping

George Hagstrom

Lab 6: Color Scheme

Color Scheme for map:

x = rnorm(n=100)
y = rnorm(n=100)
z = rcauchy(n=100,location=0,scale=1)
data = tibble(x = x, y=y,z=z)

data |> ggplot(aes(x=x,y=y,color=z)) + geom_point(size=10) +
  scale_color_gradient2(high="blue",low="red",mid = "white") +
  theme_minimal()

Lab 6: : Color Scheme

Color Scheme for map:

Lab 6: Color Scheme

Instead use perceptually uniform color scheme or pick different middle color:
- scale_color_viridis
- scale_color_gradient2(high="blue",low="red",mid = "white",midpoint=0)

Lab 6 Review: Principle of Proportional Ink

On me since I said to consider using size…..
scale_size_area, only use for positive variables

Lab 6 Review

join disordering
In SQL joins can change the order of columns

  left_join(colleges_clean, by = "unitid") |>  
  slice_max(order_by = avg_profit, n = 10) 

  slice_max(order_by = avg_profit, n = 10) 
  left_join(colleges_clean, by = "unitid") |>

Lab 6 Review: Keys

# A tibble: 132,327 × 28
    year unitid institution_name  city_txt state_cd zip_text classification_code
   <dbl>  <dbl> <chr>             <chr>    <chr>    <chr>                  <dbl>
 1  2015 100654 Alabama A & M Un… Normal   AL       35762                      2
 2  2015 100654 Alabama A & M Un… Normal   AL       35762                      2
 3  2015 100654 Alabama A & M Un… Normal   AL       35762                      2
 4  2015 100654 Alabama A & M Un… Normal   AL       35762                      2
 5  2015 100654 Alabama A & M Un… Normal   AL       35762                      2
 6  2015 100654 Alabama A & M Un… Normal   AL       35762                      2
 7  2015 100654 Alabama A & M Un… Normal   AL       35762                      2
 8  2015 100654 Alabama A & M Un… Normal   AL       35762                      2
 9  2015 100654 Alabama A & M Un… Normal   AL       35762                      2
10  2015 100654 Alabama A & M Un… Normal   AL       35762                      2
# ℹ 132,317 more rows
# ℹ 21 more variables: classification_name <chr>, classification_other <chr>,
#   ef_male_count <dbl>, ef_female_count <dbl>, ef_total_count <dbl>,
#   sector_cd <dbl>, sector_name <chr>, sportscode <dbl>, partic_men <dbl>,
#   partic_women <dbl>, partic_coed_men <dbl>, partic_coed_women <dbl>,
#   sum_partic_men <dbl>, sum_partic_women <dbl>, rev_men <dbl>,
#   rev_women <dbl>, total_rev_menwomen <dbl>, exp_men <dbl>, …

Lab 6: Primary Keys

Most common mistake on Lab 6 was not identifying keys correctly in the college sports dataset
Key ideas:
- Remove duplicate rows from new tables
- Key should uniquely identify each row
- Is year needed?
- unitid versus institution_name?

Week Ahead: Webscraping and Hierarchical Data

Read: Chapter 23 and 24
Lab 7
Lengthy Vignette

Hierarchical Data

Data stored in a tree-strucutre
Parent nodes are connected to children

cran data.tree

Hierarchical Data

One of the most common ways to store data
Easy to understand, can be fast
Has some drawbacks (read Codd 1970)

cran data.tree

List data type

List is like a vector, but data types don’t have to be the same

list1 <- list(vector = 1:4, letter = "a", logical = TRUE)

str(list1)

List of 3
 $ vector : int [1:4] 1 2 3 4
 $ letter : chr "a"
 $ logical: logi TRUE

list1[[2]]

[1] "a"

list1["vector"]

$vector
[1] 1 2 3 4

Elements can be named

List can contain lists

This allows lists to be hierarchical

country_list = list( USA = list(capital = "Washington", pop = 330, continent = "NA"),
                     Brazil = list(capital = "Brasilia", pop = 212, continent = "SA"),
                     Poland = list(capital = "Warsaw",pop = 38, continent = "EU"))

str(country_list)

List of 3
 $ USA   :List of 3
  ..$ capital  : chr "Washington"
  ..$ pop      : num 330
  ..$ continent: chr "NA"
 $ Brazil:List of 3
  ..$ capital  : chr "Brasilia"
  ..$ pop      : num 212
  ..$ continent: chr "SA"
 $ Poland:List of 3
  ..$ capital  : chr "Warsaw"
  ..$ pop      : num 38
  ..$ continent: chr "EU"

Use str or View to visualize hierarchical lists

Rectangling

To analyze hierarchical data, need to get it into a table
Use unnest functions
Two common scenarios
1. Named children w/ common structure
- unnest_wider
1. Unnamed children
- unnest_longer

`unnest_wider`

When list children have names and common structure, each list member can become a new columns:

  df1 <- tribble(
  ~x, ~y,
  1, list(a = 11, b = 12),
  2, list(a = 21, b = 22),
  3, list(a = 31, b = 32),
)
  
df1 |> unnest_wider(y)

# A tibble: 3 × 3
      x     a     b
  <dbl> <dbl> <dbl>
1     1    11    12
2     2    21    22
3     3    31    32

`unnest_longer`

When there list elements aren’t named, each list member gets put into a new row

df2 <- tribble(
  ~x, ~y,
  1, list(11, 12, 13),
  2, list(21),
  3, list(31, 32),
)

df2 |> unnest_longer(y)

# A tibble: 6 × 2
      x     y
  <dbl> <dbl>
1     1    11
2     1    12
3     1    13
4     2    21
5     3    31
6     3    32

GOT Case Study

repurrrsive package contains hierarchical datasets for practice
got_chars contains info on Game of Thrones characters

str(got_chars)

List of 30
 $ :List of 18
  ..$ url        : chr "https://www.anapioficeandfire.com/api/characters/1022"
  ..$ id         : int 1022
  ..$ name       : chr "Theon Greyjoy"
  ..$ gender     : chr "Male"
  ..$ culture    : chr "Ironborn"
  ..$ born       : chr "In 278 AC or 279 AC, at Pyke"
  ..$ died       : chr ""
  ..$ alive      : logi TRUE
  ..$ titles     : chr [1:2] "Prince of Winterfell" "Lord of the Iron Islands (by law of the green lands)"
  ..$ aliases    : chr [1:4] "Prince of Fools" "Theon Turncloak" "Reek" "Theon Kinslayer"
  ..$ father     : chr ""
  ..$ mother     : chr ""
  ..$ spouse     : chr ""
  ..$ allegiances: chr "House Greyjoy of Pyke"
  ..$ books      : chr [1:3] "A Game of Thrones" "A Storm of Swords" "A Feast for Crows"
  ..$ povBooks   : chr [1:2] "A Clash of Kings" "A Dance with Dragons"
  ..$ tvSeries   : chr [1:6] "Season 1" "Season 2" "Season 3" "Season 4" ...
  ..$ playedBy   : chr "Alfie Allen"
 $ :List of 18
  ..$ url        : chr "https://www.anapioficeandfire.com/api/characters/1052"
  ..$ id         : int 1052
  ..$ name       : chr "Tyrion Lannister"
  ..$ gender     : chr "Male"
  ..$ culture    : chr ""
  ..$ born       : chr "In 273 AC, at Casterly Rock"
  ..$ died       : chr ""
  ..$ alive      : logi TRUE
  ..$ titles     : chr [1:2] "Acting Hand of the King (former)" "Master of Coin (former)"
  ..$ aliases    : chr [1:11] "The Imp" "Halfman" "The boyman" "Giant of Lannister" ...
  ..$ father     : chr ""
  ..$ mother     : chr ""
  ..$ spouse     : chr "https://www.anapioficeandfire.com/api/characters/2044"
  ..$ allegiances: chr "House Lannister of Casterly Rock"
  ..$ books      : chr [1:2] "A Feast for Crows" "The World of Ice and Fire"
  ..$ povBooks   : chr [1:4] "A Game of Thrones" "A Clash of Kings" "A Storm of Swords" "A Dance with Dragons"
  ..$ tvSeries   : chr [1:6] "Season 1" "Season 2" "Season 3" "Season 4" ...
  ..$ playedBy   : chr "Peter Dinklage"
 $ :List of 18
  ..$ url        : chr "https://www.anapioficeandfire.com/api/characters/1074"
  ..$ id         : int 1074
  ..$ name       : chr "Victarion Greyjoy"
  ..$ gender     : chr "Male"
  ..$ culture    : chr "Ironborn"
  ..$ born       : chr "In 268 AC or before, at Pyke"
  ..$ died       : chr ""
  ..$ alive      : logi TRUE
  ..$ titles     : chr [1:2] "Lord Captain of the Iron Fleet" "Master of the Iron Victory"
  ..$ aliases    : chr "The Iron Captain"
  ..$ father     : chr ""
  ..$ mother     : chr ""
  ..$ spouse     : chr ""
  ..$ allegiances: chr "House Greyjoy of Pyke"
  ..$ books      : chr [1:3] "A Game of Thrones" "A Clash of Kings" "A Storm of Swords"
  ..$ povBooks   : chr [1:2] "A Feast for Crows" "A Dance with Dragons"
  ..$ tvSeries   : chr ""
  ..$ playedBy   : chr ""
 $ :List of 18
  ..$ url        : chr "https://www.anapioficeandfire.com/api/characters/1109"
  ..$ id         : int 1109
  ..$ name       : chr "Will"
  ..$ gender     : chr "Male"
  ..$ culture    : chr ""
  ..$ born       : chr ""
  ..$ died       : chr "In 297 AC, at Haunted Forest"
  ..$ alive      : logi FALSE
  ..$ titles     : chr ""
  ..$ aliases    : chr ""
  ..$ father     : chr ""
  ..$ mother     : chr ""
  ..$ spouse     : chr ""
  ..$ allegiances: list()
  ..$ books      : chr "A Clash of Kings"
  ..$ povBooks   : chr "A Game of Thrones"
  ..$ tvSeries   : chr ""
  ..$ playedBy   : chr "Bronson Webb"
 $ :List of 18
  ..$ url        : chr "https://www.anapioficeandfire.com/api/characters/1166"
  ..$ id         : int 1166
  ..$ name       : chr "Areo Hotah"
  ..$ gender     : chr "Male"
  ..$ culture    : chr "Norvoshi"
  ..$ born       : chr "In 257 AC or before, at Norvos"
  ..$ died       : chr ""
  ..$ alive      : logi TRUE
  ..$ titles     : chr "Captain of the Guard at Sunspear"
  ..$ aliases    : chr ""
  ..$ father     : chr ""
  ..$ mother     : chr ""
  ..$ spouse     : chr ""
  ..$ allegiances: chr "House Nymeros Martell of Sunspear"
  ..$ books      : chr [1:3] "A Game of Thrones" "A Clash of Kings" "A Storm of Swords"
  ..$ povBooks   : chr [1:2] "A Feast for Crows" "A Dance with Dragons"
  ..$ tvSeries   : chr [1:2] "Season 5" "Season 6"
  ..$ playedBy   : chr "DeObia Oparei"
 $ :List of 18
  ..$ url        : chr "https://www.anapioficeandfire.com/api/characters/1267"
  ..$ id         : int 1267
  ..$ name       : chr "Chett"
  ..$ gender     : chr "Male"
  ..$ culture    : chr ""
  ..$ born       : chr "At Hag's Mire"
  ..$ died       : chr "In 299 AC, at Fist of the First Men"
  ..$ alive      : logi FALSE
  ..$ titles     : chr ""
  ..$ aliases    : chr ""
  ..$ father     : chr ""
  ..$ mother     : chr ""
  ..$ spouse     : chr ""
  ..$ allegiances: list()
  ..$ books      : chr [1:2] "A Game of Thrones" "A Clash of Kings"
  ..$ povBooks   : chr "A Storm of Swords"
  ..$ tvSeries   : chr ""
  ..$ playedBy   : chr ""
 $ :List of 18
  ..$ url        : chr "https://www.anapioficeandfire.com/api/characters/1295"
  ..$ id         : int 1295
  ..$ name       : chr "Cressen"
  ..$ gender     : chr "Male"
  ..$ culture    : chr ""
  ..$ born       : chr "In 219 AC or 220 AC"
  ..$ died       : chr "In 299 AC, at Dragonstone"
  ..$ alive      : logi FALSE
  ..$ titles     : chr "Maester"
  ..$ aliases    : chr ""
  ..$ father     : chr ""
  ..$ mother     : chr ""
  ..$ spouse     : chr ""
  ..$ allegiances: list()
  ..$ books      : chr [1:2] "A Storm of Swords" "A Feast for Crows"
  ..$ povBooks   : chr "A Clash of Kings"
  ..$ tvSeries   : chr "Season 2"
  ..$ playedBy   : chr "Oliver Ford"
 $ :List of 18
  ..$ url        : chr "https://www.anapioficeandfire.com/api/characters/130"
  ..$ id         : int 130
  ..$ name       : chr "Arianne Martell"
  ..$ gender     : chr "Female"
  ..$ culture    : chr "Dornish"
  ..$ born       : chr "In 276 AC, at Sunspear"
  ..$ died       : chr ""
  ..$ alive      : logi TRUE
  ..$ titles     : chr "Princess of Dorne"
  ..$ aliases    : chr ""
  ..$ father     : chr ""
  ..$ mother     : chr ""
  ..$ spouse     : chr ""
  ..$ allegiances: chr "House Nymeros Martell of Sunspear"
  ..$ books      : chr [1:4] "A Game of Thrones" "A Clash of Kings" "A Storm of Swords" "A Dance with Dragons"
  ..$ povBooks   : chr "A Feast for Crows"
  ..$ tvSeries   : chr ""
  ..$ playedBy   : chr ""
 $ :List of 18
  ..$ url        : chr "https://www.anapioficeandfire.com/api/characters/1303"
  ..$ id         : int 1303
  ..$ name       : chr "Daenerys Targaryen"
  ..$ gender     : chr "Female"
  ..$ culture    : chr "Valyrian"
  ..$ born       : chr "In 284 AC, at Dragonstone"
  ..$ died       : chr ""
  ..$ alive      : logi TRUE
  ..$ titles     : chr [1:5] "Queen of the Andals and the Rhoynar and the First Men, Lord of the Seven Kingdoms" "Khaleesi of the Great Grass Sea" "Breaker of Shackles/Chains" "Queen of Meereen" ...
  ..$ aliases    : chr [1:11] "Dany" "Daenerys Stormborn" "The Unburnt" "Mother of Dragons" ...
  ..$ father     : chr ""
  ..$ mother     : chr ""
  ..$ spouse     : chr "https://www.anapioficeandfire.com/api/characters/1346"
  ..$ allegiances: chr "House Targaryen of King's Landing"
  ..$ books      : chr "A Feast for Crows"
  ..$ povBooks   : chr [1:4] "A Game of Thrones" "A Clash of Kings" "A Storm of Swords" "A Dance with Dragons"
  ..$ tvSeries   : chr [1:6] "Season 1" "Season 2" "Season 3" "Season 4" ...
  ..$ playedBy   : chr "Emilia Clarke"
 $ :List of 18
  ..$ url        : chr "https://www.anapioficeandfire.com/api/characters/1319"
  ..$ id         : int 1319
  ..$ name       : chr "Davos Seaworth"
  ..$ gender     : chr "Male"
  ..$ culture    : chr "Westeros"
  ..$ born       : chr "In 260 AC or before, at King's Landing"
  ..$ died       : chr ""
  ..$ alive      : logi TRUE
  ..$ titles     : chr [1:4] "Ser" "Lord of the Rainwood" "Admiral of the Narrow Sea" "Hand of the King"
  ..$ aliases    : chr [1:5] "Onion Knight" "Davos Shorthand" "Ser Onions" "Onion Lord" ...
  ..$ father     : chr ""
  ..$ mother     : chr ""
  ..$ spouse     : chr "https://www.anapioficeandfire.com/api/characters/1676"
  ..$ allegiances: chr [1:2] "House Baratheon of Dragonstone" "House Seaworth of Cape Wrath"
  ..$ books      : chr "A Feast for Crows"
  ..$ povBooks   : chr [1:3] "A Clash of Kings" "A Storm of Swords" "A Dance with Dragons"
  ..$ tvSeries   : chr [1:5] "Season 2" "Season 3" "Season 4" "Season 5" ...
  ..$ playedBy   : chr "Liam Cunningham"
 $ :List of 18
  ..$ url        : chr "https://www.anapioficeandfire.com/api/characters/148"
  ..$ id         : int 148
  ..$ name       : chr "Arya Stark"
  ..$ gender     : chr "Female"
  ..$ culture    : chr "Northmen"
  ..$ born       : chr "In 289 AC, at Winterfell"
  ..$ died       : chr ""
  ..$ alive      : logi TRUE
  ..$ titles     : chr "Princess"
  ..$ aliases    : chr [1:16] "Arya Horseface" "Arya Underfoot" "Arry" "Lumpyface" ...
  ..$ father     : chr ""
  ..$ mother     : chr ""
  ..$ spouse     : chr ""
  ..$ allegiances: chr "House Stark of Winterfell"
  ..$ books      : list()
  ..$ povBooks   : chr [1:5] "A Game of Thrones" "A Clash of Kings" "A Storm of Swords" "A Feast for Crows" ...
  ..$ tvSeries   : chr [1:6] "Season 1" "Season 2" "Season 3" "Season 4" ...
  ..$ playedBy   : chr "Maisie Williams"
 $ :List of 18
  ..$ url        : chr "https://www.anapioficeandfire.com/api/characters/149"
  ..$ id         : int 149
  ..$ name       : chr "Arys Oakheart"
  ..$ gender     : chr "Male"
  ..$ culture    : chr "Reach"
  ..$ born       : chr "At Old Oak"
  ..$ died       : chr "In 300 AC, at the Greenblood"
  ..$ alive      : logi FALSE
  ..$ titles     : chr "Ser"
  ..$ aliases    : chr ""
  ..$ father     : chr ""
  ..$ mother     : chr ""
  ..$ spouse     : chr ""
  ..$ allegiances: chr "House Oakheart of Old Oak"
  ..$ books      : chr [1:4] "A Game of Thrones" "A Clash of Kings" "A Storm of Swords" "A Dance with Dragons"
  ..$ povBooks   : chr "A Feast for Crows"
  ..$ tvSeries   : chr ""
  ..$ playedBy   : chr ""
 $ :List of 18
  ..$ url        : chr "https://www.anapioficeandfire.com/api/characters/150"
  ..$ id         : int 150
  ..$ name       : chr "Asha Greyjoy"
  ..$ gender     : chr "Female"
  ..$ culture    : chr "Ironborn"
  ..$ born       : chr "In 275 AC or 276 AC, at Pyke"
  ..$ died       : chr ""
  ..$ alive      : logi TRUE
  ..$ titles     : chr [1:3] "Princess" "Captain of the Black Wind" "Conqueror of Deepwood Motte"
  ..$ aliases    : chr [1:2] "Esgred" "The Kraken's Daughter"
  ..$ father     : chr ""
  ..$ mother     : chr ""
  ..$ spouse     : chr "https://www.anapioficeandfire.com/api/characters/1372"
  ..$ allegiances: chr [1:2] "House Greyjoy of Pyke" "House Ironmaker"
  ..$ books      : chr [1:2] "A Game of Thrones" "A Clash of Kings"
  ..$ povBooks   : chr [1:2] "A Feast for Crows" "A Dance with Dragons"
  ..$ tvSeries   : chr [1:3] "Season 2" "Season 3" "Season 4"
  ..$ playedBy   : chr "Gemma Whelan"
 $ :List of 18
  ..$ url        : chr "https://www.anapioficeandfire.com/api/characters/168"
  ..$ id         : int 168
  ..$ name       : chr "Barristan Selmy"
  ..$ gender     : chr "Male"
  ..$ culture    : chr "Westeros"
  ..$ born       : chr "In 237 AC"
  ..$ died       : chr ""
  ..$ alive      : logi TRUE
  ..$ titles     : chr [1:2] "Ser" "Hand of the Queen"
  ..$ aliases    : chr [1:5] "Barristan the Bold" "Arstan Whitebeard" "Ser Grandfather" "Barristan the Old" ...
  ..$ father     : chr ""
  ..$ mother     : chr ""
  ..$ spouse     : chr ""
  ..$ allegiances: chr [1:2] "House Selmy of Harvest Hall" "House Targaryen of King's Landing"
  ..$ books      : chr [1:5] "A Game of Thrones" "A Clash of Kings" "A Storm of Swords" "A Feast for Crows" ...
  ..$ povBooks   : chr "A Dance with Dragons"
  ..$ tvSeries   : chr [1:4] "Season 1" "Season 3" "Season 4" "Season 5"
  ..$ playedBy   : chr "Ian McElhinney"
 $ :List of 18
  ..$ url        : chr "https://www.anapioficeandfire.com/api/characters/2066"
  ..$ id         : int 2066
  ..$ name       : chr "Varamyr"
  ..$ gender     : chr "Male"
  ..$ culture    : chr "Free Folk"
  ..$ born       : chr "At a village Beyond the Wall"
  ..$ died       : chr "In 300 AC, at a village Beyond the Wall"
  ..$ alive      : logi FALSE
  ..$ titles     : chr ""
  ..$ aliases    : chr [1:3] "Varamyr Sixskins" "Haggon" "Lump"
  ..$ father     : chr ""
  ..$ mother     : chr ""
  ..$ spouse     : chr ""
  ..$ allegiances: list()
  ..$ books      : chr "A Storm of Swords"
  ..$ povBooks   : chr "A Dance with Dragons"
  ..$ tvSeries   : chr ""
  ..$ playedBy   : chr ""
 $ :List of 18
  ..$ url        : chr "https://www.anapioficeandfire.com/api/characters/208"
  ..$ id         : int 208
  ..$ name       : chr "Brandon Stark"
  ..$ gender     : chr "Male"
  ..$ culture    : chr "Northmen"
  ..$ born       : chr "In 290 AC, at Winterfell"
  ..$ died       : chr ""
  ..$ alive      : logi TRUE
  ..$ titles     : chr "Prince of Winterfell"
  ..$ aliases    : chr [1:3] "Bran" "Bran the Broken" "The Winged Wolf"
  ..$ father     : chr ""
  ..$ mother     : chr ""
  ..$ spouse     : chr ""
  ..$ allegiances: chr "House Stark of Winterfell"
  ..$ books      : chr "A Feast for Crows"
  ..$ povBooks   : chr [1:4] "A Game of Thrones" "A Clash of Kings" "A Storm of Swords" "A Dance with Dragons"
  ..$ tvSeries   : chr [1:5] "Season 1" "Season 2" "Season 3" "Season 4" ...
  ..$ playedBy   : chr "Isaac Hempstead-Wright"
 $ :List of 18
  ..$ url        : chr "https://www.anapioficeandfire.com/api/characters/216"
  ..$ id         : int 216
  ..$ name       : chr "Brienne of Tarth"
  ..$ gender     : chr "Female"
  ..$ culture    : chr ""
  ..$ born       : chr "In 280 AC"
  ..$ died       : chr ""
  ..$ alive      : logi TRUE
  ..$ titles     : chr ""
  ..$ aliases    : chr [1:3] "The Maid of Tarth" "Brienne the Beauty" "Brienne the Blue"
  ..$ father     : chr ""
  ..$ mother     : chr ""
  ..$ spouse     : chr ""
  ..$ allegiances: chr [1:3] "House Baratheon of Storm's End" "House Stark of Winterfell" "House Tarth of Evenfall Hall"
  ..$ books      : chr [1:3] "A Clash of Kings" "A Storm of Swords" "A Dance with Dragons"
  ..$ povBooks   : chr "A Feast for Crows"
  ..$ tvSeries   : chr [1:5] "Season 2" "Season 3" "Season 4" "Season 5" ...
  ..$ playedBy   : chr "Gwendoline Christie"
 $ :List of 18
  ..$ url        : chr "https://www.anapioficeandfire.com/api/characters/232"
  ..$ id         : int 232
  ..$ name       : chr "Catelyn Stark"
  ..$ gender     : chr "Female"
  ..$ culture    : chr "Rivermen"
  ..$ born       : chr "In 264 AC, at Riverrun"
  ..$ died       : chr "In 299 AC, at the Twins"
  ..$ alive      : logi FALSE
  ..$ titles     : chr "Lady of Winterfell"
  ..$ aliases    : chr [1:5] "Catelyn Tully" "Lady Stoneheart" "The Silent Sistet" "Mother Mercilesr" ...
  ..$ father     : chr ""
  ..$ mother     : chr ""
  ..$ spouse     : chr "https://www.anapioficeandfire.com/api/characters/339"
  ..$ allegiances: chr [1:2] "House Stark of Winterfell" "House Tully of Riverrun"
  ..$ books      : chr [1:2] "A Feast for Crows" "A Dance with Dragons"
  ..$ povBooks   : chr [1:3] "A Game of Thrones" "A Clash of Kings" "A Storm of Swords"
  ..$ tvSeries   : chr [1:3] "Season 1" "Season 2" "Season 3"
  ..$ playedBy   : chr "Michelle Fairley"
 $ :List of 18
  ..$ url        : chr "https://www.anapioficeandfire.com/api/characters/238"
  ..$ id         : int 238
  ..$ name       : chr "Cersei Lannister"
  ..$ gender     : chr "Female"
  ..$ culture    : chr "Westerman"
  ..$ born       : chr "In 266 AC, at Casterly Rock"
  ..$ died       : chr ""
  ..$ alive      : logi TRUE
  ..$ titles     : chr [1:5] "Light of the West" "Queen Dowager" "Protector of the Realm" "Lady of Casterly Rock" ...
  ..$ aliases    : chr ""
  ..$ father     : chr ""
  ..$ mother     : chr ""
  ..$ spouse     : chr "https://www.anapioficeandfire.com/api/characters/901"
  ..$ allegiances: chr "House Lannister of Casterly Rock"
  ..$ books      : chr [1:3] "A Game of Thrones" "A Clash of Kings" "A Storm of Swords"
  ..$ povBooks   : chr [1:2] "A Feast for Crows" "A Dance with Dragons"
  ..$ tvSeries   : chr [1:6] "Season 1" "Season 2" "Season 3" "Season 4" ...
  ..$ playedBy   : chr "Lena Headey"
 $ :List of 18
  ..$ url        : chr "https://www.anapioficeandfire.com/api/characters/339"
  ..$ id         : int 339
  ..$ name       : chr "Eddard Stark"
  ..$ gender     : chr "Male"
  ..$ culture    : chr "Northmen"
  ..$ born       : chr "In 263 AC, at Winterfell"
  ..$ died       : chr "In 299 AC, at Great Sept of Baelor in King's Landing"
  ..$ alive      : logi FALSE
  ..$ titles     : chr [1:5] "Lord of Winterfell" "Warden of the North" "Hand of the King" "Protector of the Realm" ...
  ..$ aliases    : chr [1:3] "Ned" "The Ned" "The Quiet Wolf"
  ..$ father     : chr ""
  ..$ mother     : chr ""
  ..$ spouse     : chr "https://www.anapioficeandfire.com/api/characters/232"
  ..$ allegiances: chr "House Stark of Winterfell"
  ..$ books      : chr [1:5] "A Clash of Kings" "A Storm of Swords" "A Feast for Crows" "A Dance with Dragons" ...
  ..$ povBooks   : chr "A Game of Thrones"
  ..$ tvSeries   : chr [1:2] "Season 1" "Season 6"
  ..$ playedBy   : chr [1:3] "Sean Bean" "Sebastian Croft" "Robert Aramayo"
 $ :List of 18
  ..$ url        : chr "https://www.anapioficeandfire.com/api/characters/529"
  ..$ id         : int 529
  ..$ name       : chr "Jaime Lannister"
  ..$ gender     : chr "Male"
  ..$ culture    : chr "Westerlands"
  ..$ born       : chr "In 266 AC, at Casterly Rock"
  ..$ died       : chr ""
  ..$ alive      : logi TRUE
  ..$ titles     : chr [1:3] "Ser" "Lord Commander of the Kingsguard" "Warden of the East (formerly)"
  ..$ aliases    : chr [1:4] "The Kingslayer" "The Lion of Lannister" "The Young Lion" "Cripple"
  ..$ father     : chr ""
  ..$ mother     : chr ""
  ..$ spouse     : chr ""
  ..$ allegiances: chr "House Lannister of Casterly Rock"
  ..$ books      : chr [1:2] "A Game of Thrones" "A Clash of Kings"
  ..$ povBooks   : chr [1:3] "A Storm of Swords" "A Feast for Crows" "A Dance with Dragons"
  ..$ tvSeries   : chr [1:5] "Season 1" "Season 2" "Season 3" "Season 4" ...
  ..$ playedBy   : chr "Nikolaj Coster-Waldau"
 $ :List of 18
  ..$ url        : chr "https://www.anapioficeandfire.com/api/characters/576"
  ..$ id         : int 576
  ..$ name       : chr "Jon Connington"
  ..$ gender     : chr "Male"
  ..$ culture    : chr "Stormlands"
  ..$ born       : chr "In or between 263 AC and 265 AC"
  ..$ died       : chr ""
  ..$ alive      : logi TRUE
  ..$ titles     : chr [1:3] "Lord of Griffin's Roost" "Hand of the King" "Hand of the True King"
  ..$ aliases    : chr "Griffthe Mad King's Hand"
  ..$ father     : chr ""
  ..$ mother     : chr ""
  ..$ spouse     : chr ""
  ..$ allegiances: chr [1:2] "House Connington of Griffin's Roost" "House Targaryen of King's Landing"
  ..$ books      : chr [1:3] "A Storm of Swords" "A Feast for Crows" "The World of Ice and Fire"
  ..$ povBooks   : chr "A Dance with Dragons"
  ..$ tvSeries   : chr ""
  ..$ playedBy   : chr ""
 $ :List of 18
  ..$ url        : chr "https://www.anapioficeandfire.com/api/characters/583"
  ..$ id         : int 583
  ..$ name       : chr "Jon Snow"
  ..$ gender     : chr "Male"
  ..$ culture    : chr "Northmen"
  ..$ born       : chr "In 283 AC"
  ..$ died       : chr ""
  ..$ alive      : logi TRUE
  ..$ titles     : chr "Lord Commander of the Night's Watch"
  ..$ aliases    : chr [1:8] "Lord Snow" "Ned Stark's Bastard" "The Snow of Winterfell" "The Crow-Come-Over" ...
  ..$ father     : chr ""
  ..$ mother     : chr ""
  ..$ spouse     : chr ""
  ..$ allegiances: chr "House Stark of Winterfell"
  ..$ books      : chr "A Feast for Crows"
  ..$ povBooks   : chr [1:4] "A Game of Thrones" "A Clash of Kings" "A Storm of Swords" "A Dance with Dragons"
  ..$ tvSeries   : chr [1:6] "Season 1" "Season 2" "Season 3" "Season 4" ...
  ..$ playedBy   : chr "Kit Harington"
 $ :List of 18
  ..$ url        : chr "https://www.anapioficeandfire.com/api/characters/60"
  ..$ id         : int 60
  ..$ name       : chr "Aeron Greyjoy"
  ..$ gender     : chr "Male"
  ..$ culture    : chr "Ironborn"
  ..$ born       : chr "In or between 269 AC and 273 AC, at Pyke"
  ..$ died       : chr ""
  ..$ alive      : logi TRUE
  ..$ titles     : chr [1:2] "Priest of the Drowned God" "Captain of the Golden Storm (formerly)"
  ..$ aliases    : chr [1:2] "The Damphair" "Aeron Damphair"
  ..$ father     : chr ""
  ..$ mother     : chr ""
  ..$ spouse     : chr ""
  ..$ allegiances: chr "House Greyjoy of Pyke"
  ..$ books      : chr [1:4] "A Game of Thrones" "A Clash of Kings" "A Storm of Swords" "A Dance with Dragons"
  ..$ povBooks   : chr "A Feast for Crows"
  ..$ tvSeries   : chr "Season 6"
  ..$ playedBy   : chr "Michael Feast"
 $ :List of 18
  ..$ url        : chr "https://www.anapioficeandfire.com/api/characters/605"
  ..$ id         : int 605
  ..$ name       : chr "Kevan Lannister"
  ..$ gender     : chr "Male"
  ..$ culture    : chr ""
  ..$ born       : chr "In 244 AC"
  ..$ died       : chr "In 300 AC, at King's Landing"
  ..$ alive      : logi FALSE
  ..$ titles     : chr [1:4] "Ser" "Master of laws" "Lord Regent" "Protector of the Realm"
  ..$ aliases    : chr ""
  ..$ father     : chr ""
  ..$ mother     : chr ""
  ..$ spouse     : chr "https://www.anapioficeandfire.com/api/characters/327"
  ..$ allegiances: chr "House Lannister of Casterly Rock"
  ..$ books      : chr [1:4] "A Game of Thrones" "A Clash of Kings" "A Storm of Swords" "A Feast for Crows"
  ..$ povBooks   : chr "A Dance with Dragons"
  ..$ tvSeries   : chr [1:4] "Season 1" "Season 2" "Season 5" "Season 6"
  ..$ playedBy   : chr "Ian Gelder"
 $ :List of 18
  ..$ url        : chr "https://www.anapioficeandfire.com/api/characters/743"
  ..$ id         : int 743
  ..$ name       : chr "Melisandre"
  ..$ gender     : chr "Female"
  ..$ culture    : chr "Asshai"
  ..$ born       : chr "At Unknown"
  ..$ died       : chr ""
  ..$ alive      : logi TRUE
  ..$ titles     : chr ""
  ..$ aliases    : chr [1:5] "The Red Priestess" "The Red Woman" "The King's Red Shadow" "Lady Red" ...
  ..$ father     : chr ""
  ..$ mother     : chr ""
  ..$ spouse     : chr ""
  ..$ allegiances: list()
  ..$ books      : chr [1:3] "A Clash of Kings" "A Storm of Swords" "A Feast for Crows"
  ..$ povBooks   : chr "A Dance with Dragons"
  ..$ tvSeries   : chr [1:5] "Season 2" "Season 3" "Season 4" "Season 5" ...
  ..$ playedBy   : chr "Carice van Houten"
 $ :List of 18
  ..$ url        : chr "https://www.anapioficeandfire.com/api/characters/751"
  ..$ id         : int 751
  ..$ name       : chr "Merrett Frey"
  ..$ gender     : chr "Male"
  ..$ culture    : chr "Rivermen"
  ..$ born       : chr "In 262 AC"
  ..$ died       : chr "In 300 AC, at Near Oldstones"
  ..$ alive      : logi FALSE
  ..$ titles     : chr ""
  ..$ aliases    : chr "Merrett Muttonhead"
  ..$ father     : chr ""
  ..$ mother     : chr ""
  ..$ spouse     : chr "https://www.anapioficeandfire.com/api/characters/712"
  ..$ allegiances: chr "House Frey of the Crossing"
  ..$ books      : chr [1:4] "A Game of Thrones" "A Clash of Kings" "A Feast for Crows" "A Dance with Dragons"
  ..$ povBooks   : chr "A Storm of Swords"
  ..$ tvSeries   : chr ""
  ..$ playedBy   : chr ""
 $ :List of 18
  ..$ url        : chr "https://www.anapioficeandfire.com/api/characters/844"
  ..$ id         : int 844
  ..$ name       : chr "Quentyn Martell"
  ..$ gender     : chr "Male"
  ..$ culture    : chr "Dornish"
  ..$ born       : chr "In 281 AC, at Sunspear, Dorne"
  ..$ died       : chr "In 300 AC, at Meereen"
  ..$ alive      : logi FALSE
  ..$ titles     : chr "Prince"
  ..$ aliases    : chr [1:4] "Frog" "Prince Frog" "The prince who came too late" "The Dragonrider"
  ..$ father     : chr ""
  ..$ mother     : chr ""
  ..$ spouse     : chr ""
  ..$ allegiances: chr "House Nymeros Martell of Sunspear"
  ..$ books      : chr [1:4] "A Game of Thrones" "A Clash of Kings" "A Storm of Swords" "A Feast for Crows"
  ..$ povBooks   : chr "A Dance with Dragons"
  ..$ tvSeries   : chr ""
  ..$ playedBy   : chr ""
 $ :List of 18
  ..$ url        : chr "https://www.anapioficeandfire.com/api/characters/954"
  ..$ id         : int 954
  ..$ name       : chr "Samwell Tarly"
  ..$ gender     : chr "Male"
  ..$ culture    : chr "Andal"
  ..$ born       : chr "In 283 AC, at Horn Hill"
  ..$ died       : chr ""
  ..$ alive      : logi TRUE
  ..$ titles     : chr ""
  ..$ aliases    : chr [1:7] "Sam" "Ser Piggy" "Prince Pork-chop" "Lady Piggy" ...
  ..$ father     : chr ""
  ..$ mother     : chr ""
  ..$ spouse     : chr ""
  ..$ allegiances: chr "House Tarly of Horn Hill"
  ..$ books      : chr [1:3] "A Game of Thrones" "A Clash of Kings" "A Dance with Dragons"
  ..$ povBooks   : chr [1:2] "A Storm of Swords" "A Feast for Crows"
  ..$ tvSeries   : chr [1:6] "Season 1" "Season 2" "Season 3" "Season 4" ...
  ..$ playedBy   : chr "John Bradley-West"
 $ :List of 18
  ..$ url        : chr "https://www.anapioficeandfire.com/api/characters/957"
  ..$ id         : int 957
  ..$ name       : chr "Sansa Stark"
  ..$ gender     : chr "Female"
  ..$ culture    : chr "Northmen"
  ..$ born       : chr "In 286 AC, at Winterfell"
  ..$ died       : chr ""
  ..$ alive      : logi TRUE
  ..$ titles     : chr "Princess"
  ..$ aliases    : chr [1:3] "Little bird" "Alayne Stone" "Jonquil"
  ..$ father     : chr ""
  ..$ mother     : chr ""
  ..$ spouse     : chr "https://www.anapioficeandfire.com/api/characters/1052"
  ..$ allegiances: chr [1:2] "House Baelish of Harrenhal" "House Stark of Winterfell"
  ..$ books      : chr "A Dance with Dragons"
  ..$ povBooks   : chr [1:4] "A Game of Thrones" "A Clash of Kings" "A Storm of Swords" "A Feast for Crows"
  ..$ tvSeries   : chr [1:6] "Season 1" "Season 2" "Season 3" "Season 4" ...
  ..$ playedBy   : chr "Sophie Turner"

Rectangling GOT

Start by turning it into tibble:

chars = tibble(json = got_chars)
chars

# A tibble: 30 × 1
   json             
   <list>           
 1 <named list [18]>
 2 <named list [18]>
 3 <named list [18]>
 4 <named list [18]>
 5 <named list [18]>
 6 <named list [18]>
 7 <named list [18]>
 8 <named list [18]>
 9 <named list [18]>
10 <named list [18]>
# ℹ 20 more rows

Rectangling GOT

Lists are named, structure is same, so use unnest_wider

chars |> unnest_wider(json)

# A tibble: 30 × 18
   url           id name  gender culture born  died  alive titles aliases father
   <chr>      <int> <chr> <chr>  <chr>   <chr> <chr> <lgl> <list> <list>  <chr> 
 1 https://w…  1022 Theo… Male   "Ironb… "In … ""    TRUE  <chr>  <chr>   ""    
 2 https://w…  1052 Tyri… Male   ""      "In … ""    TRUE  <chr>  <chr>   ""    
 3 https://w…  1074 Vict… Male   "Ironb… "In … ""    TRUE  <chr>  <chr>   ""    
 4 https://w…  1109 Will  Male   ""      ""    "In … FALSE <chr>  <chr>   ""    
 5 https://w…  1166 Areo… Male   "Norvo… "In … ""    TRUE  <chr>  <chr>   ""    
 6 https://w…  1267 Chett Male   ""      "At … "In … FALSE <chr>  <chr>   ""    
 7 https://w…  1295 Cres… Male   ""      "In … "In … FALSE <chr>  <chr>   ""    
 8 https://w…   130 Aria… Female "Dorni… "In … ""    TRUE  <chr>  <chr>   ""    
 9 https://w…  1303 Daen… Female "Valyr… "In … ""    TRUE  <chr>  <chr>   ""    
10 https://w…  1319 Davo… Male   "Weste… "In … ""    TRUE  <chr>  <chr>   ""    
# ℹ 20 more rows
# ℹ 7 more variables: mother <chr>, spouse <chr>, allegiances <list>,
#   books <list>, povBooks <list>, tvSeries <list>, playedBy <list>

Create `povBooks` table

In Game of Thrones books, some characters had point of view chapters
Let’s create a table with all the data on that

chars |>
  unnest_wider(json) |> 
  select(id,name,povBooks) |> head(5)

# A tibble: 5 × 3
     id name              povBooks 
  <int> <chr>             <list>   
1  1022 Theon Greyjoy     <chr [2]>
2  1052 Tyrion Lannister  <chr [4]>
3  1074 Victarion Greyjoy <chr [2]>
4  1109 Will              <chr [1]>
5  1166 Areo Hotah        <chr [2]>

Create `povBooks` table

The lists aren’t named, so we need to use unnest_longer

chars |> 
  unnest_wider(json) |> 
  select(id,name,povBooks) |> 
  unnest_longer(povBooks) |> head(10)

# A tibble: 10 × 3
      id name              povBooks            
   <int> <chr>             <chr>               
 1  1022 Theon Greyjoy     A Clash of Kings    
 2  1022 Theon Greyjoy     A Dance with Dragons
 3  1052 Tyrion Lannister  A Game of Thrones   
 4  1052 Tyrion Lannister  A Clash of Kings    
 5  1052 Tyrion Lannister  A Storm of Swords   
 6  1052 Tyrion Lannister  A Dance with Dragons
 7  1074 Victarion Greyjoy A Feast for Crows   
 8  1074 Victarion Greyjoy A Dance with Dragons
 9  1109 Will              A Game of Thrones   
10  1166 Areo Hotah        A Feast for Crows

JSON Files

Hierarchical data format commonly used on the internet

[1] │ {"menu": {
    │   "id": "file",
    │   "value": "File",
    │   "popup": {
    │     "menuitem": [
    │       {"value": "New", "onclick": "CreateNewDoc()"},
    │       {"value": "Open", "onclick": "OpenDoc()"},
    │       {"value": "Close", "onclick": "CloseDoc()"}
    │     ]
    │   }
    │ }}

JSON Files

string, number, bool, null
Array: [1.0, 3.0, 1, "Cat"]
Object: {"Capital":"Washington","Pop":330000000}
- Nested "key":value pairs

`jsonlite`

Several functions exist for reading and parsing json files

got_chars_json()

[1] "/home/georgehagstrom/R/x86_64-pc-linux-gnu-library/4.5/repurrrsive/extdata/got_chars.json"

got_chars_json() |> read_json()

[[1]]
[[1]]$url
[1] "https://www.anapioficeandfire.com/api/characters/1022"

[[1]]$id
[1] 1022

[[1]]$name
[1] "Theon Greyjoy"

[[1]]$gender
[1] "Male"

[[1]]$culture
[1] "Ironborn"

[[1]]$born
[1] "In 278 AC or 279 AC, at Pyke"

[[1]]$died
[1] ""

[[1]]$alive
[1] TRUE

[[1]]$titles
[[1]]$titles[[1]]
[1] "Prince of Winterfell"

[[1]]$titles[[2]]
[1] "Lord of the Iron Islands (by law of the green lands)"


[[1]]$aliases
[[1]]$aliases[[1]]
[1] "Prince of Fools"

[[1]]$aliases[[2]]
[1] "Theon Turncloak"

[[1]]$aliases[[3]]
[1] "Reek"

[[1]]$aliases[[4]]
[1] "Theon Kinslayer"


[[1]]$father
[1] ""

[[1]]$mother
[1] ""

[[1]]$spouse
[1] ""

[[1]]$allegiances
[1] "House Greyjoy of Pyke"

[[1]]$books
[[1]]$books[[1]]
[1] "A Game of Thrones"

[[1]]$books[[2]]
[1] "A Storm of Swords"

[[1]]$books[[3]]
[1] "A Feast for Crows"


[[1]]$povBooks
[[1]]$povBooks[[1]]
[1] "A Clash of Kings"

[[1]]$povBooks[[2]]
[1] "A Dance with Dragons"


[[1]]$tvSeries
[[1]]$tvSeries[[1]]
[1] "Season 1"

[[1]]$tvSeries[[2]]
[1] "Season 2"

[[1]]$tvSeries[[3]]
[1] "Season 3"

[[1]]$tvSeries[[4]]
[1] "Season 4"

[[1]]$tvSeries[[5]]
[1] "Season 5"

[[1]]$tvSeries[[6]]
[1] "Season 6"


[[1]]$playedBy
[1] "Alfie Allen"


[[2]]
[[2]]$url
[1] "https://www.anapioficeandfire.com/api/characters/1052"

[[2]]$id
[1] 1052

[[2]]$name
[1] "Tyrion Lannister"

[[2]]$gender
[1] "Male"

[[2]]$culture
[1] ""

[[2]]$born
[1] "In 273 AC, at Casterly Rock"

[[2]]$died
[1] ""

[[2]]$alive
[1] TRUE

[[2]]$titles
[[2]]$titles[[1]]
[1] "Acting Hand of the King (former)"

[[2]]$titles[[2]]
[1] "Master of Coin (former)"


[[2]]$aliases
[[2]]$aliases[[1]]
[1] "The Imp"

[[2]]$aliases[[2]]
[1] "Halfman"

[[2]]$aliases[[3]]
[1] "The boyman"

[[2]]$aliases[[4]]
[1] "Giant of Lannister"

[[2]]$aliases[[5]]
[1] "Lord Tywin's Doom"

[[2]]$aliases[[6]]
[1] "Lord Tywin's Bane"

[[2]]$aliases[[7]]
[1] "Yollo"

[[2]]$aliases[[8]]
[1] "Hugor Hill"

[[2]]$aliases[[9]]
[1] "No-Nose"

[[2]]$aliases[[10]]
[1] "Freak"

[[2]]$aliases[[11]]
[1] "Dwarf"


[[2]]$father
[1] ""

[[2]]$mother
[1] ""

[[2]]$spouse
[1] "https://www.anapioficeandfire.com/api/characters/2044"

[[2]]$allegiances
[1] "House Lannister of Casterly Rock"

[[2]]$books
[[2]]$books[[1]]
[1] "A Feast for Crows"

[[2]]$books[[2]]
[1] "The World of Ice and Fire"


[[2]]$povBooks
[[2]]$povBooks[[1]]
[1] "A Game of Thrones"

[[2]]$povBooks[[2]]
[1] "A Clash of Kings"

[[2]]$povBooks[[3]]
[1] "A Storm of Swords"

[[2]]$povBooks[[4]]
[1] "A Dance with Dragons"


[[2]]$tvSeries
[[2]]$tvSeries[[1]]
[1] "Season 1"

[[2]]$tvSeries[[2]]
[1] "Season 2"

[[2]]$tvSeries[[3]]
[1] "Season 3"

[[2]]$tvSeries[[4]]
[1] "Season 4"

[[2]]$tvSeries[[5]]
[1] "Season 5"

[[2]]$tvSeries[[6]]
[1] "Season 6"


[[2]]$playedBy
[1] "Peter Dinklage"


[[3]]
[[3]]$url
[1] "https://www.anapioficeandfire.com/api/characters/1074"

[[3]]$id
[1] 1074

[[3]]$name
[1] "Victarion Greyjoy"

[[3]]$gender
[1] "Male"

[[3]]$culture
[1] "Ironborn"

[[3]]$born
[1] "In 268 AC or before, at Pyke"

[[3]]$died
[1] ""

[[3]]$alive
[1] TRUE

[[3]]$titles
[[3]]$titles[[1]]
[1] "Lord Captain of the Iron Fleet"

[[3]]$titles[[2]]
[1] "Master of the Iron Victory"


[[3]]$aliases
[1] "The Iron Captain"

[[3]]$father
[1] ""

[[3]]$mother
[1] ""

[[3]]$spouse
[1] ""

[[3]]$allegiances
[1] "House Greyjoy of Pyke"

[[3]]$books
[[3]]$books[[1]]
[1] "A Game of Thrones"

[[3]]$books[[2]]
[1] "A Clash of Kings"

[[3]]$books[[3]]
[1] "A Storm of Swords"


[[3]]$povBooks
[[3]]$povBooks[[1]]
[1] "A Feast for Crows"

[[3]]$povBooks[[2]]
[1] "A Dance with Dragons"


[[3]]$tvSeries
[1] ""

[[3]]$playedBy
[1] ""


[[4]]
[[4]]$url
[1] "https://www.anapioficeandfire.com/api/characters/1109"

[[4]]$id
[1] 1109

[[4]]$name
[1] "Will"

[[4]]$gender
[1] "Male"

[[4]]$culture
[1] ""

[[4]]$born
[1] ""

[[4]]$died
[1] "In 297 AC, at Haunted Forest"

[[4]]$alive
[1] FALSE

[[4]]$titles
[1] ""

[[4]]$aliases
[1] ""

[[4]]$father
[1] ""

[[4]]$mother
[1] ""

[[4]]$spouse
[1] ""

[[4]]$allegiances
list()

[[4]]$books
[1] "A Clash of Kings"

[[4]]$povBooks
[1] "A Game of Thrones"

[[4]]$tvSeries
[1] ""

[[4]]$playedBy
[1] "Bronson Webb"


[[5]]
[[5]]$url
[1] "https://www.anapioficeandfire.com/api/characters/1166"

[[5]]$id
[1] 1166

[[5]]$name
[1] "Areo Hotah"

[[5]]$gender
[1] "Male"

[[5]]$culture
[1] "Norvoshi"

[[5]]$born
[1] "In 257 AC or before, at Norvos"

[[5]]$died
[1] ""

[[5]]$alive
[1] TRUE

[[5]]$titles
[1] "Captain of the Guard at Sunspear"

[[5]]$aliases
[1] ""

[[5]]$father
[1] ""

[[5]]$mother
[1] ""

[[5]]$spouse
[1] ""

[[5]]$allegiances
[1] "House Nymeros Martell of Sunspear"

[[5]]$books
[[5]]$books[[1]]
[1] "A Game of Thrones"

[[5]]$books[[2]]
[1] "A Clash of Kings"

[[5]]$books[[3]]
[1] "A Storm of Swords"


[[5]]$povBooks
[[5]]$povBooks[[1]]
[1] "A Feast for Crows"

[[5]]$povBooks[[2]]
[1] "A Dance with Dragons"


[[5]]$tvSeries
[[5]]$tvSeries[[1]]
[1] "Season 5"

[[5]]$tvSeries[[2]]
[1] "Season 6"


[[5]]$playedBy
[1] "DeObia Oparei"


[[6]]
[[6]]$url
[1] "https://www.anapioficeandfire.com/api/characters/1267"

[[6]]$id
[1] 1267

[[6]]$name
[1] "Chett"

[[6]]$gender
[1] "Male"

[[6]]$culture
[1] ""

[[6]]$born
[1] "At Hag's Mire"

[[6]]$died
[1] "In 299 AC, at Fist of the First Men"

[[6]]$alive
[1] FALSE

[[6]]$titles
[1] ""

[[6]]$aliases
[1] ""

[[6]]$father
[1] ""

[[6]]$mother
[1] ""

[[6]]$spouse
[1] ""

[[6]]$allegiances
list()

[[6]]$books
[[6]]$books[[1]]
[1] "A Game of Thrones"

[[6]]$books[[2]]
[1] "A Clash of Kings"


[[6]]$povBooks
[1] "A Storm of Swords"

[[6]]$tvSeries
[1] ""

[[6]]$playedBy
[1] ""


[[7]]
[[7]]$url
[1] "https://www.anapioficeandfire.com/api/characters/1295"

[[7]]$id
[1] 1295

[[7]]$name
[1] "Cressen"

[[7]]$gender
[1] "Male"

[[7]]$culture
[1] ""

[[7]]$born
[1] "In 219 AC or 220 AC"

[[7]]$died
[1] "In 299 AC, at Dragonstone"

[[7]]$alive
[1] FALSE

[[7]]$titles
[1] "Maester"

[[7]]$aliases
[1] ""

[[7]]$father
[1] ""

[[7]]$mother
[1] ""

[[7]]$spouse
[1] ""

[[7]]$allegiances
list()

[[7]]$books
[[7]]$books[[1]]
[1] "A Storm of Swords"

[[7]]$books[[2]]
[1] "A Feast for Crows"


[[7]]$povBooks
[1] "A Clash of Kings"

[[7]]$tvSeries
[1] "Season 2"

[[7]]$playedBy
[1] "Oliver Ford"


[[8]]
[[8]]$url
[1] "https://www.anapioficeandfire.com/api/characters/130"

[[8]]$id
[1] 130

[[8]]$name
[1] "Arianne Martell"

[[8]]$gender
[1] "Female"

[[8]]$culture
[1] "Dornish"

[[8]]$born
[1] "In 276 AC, at Sunspear"

[[8]]$died
[1] ""

[[8]]$alive
[1] TRUE

[[8]]$titles
[1] "Princess of Dorne"

[[8]]$aliases
[1] ""

[[8]]$father
[1] ""

[[8]]$mother
[1] ""

[[8]]$spouse
[1] ""

[[8]]$allegiances
[1] "House Nymeros Martell of Sunspear"

[[8]]$books
[[8]]$books[[1]]
[1] "A Game of Thrones"

[[8]]$books[[2]]
[1] "A Clash of Kings"

[[8]]$books[[3]]
[1] "A Storm of Swords"

[[8]]$books[[4]]
[1] "A Dance with Dragons"


[[8]]$povBooks
[1] "A Feast for Crows"

[[8]]$tvSeries
[1] ""

[[8]]$playedBy
[1] ""


[[9]]
[[9]]$url
[1] "https://www.anapioficeandfire.com/api/characters/1303"

[[9]]$id
[1] 1303

[[9]]$name
[1] "Daenerys Targaryen"

[[9]]$gender
[1] "Female"

[[9]]$culture
[1] "Valyrian"

[[9]]$born
[1] "In 284 AC, at Dragonstone"

[[9]]$died
[1] ""

[[9]]$alive
[1] TRUE

[[9]]$titles
[[9]]$titles[[1]]
[1] "Queen of the Andals and the Rhoynar and the First Men, Lord of the Seven Kingdoms"

[[9]]$titles[[2]]
[1] "Khaleesi of the Great Grass Sea"

[[9]]$titles[[3]]
[1] "Breaker of Shackles/Chains"

[[9]]$titles[[4]]
[1] "Queen of Meereen"

[[9]]$titles[[5]]
[1] "Princess of Dragonstone"


[[9]]$aliases
[[9]]$aliases[[1]]
[1] "Dany"

[[9]]$aliases[[2]]
[1] "Daenerys Stormborn"

[[9]]$aliases[[3]]
[1] "The Unburnt"

[[9]]$aliases[[4]]
[1] "Mother of Dragons"

[[9]]$aliases[[5]]
[1] "Mother"

[[9]]$aliases[[6]]
[1] "Mhysa"

[[9]]$aliases[[7]]
[1] "The Silver Queen"

[[9]]$aliases[[8]]
[1] "Silver Lady"

[[9]]$aliases[[9]]
[1] "Dragonmother"

[[9]]$aliases[[10]]
[1] "The Dragon Queen"

[[9]]$aliases[[11]]
[1] "The Mad King's daughter"


[[9]]$father
[1] ""

[[9]]$mother
[1] ""

[[9]]$spouse
[1] "https://www.anapioficeandfire.com/api/characters/1346"

[[9]]$allegiances
[1] "House Targaryen of King's Landing"

[[9]]$books
[1] "A Feast for Crows"

[[9]]$povBooks
[[9]]$povBooks[[1]]
[1] "A Game of Thrones"

[[9]]$povBooks[[2]]
[1] "A Clash of Kings"

[[9]]$povBooks[[3]]
[1] "A Storm of Swords"

[[9]]$povBooks[[4]]
[1] "A Dance with Dragons"


[[9]]$tvSeries
[[9]]$tvSeries[[1]]
[1] "Season 1"

[[9]]$tvSeries[[2]]
[1] "Season 2"

[[9]]$tvSeries[[3]]
[1] "Season 3"

[[9]]$tvSeries[[4]]
[1] "Season 4"

[[9]]$tvSeries[[5]]
[1] "Season 5"

[[9]]$tvSeries[[6]]
[1] "Season 6"


[[9]]$playedBy
[1] "Emilia Clarke"


[[10]]
[[10]]$url
[1] "https://www.anapioficeandfire.com/api/characters/1319"

[[10]]$id
[1] 1319

[[10]]$name
[1] "Davos Seaworth"

[[10]]$gender
[1] "Male"

[[10]]$culture
[1] "Westeros"

[[10]]$born
[1] "In 260 AC or before, at King's Landing"

[[10]]$died
[1] ""

[[10]]$alive
[1] TRUE

[[10]]$titles
[[10]]$titles[[1]]
[1] "Ser"

[[10]]$titles[[2]]
[1] "Lord of the Rainwood"

[[10]]$titles[[3]]
[1] "Admiral of the Narrow Sea"

[[10]]$titles[[4]]
[1] "Hand of the King"


[[10]]$aliases
[[10]]$aliases[[1]]
[1] "Onion Knight"

[[10]]$aliases[[2]]
[1] "Davos Shorthand"

[[10]]$aliases[[3]]
[1] "Ser Onions"

[[10]]$aliases[[4]]
[1] "Onion Lord"

[[10]]$aliases[[5]]
[1] "Smuggler"


[[10]]$father
[1] ""

[[10]]$mother
[1] ""

[[10]]$spouse
[1] "https://www.anapioficeandfire.com/api/characters/1676"

[[10]]$allegiances
[[10]]$allegiances[[1]]
[1] "House Baratheon of Dragonstone"

[[10]]$allegiances[[2]]
[1] "House Seaworth of Cape Wrath"


[[10]]$books
[1] "A Feast for Crows"

[[10]]$povBooks
[[10]]$povBooks[[1]]
[1] "A Clash of Kings"

[[10]]$povBooks[[2]]
[1] "A Storm of Swords"

[[10]]$povBooks[[3]]
[1] "A Dance with Dragons"


[[10]]$tvSeries
[[10]]$tvSeries[[1]]
[1] "Season 2"

[[10]]$tvSeries[[2]]
[1] "Season 3"

[[10]]$tvSeries[[3]]
[1] "Season 4"

[[10]]$tvSeries[[4]]
[1] "Season 5"

[[10]]$tvSeries[[5]]
[1] "Season 6"


[[10]]$playedBy
[1] "Liam Cunningham"


[[11]]
[[11]]$url
[1] "https://www.anapioficeandfire.com/api/characters/148"

[[11]]$id
[1] 148

[[11]]$name
[1] "Arya Stark"

[[11]]$gender
[1] "Female"

[[11]]$culture
[1] "Northmen"

[[11]]$born
[1] "In 289 AC, at Winterfell"

[[11]]$died
[1] ""

[[11]]$alive
[1] TRUE

[[11]]$titles
[1] "Princess"

[[11]]$aliases
[[11]]$aliases[[1]]
[1] "Arya Horseface"

[[11]]$aliases[[2]]
[1] "Arya Underfoot"

[[11]]$aliases[[3]]
[1] "Arry"

[[11]]$aliases[[4]]
[1] "Lumpyface"

[[11]]$aliases[[5]]
[1] "Lumpyhead"

[[11]]$aliases[[6]]
[1] "Stickboy"

[[11]]$aliases[[7]]
[1] "Weasel"

[[11]]$aliases[[8]]
[1] "Nymeria"

[[11]]$aliases[[9]]
[1] "Squan"

[[11]]$aliases[[10]]
[1] "Saltb"

[[11]]$aliases[[11]]
[1] "Cat of the Canaly"

[[11]]$aliases[[12]]
[1] "Bets"

[[11]]$aliases[[13]]
[1] "The Blind Girh"

[[11]]$aliases[[14]]
[1] "The Ugly Little Girl"

[[11]]$aliases[[15]]
[1] "Mercedenl"

[[11]]$aliases[[16]]
[1] "Mercye"


[[11]]$father
[1] ""

[[11]]$mother
[1] ""

[[11]]$spouse
[1] ""

[[11]]$allegiances
[1] "House Stark of Winterfell"

[[11]]$books
list()

[[11]]$povBooks
[[11]]$povBooks[[1]]
[1] "A Game of Thrones"

[[11]]$povBooks[[2]]
[1] "A Clash of Kings"

[[11]]$povBooks[[3]]
[1] "A Storm of Swords"

[[11]]$povBooks[[4]]
[1] "A Feast for Crows"

[[11]]$povBooks[[5]]
[1] "A Dance with Dragons"


[[11]]$tvSeries
[[11]]$tvSeries[[1]]
[1] "Season 1"

[[11]]$tvSeries[[2]]
[1] "Season 2"

[[11]]$tvSeries[[3]]
[1] "Season 3"

[[11]]$tvSeries[[4]]
[1] "Season 4"

[[11]]$tvSeries[[5]]
[1] "Season 5"

[[11]]$tvSeries[[6]]
[1] "Season 6"


[[11]]$playedBy
[1] "Maisie Williams"


[[12]]
[[12]]$url
[1] "https://www.anapioficeandfire.com/api/characters/149"

[[12]]$id
[1] 149

[[12]]$name
[1] "Arys Oakheart"

[[12]]$gender
[1] "Male"

[[12]]$culture
[1] "Reach"

[[12]]$born
[1] "At Old Oak"

[[12]]$died
[1] "In 300 AC, at the Greenblood"

[[12]]$alive
[1] FALSE

[[12]]$titles
[1] "Ser"

[[12]]$aliases
[1] ""

[[12]]$father
[1] ""

[[12]]$mother
[1] ""

[[12]]$spouse
[1] ""

[[12]]$allegiances
[1] "House Oakheart of Old Oak"

[[12]]$books
[[12]]$books[[1]]
[1] "A Game of Thrones"

[[12]]$books[[2]]
[1] "A Clash of Kings"

[[12]]$books[[3]]
[1] "A Storm of Swords"

[[12]]$books[[4]]
[1] "A Dance with Dragons"


[[12]]$povBooks
[1] "A Feast for Crows"

[[12]]$tvSeries
[1] ""

[[12]]$playedBy
[1] ""


[[13]]
[[13]]$url
[1] "https://www.anapioficeandfire.com/api/characters/150"

[[13]]$id
[1] 150

[[13]]$name
[1] "Asha Greyjoy"

[[13]]$gender
[1] "Female"

[[13]]$culture
[1] "Ironborn"

[[13]]$born
[1] "In 275 AC or 276 AC, at Pyke"

[[13]]$died
[1] ""

[[13]]$alive
[1] TRUE

[[13]]$titles
[[13]]$titles[[1]]
[1] "Princess"

[[13]]$titles[[2]]
[1] "Captain of the Black Wind"

[[13]]$titles[[3]]
[1] "Conqueror of Deepwood Motte"


[[13]]$aliases
[[13]]$aliases[[1]]
[1] "Esgred"

[[13]]$aliases[[2]]
[1] "The Kraken's Daughter"


[[13]]$father
[1] ""

[[13]]$mother
[1] ""

[[13]]$spouse
[1] "https://www.anapioficeandfire.com/api/characters/1372"

[[13]]$allegiances
[[13]]$allegiances[[1]]
[1] "House Greyjoy of Pyke"

[[13]]$allegiances[[2]]
[1] "House Ironmaker"


[[13]]$books
[[13]]$books[[1]]
[1] "A Game of Thrones"

[[13]]$books[[2]]
[1] "A Clash of Kings"


[[13]]$povBooks
[[13]]$povBooks[[1]]
[1] "A Feast for Crows"

[[13]]$povBooks[[2]]
[1] "A Dance with Dragons"


[[13]]$tvSeries
[[13]]$tvSeries[[1]]
[1] "Season 2"

[[13]]$tvSeries[[2]]
[1] "Season 3"

[[13]]$tvSeries[[3]]
[1] "Season 4"


[[13]]$playedBy
[1] "Gemma Whelan"


[[14]]
[[14]]$url
[1] "https://www.anapioficeandfire.com/api/characters/168"

[[14]]$id
[1] 168

[[14]]$name
[1] "Barristan Selmy"

[[14]]$gender
[1] "Male"

[[14]]$culture
[1] "Westeros"

[[14]]$born
[1] "In 237 AC"

[[14]]$died
[1] ""

[[14]]$alive
[1] TRUE

[[14]]$titles
[[14]]$titles[[1]]
[1] "Ser"

[[14]]$titles[[2]]
[1] "Hand of the Queen"


[[14]]$aliases
[[14]]$aliases[[1]]
[1] "Barristan the Bold"

[[14]]$aliases[[2]]
[1] "Arstan Whitebeard"

[[14]]$aliases[[3]]
[1] "Ser Grandfather"

[[14]]$aliases[[4]]
[1] "Barristan the Old"

[[14]]$aliases[[5]]
[1] "Old Ser"


[[14]]$father
[1] ""

[[14]]$mother
[1] ""

[[14]]$spouse
[1] ""

[[14]]$allegiances
[[14]]$allegiances[[1]]
[1] "House Selmy of Harvest Hall"

[[14]]$allegiances[[2]]
[1] "House Targaryen of King's Landing"


[[14]]$books
[[14]]$books[[1]]
[1] "A Game of Thrones"

[[14]]$books[[2]]
[1] "A Clash of Kings"

[[14]]$books[[3]]
[1] "A Storm of Swords"

[[14]]$books[[4]]
[1] "A Feast for Crows"

[[14]]$books[[5]]
[1] "The World of Ice and Fire"


[[14]]$povBooks
[1] "A Dance with Dragons"

[[14]]$tvSeries
[[14]]$tvSeries[[1]]
[1] "Season 1"

[[14]]$tvSeries[[2]]
[1] "Season 3"

[[14]]$tvSeries[[3]]
[1] "Season 4"

[[14]]$tvSeries[[4]]
[1] "Season 5"


[[14]]$playedBy
[1] "Ian McElhinney"


[[15]]
[[15]]$url
[1] "https://www.anapioficeandfire.com/api/characters/2066"

[[15]]$id
[1] 2066

[[15]]$name
[1] "Varamyr"

[[15]]$gender
[1] "Male"

[[15]]$culture
[1] "Free Folk"

[[15]]$born
[1] "At a village Beyond the Wall"

[[15]]$died
[1] "In 300 AC, at a village Beyond the Wall"

[[15]]$alive
[1] FALSE

[[15]]$titles
[1] ""

[[15]]$aliases
[[15]]$aliases[[1]]
[1] "Varamyr Sixskins"

[[15]]$aliases[[2]]
[1] "Haggon"

[[15]]$aliases[[3]]
[1] "Lump"


[[15]]$father
[1] ""

[[15]]$mother
[1] ""

[[15]]$spouse
[1] ""

[[15]]$allegiances
list()

[[15]]$books
[1] "A Storm of Swords"

[[15]]$povBooks
[1] "A Dance with Dragons"

[[15]]$tvSeries
[1] ""

[[15]]$playedBy
[1] ""


[[16]]
[[16]]$url
[1] "https://www.anapioficeandfire.com/api/characters/208"

[[16]]$id
[1] 208

[[16]]$name
[1] "Brandon Stark"

[[16]]$gender
[1] "Male"

[[16]]$culture
[1] "Northmen"

[[16]]$born
[1] "In 290 AC, at Winterfell"

[[16]]$died
[1] ""

[[16]]$alive
[1] TRUE

[[16]]$titles
[1] "Prince of Winterfell"

[[16]]$aliases
[[16]]$aliases[[1]]
[1] "Bran"

[[16]]$aliases[[2]]
[1] "Bran the Broken"

[[16]]$aliases[[3]]
[1] "The Winged Wolf"


[[16]]$father
[1] ""

[[16]]$mother
[1] ""

[[16]]$spouse
[1] ""

[[16]]$allegiances
[1] "House Stark of Winterfell"

[[16]]$books
[1] "A Feast for Crows"

[[16]]$povBooks
[[16]]$povBooks[[1]]
[1] "A Game of Thrones"

[[16]]$povBooks[[2]]
[1] "A Clash of Kings"

[[16]]$povBooks[[3]]
[1] "A Storm of Swords"

[[16]]$povBooks[[4]]
[1] "A Dance with Dragons"


[[16]]$tvSeries
[[16]]$tvSeries[[1]]
[1] "Season 1"

[[16]]$tvSeries[[2]]
[1] "Season 2"

[[16]]$tvSeries[[3]]
[1] "Season 3"

[[16]]$tvSeries[[4]]
[1] "Season 4"

[[16]]$tvSeries[[5]]
[1] "Season 6"


[[16]]$playedBy
[1] "Isaac Hempstead-Wright"


[[17]]
[[17]]$url
[1] "https://www.anapioficeandfire.com/api/characters/216"

[[17]]$id
[1] 216

[[17]]$name
[1] "Brienne of Tarth"

[[17]]$gender
[1] "Female"

[[17]]$culture
[1] ""

[[17]]$born
[1] "In 280 AC"

[[17]]$died
[1] ""

[[17]]$alive
[1] TRUE

[[17]]$titles
[1] ""

[[17]]$aliases
[[17]]$aliases[[1]]
[1] "The Maid of Tarth"

[[17]]$aliases[[2]]
[1] "Brienne the Beauty"

[[17]]$aliases[[3]]
[1] "Brienne the Blue"


[[17]]$father
[1] ""

[[17]]$mother
[1] ""

[[17]]$spouse
[1] ""

[[17]]$allegiances
[[17]]$allegiances[[1]]
[1] "House Baratheon of Storm's End"

[[17]]$allegiances[[2]]
[1] "House Stark of Winterfell"

[[17]]$allegiances[[3]]
[1] "House Tarth of Evenfall Hall"


[[17]]$books
[[17]]$books[[1]]
[1] "A Clash of Kings"

[[17]]$books[[2]]
[1] "A Storm of Swords"

[[17]]$books[[3]]
[1] "A Dance with Dragons"


[[17]]$povBooks
[1] "A Feast for Crows"

[[17]]$tvSeries
[[17]]$tvSeries[[1]]
[1] "Season 2"

[[17]]$tvSeries[[2]]
[1] "Season 3"

[[17]]$tvSeries[[3]]
[1] "Season 4"

[[17]]$tvSeries[[4]]
[1] "Season 5"

[[17]]$tvSeries[[5]]
[1] "Season 6"


[[17]]$playedBy
[1] "Gwendoline Christie"


[[18]]
[[18]]$url
[1] "https://www.anapioficeandfire.com/api/characters/232"

[[18]]$id
[1] 232

[[18]]$name
[1] "Catelyn Stark"

[[18]]$gender
[1] "Female"

[[18]]$culture
[1] "Rivermen"

[[18]]$born
[1] "In 264 AC, at Riverrun"

[[18]]$died
[1] "In 299 AC, at the Twins"

[[18]]$alive
[1] FALSE

[[18]]$titles
[1] "Lady of Winterfell"

[[18]]$aliases
[[18]]$aliases[[1]]
[1] "Catelyn Tully"

[[18]]$aliases[[2]]
[1] "Lady Stoneheart"

[[18]]$aliases[[3]]
[1] "The Silent Sistet"

[[18]]$aliases[[4]]
[1] "Mother Mercilesr"

[[18]]$aliases[[5]]
[1] "The Hangwomans"


[[18]]$father
[1] ""

[[18]]$mother
[1] ""

[[18]]$spouse
[1] "https://www.anapioficeandfire.com/api/characters/339"

[[18]]$allegiances
[[18]]$allegiances[[1]]
[1] "House Stark of Winterfell"

[[18]]$allegiances[[2]]
[1] "House Tully of Riverrun"


[[18]]$books
[[18]]$books[[1]]
[1] "A Feast for Crows"

[[18]]$books[[2]]
[1] "A Dance with Dragons"


[[18]]$povBooks
[[18]]$povBooks[[1]]
[1] "A Game of Thrones"

[[18]]$povBooks[[2]]
[1] "A Clash of Kings"

[[18]]$povBooks[[3]]
[1] "A Storm of Swords"


[[18]]$tvSeries
[[18]]$tvSeries[[1]]
[1] "Season 1"

[[18]]$tvSeries[[2]]
[1] "Season 2"

[[18]]$tvSeries[[3]]
[1] "Season 3"


[[18]]$playedBy
[1] "Michelle Fairley"


[[19]]
[[19]]$url
[1] "https://www.anapioficeandfire.com/api/characters/238"

[[19]]$id
[1] 238

[[19]]$name
[1] "Cersei Lannister"

[[19]]$gender
[1] "Female"

[[19]]$culture
[1] "Westerman"

[[19]]$born
[1] "In 266 AC, at Casterly Rock"

[[19]]$died
[1] ""

[[19]]$alive
[1] TRUE

[[19]]$titles
[[19]]$titles[[1]]
[1] "Light of the West"

[[19]]$titles[[2]]
[1] "Queen Dowager"

[[19]]$titles[[3]]
[1] "Protector of the Realm"

[[19]]$titles[[4]]
[1] "Lady of Casterly Rock"

[[19]]$titles[[5]]
[1] "Queen Regent"


[[19]]$aliases
[1] ""

[[19]]$father
[1] ""

[[19]]$mother
[1] ""

[[19]]$spouse
[1] "https://www.anapioficeandfire.com/api/characters/901"

[[19]]$allegiances
[1] "House Lannister of Casterly Rock"

[[19]]$books
[[19]]$books[[1]]
[1] "A Game of Thrones"

[[19]]$books[[2]]
[1] "A Clash of Kings"

[[19]]$books[[3]]
[1] "A Storm of Swords"


[[19]]$povBooks
[[19]]$povBooks[[1]]
[1] "A Feast for Crows"

[[19]]$povBooks[[2]]
[1] "A Dance with Dragons"


[[19]]$tvSeries
[[19]]$tvSeries[[1]]
[1] "Season 1"

[[19]]$tvSeries[[2]]
[1] "Season 2"

[[19]]$tvSeries[[3]]
[1] "Season 3"

[[19]]$tvSeries[[4]]
[1] "Season 4"

[[19]]$tvSeries[[5]]
[1] "Season 5"

[[19]]$tvSeries[[6]]
[1] "Season 6"


[[19]]$playedBy
[1] "Lena Headey"


[[20]]
[[20]]$url
[1] "https://www.anapioficeandfire.com/api/characters/339"

[[20]]$id
[1] 339

[[20]]$name
[1] "Eddard Stark"

[[20]]$gender
[1] "Male"

[[20]]$culture
[1] "Northmen"

[[20]]$born
[1] "In 263 AC, at Winterfell"

[[20]]$died
[1] "In 299 AC, at Great Sept of Baelor in King's Landing"

[[20]]$alive
[1] FALSE

[[20]]$titles
[[20]]$titles[[1]]
[1] "Lord of Winterfell"

[[20]]$titles[[2]]
[1] "Warden of the North"

[[20]]$titles[[3]]
[1] "Hand of the King"

[[20]]$titles[[4]]
[1] "Protector of the Realm"

[[20]]$titles[[5]]
[1] "Regent"


[[20]]$aliases
[[20]]$aliases[[1]]
[1] "Ned"

[[20]]$aliases[[2]]
[1] "The Ned"

[[20]]$aliases[[3]]
[1] "The Quiet Wolf"


[[20]]$father
[1] ""

[[20]]$mother
[1] ""

[[20]]$spouse
[1] "https://www.anapioficeandfire.com/api/characters/232"

[[20]]$allegiances
[1] "House Stark of Winterfell"

[[20]]$books
[[20]]$books[[1]]
[1] "A Clash of Kings"

[[20]]$books[[2]]
[1] "A Storm of Swords"

[[20]]$books[[3]]
[1] "A Feast for Crows"

[[20]]$books[[4]]
[1] "A Dance with Dragons"

[[20]]$books[[5]]
[1] "The World of Ice and Fire"


[[20]]$povBooks
[1] "A Game of Thrones"

[[20]]$tvSeries
[[20]]$tvSeries[[1]]
[1] "Season 1"

[[20]]$tvSeries[[2]]
[1] "Season 6"


[[20]]$playedBy
[[20]]$playedBy[[1]]
[1] "Sean Bean"

[[20]]$playedBy[[2]]
[1] "Sebastian Croft"

[[20]]$playedBy[[3]]
[1] "Robert Aramayo"



[[21]]
[[21]]$url
[1] "https://www.anapioficeandfire.com/api/characters/529"

[[21]]$id
[1] 529

[[21]]$name
[1] "Jaime Lannister"

[[21]]$gender
[1] "Male"

[[21]]$culture
[1] "Westerlands"

[[21]]$born
[1] "In 266 AC, at Casterly Rock"

[[21]]$died
[1] ""

[[21]]$alive
[1] TRUE

[[21]]$titles
[[21]]$titles[[1]]
[1] "Ser"

[[21]]$titles[[2]]
[1] "Lord Commander of the Kingsguard"

[[21]]$titles[[3]]
[1] "Warden of the East (formerly)"


[[21]]$aliases
[[21]]$aliases[[1]]
[1] "The Kingslayer"

[[21]]$aliases[[2]]
[1] "The Lion of Lannister"

[[21]]$aliases[[3]]
[1] "The Young Lion"

[[21]]$aliases[[4]]
[1] "Cripple"


[[21]]$father
[1] ""

[[21]]$mother
[1] ""

[[21]]$spouse
[1] ""

[[21]]$allegiances
[1] "House Lannister of Casterly Rock"

[[21]]$books
[[21]]$books[[1]]
[1] "A Game of Thrones"

[[21]]$books[[2]]
[1] "A Clash of Kings"


[[21]]$povBooks
[[21]]$povBooks[[1]]
[1] "A Storm of Swords"

[[21]]$povBooks[[2]]
[1] "A Feast for Crows"

[[21]]$povBooks[[3]]
[1] "A Dance with Dragons"


[[21]]$tvSeries
[[21]]$tvSeries[[1]]
[1] "Season 1"

[[21]]$tvSeries[[2]]
[1] "Season 2"

[[21]]$tvSeries[[3]]
[1] "Season 3"

[[21]]$tvSeries[[4]]
[1] "Season 4"

[[21]]$tvSeries[[5]]
[1] "Season 5"


[[21]]$playedBy
[1] "Nikolaj Coster-Waldau"


[[22]]
[[22]]$url
[1] "https://www.anapioficeandfire.com/api/characters/576"

[[22]]$id
[1] 576

[[22]]$name
[1] "Jon Connington"

[[22]]$gender
[1] "Male"

[[22]]$culture
[1] "Stormlands"

[[22]]$born
[1] "In or between 263 AC and 265 AC"

[[22]]$died
[1] ""

[[22]]$alive
[1] TRUE

[[22]]$titles
[[22]]$titles[[1]]
[1] "Lord of Griffin's Roost"

[[22]]$titles[[2]]
[1] "Hand of the King"

[[22]]$titles[[3]]
[1] "Hand of the True King"


[[22]]$aliases
[1] "Griffthe Mad King's Hand"

[[22]]$father
[1] ""

[[22]]$mother
[1] ""

[[22]]$spouse
[1] ""

[[22]]$allegiances
[[22]]$allegiances[[1]]
[1] "House Connington of Griffin's Roost"

[[22]]$allegiances[[2]]
[1] "House Targaryen of King's Landing"


[[22]]$books
[[22]]$books[[1]]
[1] "A Storm of Swords"

[[22]]$books[[2]]
[1] "A Feast for Crows"

[[22]]$books[[3]]
[1] "The World of Ice and Fire"


[[22]]$povBooks
[1] "A Dance with Dragons"

[[22]]$tvSeries
[1] ""

[[22]]$playedBy
[1] ""


[[23]]
[[23]]$url
[1] "https://www.anapioficeandfire.com/api/characters/583"

[[23]]$id
[1] 583

[[23]]$name
[1] "Jon Snow"

[[23]]$gender
[1] "Male"

[[23]]$culture
[1] "Northmen"

[[23]]$born
[1] "In 283 AC"

[[23]]$died
[1] ""

[[23]]$alive
[1] TRUE

[[23]]$titles
[1] "Lord Commander of the Night's Watch"

[[23]]$aliases
[[23]]$aliases[[1]]
[1] "Lord Snow"

[[23]]$aliases[[2]]
[1] "Ned Stark's Bastard"

[[23]]$aliases[[3]]
[1] "The Snow of Winterfell"

[[23]]$aliases[[4]]
[1] "The Crow-Come-Over"

[[23]]$aliases[[5]]
[1] "The 998th Lord Commander of the Night's Watch"

[[23]]$aliases[[6]]
[1] "The Bastard of Winterfell"

[[23]]$aliases[[7]]
[1] "The Black Bastard of the Wall"

[[23]]$aliases[[8]]
[1] "Lord Crow"


[[23]]$father
[1] ""

[[23]]$mother
[1] ""

[[23]]$spouse
[1] ""

[[23]]$allegiances
[1] "House Stark of Winterfell"

[[23]]$books
[1] "A Feast for Crows"

[[23]]$povBooks
[[23]]$povBooks[[1]]
[1] "A Game of Thrones"

[[23]]$povBooks[[2]]
[1] "A Clash of Kings"

[[23]]$povBooks[[3]]
[1] "A Storm of Swords"

[[23]]$povBooks[[4]]
[1] "A Dance with Dragons"


[[23]]$tvSeries
[[23]]$tvSeries[[1]]
[1] "Season 1"

[[23]]$tvSeries[[2]]
[1] "Season 2"

[[23]]$tvSeries[[3]]
[1] "Season 3"

[[23]]$tvSeries[[4]]
[1] "Season 4"

[[23]]$tvSeries[[5]]
[1] "Season 5"

[[23]]$tvSeries[[6]]
[1] "Season 6"


[[23]]$playedBy
[1] "Kit Harington"


[[24]]
[[24]]$url
[1] "https://www.anapioficeandfire.com/api/characters/60"

[[24]]$id
[1] 60

[[24]]$name
[1] "Aeron Greyjoy"

[[24]]$gender
[1] "Male"

[[24]]$culture
[1] "Ironborn"

[[24]]$born
[1] "In or between 269 AC and 273 AC, at Pyke"

[[24]]$died
[1] ""

[[24]]$alive
[1] TRUE

[[24]]$titles
[[24]]$titles[[1]]
[1] "Priest of the Drowned God"

[[24]]$titles[[2]]
[1] "Captain of the Golden Storm (formerly)"


[[24]]$aliases
[[24]]$aliases[[1]]
[1] "The Damphair"

[[24]]$aliases[[2]]
[1] "Aeron Damphair"


[[24]]$father
[1] ""

[[24]]$mother
[1] ""

[[24]]$spouse
[1] ""

[[24]]$allegiances
[1] "House Greyjoy of Pyke"

[[24]]$books
[[24]]$books[[1]]
[1] "A Game of Thrones"

[[24]]$books[[2]]
[1] "A Clash of Kings"

[[24]]$books[[3]]
[1] "A Storm of Swords"

[[24]]$books[[4]]
[1] "A Dance with Dragons"


[[24]]$povBooks
[1] "A Feast for Crows"

[[24]]$tvSeries
[1] "Season 6"

[[24]]$playedBy
[1] "Michael Feast"


[[25]]
[[25]]$url
[1] "https://www.anapioficeandfire.com/api/characters/605"

[[25]]$id
[1] 605

[[25]]$name
[1] "Kevan Lannister"

[[25]]$gender
[1] "Male"

[[25]]$culture
[1] ""

[[25]]$born
[1] "In 244 AC"

[[25]]$died
[1] "In 300 AC, at King's Landing"

[[25]]$alive
[1] FALSE

[[25]]$titles
[[25]]$titles[[1]]
[1] "Ser"

[[25]]$titles[[2]]
[1] "Master of laws"

[[25]]$titles[[3]]
[1] "Lord Regent"

[[25]]$titles[[4]]
[1] "Protector of the Realm"


[[25]]$aliases
[1] ""

[[25]]$father
[1] ""

[[25]]$mother
[1] ""

[[25]]$spouse
[1] "https://www.anapioficeandfire.com/api/characters/327"

[[25]]$allegiances
[1] "House Lannister of Casterly Rock"

[[25]]$books
[[25]]$books[[1]]
[1] "A Game of Thrones"

[[25]]$books[[2]]
[1] "A Clash of Kings"

[[25]]$books[[3]]
[1] "A Storm of Swords"

[[25]]$books[[4]]
[1] "A Feast for Crows"


[[25]]$povBooks
[1] "A Dance with Dragons"

[[25]]$tvSeries
[[25]]$tvSeries[[1]]
[1] "Season 1"

[[25]]$tvSeries[[2]]
[1] "Season 2"

[[25]]$tvSeries[[3]]
[1] "Season 5"

[[25]]$tvSeries[[4]]
[1] "Season 6"


[[25]]$playedBy
[1] "Ian Gelder"


[[26]]
[[26]]$url
[1] "https://www.anapioficeandfire.com/api/characters/743"

[[26]]$id
[1] 743

[[26]]$name
[1] "Melisandre"

[[26]]$gender
[1] "Female"

[[26]]$culture
[1] "Asshai"

[[26]]$born
[1] "At Unknown"

[[26]]$died
[1] ""

[[26]]$alive
[1] TRUE

[[26]]$titles
[1] ""

[[26]]$aliases
[[26]]$aliases[[1]]
[1] "The Red Priestess"

[[26]]$aliases[[2]]
[1] "The Red Woman"

[[26]]$aliases[[3]]
[1] "The King's Red Shadow"

[[26]]$aliases[[4]]
[1] "Lady Red"

[[26]]$aliases[[5]]
[1] "Lot Seven"


[[26]]$father
[1] ""

[[26]]$mother
[1] ""

[[26]]$spouse
[1] ""

[[26]]$allegiances
list()

[[26]]$books
[[26]]$books[[1]]
[1] "A Clash of Kings"

[[26]]$books[[2]]
[1] "A Storm of Swords"

[[26]]$books[[3]]
[1] "A Feast for Crows"


[[26]]$povBooks
[1] "A Dance with Dragons"

[[26]]$tvSeries
[[26]]$tvSeries[[1]]
[1] "Season 2"

[[26]]$tvSeries[[2]]
[1] "Season 3"

[[26]]$tvSeries[[3]]
[1] "Season 4"

[[26]]$tvSeries[[4]]
[1] "Season 5"

[[26]]$tvSeries[[5]]
[1] "Season 6"


[[26]]$playedBy
[1] "Carice van Houten"


[[27]]
[[27]]$url
[1] "https://www.anapioficeandfire.com/api/characters/751"

[[27]]$id
[1] 751

[[27]]$name
[1] "Merrett Frey"

[[27]]$gender
[1] "Male"

[[27]]$culture
[1] "Rivermen"

[[27]]$born
[1] "In 262 AC"

[[27]]$died
[1] "In 300 AC, at Near Oldstones"

[[27]]$alive
[1] FALSE

[[27]]$titles
[1] ""

[[27]]$aliases
[1] "Merrett Muttonhead"

[[27]]$father
[1] ""

[[27]]$mother
[1] ""

[[27]]$spouse
[1] "https://www.anapioficeandfire.com/api/characters/712"

[[27]]$allegiances
[1] "House Frey of the Crossing"

[[27]]$books
[[27]]$books[[1]]
[1] "A Game of Thrones"

[[27]]$books[[2]]
[1] "A Clash of Kings"

[[27]]$books[[3]]
[1] "A Feast for Crows"

[[27]]$books[[4]]
[1] "A Dance with Dragons"


[[27]]$povBooks
[1] "A Storm of Swords"

[[27]]$tvSeries
[1] ""

[[27]]$playedBy
[1] ""


[[28]]
[[28]]$url
[1] "https://www.anapioficeandfire.com/api/characters/844"

[[28]]$id
[1] 844

[[28]]$name
[1] "Quentyn Martell"

[[28]]$gender
[1] "Male"

[[28]]$culture
[1] "Dornish"

[[28]]$born
[1] "In 281 AC, at Sunspear, Dorne"

[[28]]$died
[1] "In 300 AC, at Meereen"

[[28]]$alive
[1] FALSE

[[28]]$titles
[1] "Prince"

[[28]]$aliases
[[28]]$aliases[[1]]
[1] "Frog"

[[28]]$aliases[[2]]
[1] "Prince Frog"

[[28]]$aliases[[3]]
[1] "The prince who came too late"

[[28]]$aliases[[4]]
[1] "The Dragonrider"


[[28]]$father
[1] ""

[[28]]$mother
[1] ""

[[28]]$spouse
[1] ""

[[28]]$allegiances
[1] "House Nymeros Martell of Sunspear"

[[28]]$books
[[28]]$books[[1]]
[1] "A Game of Thrones"

[[28]]$books[[2]]
[1] "A Clash of Kings"

[[28]]$books[[3]]
[1] "A Storm of Swords"

[[28]]$books[[4]]
[1] "A Feast for Crows"


[[28]]$povBooks
[1] "A Dance with Dragons"

[[28]]$tvSeries
[1] ""

[[28]]$playedBy
[1] ""


[[29]]
[[29]]$url
[1] "https://www.anapioficeandfire.com/api/characters/954"

[[29]]$id
[1] 954

[[29]]$name
[1] "Samwell Tarly"

[[29]]$gender
[1] "Male"

[[29]]$culture
[1] "Andal"

[[29]]$born
[1] "In 283 AC, at Horn Hill"

[[29]]$died
[1] ""

[[29]]$alive
[1] TRUE

[[29]]$titles
[1] ""

[[29]]$aliases
[[29]]$aliases[[1]]
[1] "Sam"

[[29]]$aliases[[2]]
[1] "Ser Piggy"

[[29]]$aliases[[3]]
[1] "Prince Pork-chop"

[[29]]$aliases[[4]]
[1] "Lady Piggy"

[[29]]$aliases[[5]]
[1] "Sam the Slayer"

[[29]]$aliases[[6]]
[1] "Black Sam"

[[29]]$aliases[[7]]
[1] "Lord of Ham"


[[29]]$father
[1] ""

[[29]]$mother
[1] ""

[[29]]$spouse
[1] ""

[[29]]$allegiances
[1] "House Tarly of Horn Hill"

[[29]]$books
[[29]]$books[[1]]
[1] "A Game of Thrones"

[[29]]$books[[2]]
[1] "A Clash of Kings"

[[29]]$books[[3]]
[1] "A Dance with Dragons"


[[29]]$povBooks
[[29]]$povBooks[[1]]
[1] "A Storm of Swords"

[[29]]$povBooks[[2]]
[1] "A Feast for Crows"


[[29]]$tvSeries
[[29]]$tvSeries[[1]]
[1] "Season 1"

[[29]]$tvSeries[[2]]
[1] "Season 2"

[[29]]$tvSeries[[3]]
[1] "Season 3"

[[29]]$tvSeries[[4]]
[1] "Season 4"

[[29]]$tvSeries[[5]]
[1] "Season 5"

[[29]]$tvSeries[[6]]
[1] "Season 6"


[[29]]$playedBy
[1] "John Bradley-West"


[[30]]
[[30]]$url
[1] "https://www.anapioficeandfire.com/api/characters/957"

[[30]]$id
[1] 957

[[30]]$name
[1] "Sansa Stark"

[[30]]$gender
[1] "Female"

[[30]]$culture
[1] "Northmen"

[[30]]$born
[1] "In 286 AC, at Winterfell"

[[30]]$died
[1] ""

[[30]]$alive
[1] TRUE

[[30]]$titles
[1] "Princess"

[[30]]$aliases
[[30]]$aliases[[1]]
[1] "Little bird"

[[30]]$aliases[[2]]
[1] "Alayne Stone"

[[30]]$aliases[[3]]
[1] "Jonquil"


[[30]]$father
[1] ""

[[30]]$mother
[1] ""

[[30]]$spouse
[1] "https://www.anapioficeandfire.com/api/characters/1052"

[[30]]$allegiances
[[30]]$allegiances[[1]]
[1] "House Baelish of Harrenhal"

[[30]]$allegiances[[2]]
[1] "House Stark of Winterfell"


[[30]]$books
[1] "A Dance with Dragons"

[[30]]$povBooks
[[30]]$povBooks[[1]]
[1] "A Game of Thrones"

[[30]]$povBooks[[2]]
[1] "A Clash of Kings"

[[30]]$povBooks[[3]]
[1] "A Storm of Swords"

[[30]]$povBooks[[4]]
[1] "A Feast for Crows"


[[30]]$tvSeries
[[30]]$tvSeries[[1]]
[1] "Season 1"

[[30]]$tvSeries[[2]]
[1] "Season 2"

[[30]]$tvSeries[[3]]
[1] "Season 3"

[[30]]$tvSeries[[4]]
[1] "Season 4"

[[30]]$tvSeries[[5]]
[1] "Season 5"

[[30]]$tvSeries[[6]]
[1] "Season 6"


[[30]]$playedBy
[1] "Sophie Turner"

JSON Databases

JSON databases are an increasingly popular alternative to the relational database model based around SQL
Mongo, NoSQL
Do not require database schemas
Very flexible and scalable
Downsides:
- No Joins
- No ACID (Atomicity, Consistency, Isolation, Durability)

Webscraping

Webscraping is the process of extracting data from websites
We will use rvest package which pulls HTML from websites and lets you process it
Webscraping can be very messy, powerful but maybe a last resort
Alternative: APIs

HTML

Webpages are usually HTML files:

<html>
<head>
  <title>Page title</title>
</head>
<body>
  <h1 id='first'>A heading</h1>
  <p>Some text &amp; <b>some bold text.</b></p>
  <img src='myimg.png' width='100' height='100'>
</body>

Nested “tags”: body, head, h1, p, b
Attributes: id, src, width, height

Ethics: OK Cupid Case Study

Researchers in Denmark scraped dating site OkCupid and published a dataset

vox

Ethics: OK Cupid Case Study

Researchers in Denmark scraped dating site OkCupid and published a dataset

vox

Ethics: OK Cupid Case Study

Researchers in Denmark scraped dating site OkCupid and published a dataset

vox

Cambridge Analytica

Cambridge Analytica harvested data from FB users using questionaires/quizzes
Built psychological profiles of users and their friends
Used data for targetting in political campaigns
- Resulted in Cambridge Analytica bankruptcy
- FB paid billions in fines, lost market cap

Cambridge Analytica

ftc

Guidelines:

Just because you can, doesn’t mean you should
Red Flags:
- Personal Data
- Data you think is only available accidentally
Social Science Research Study Ethics:
1. Did participants consent?
2. Is privacy maintained?
3. Is there the potential for harm?

`robotstxt`

How to know if you can scrape a website?

library(robotstxt)
paths_allowed("http://www.facebook.com")

[1] FALSE

paths_allowed("http://www.wikipedia.com")

[1] TRUE

Meetup Reflection/One Minute Paper

Please fill out the following google form after the meeting or watching the video:

Click Here

Meetup 9: Hierarchical Data and Webscraping

Lab 6: Color Scheme

Lab 6: : Color Scheme

Lab 6: Color Scheme

Lab 6 Review: Principle of Proportional Ink

Lab 6 Review

Lab 6 Review: Keys

Lab 6: Primary Keys

Week Ahead: Webscraping and Hierarchical Data

Hierarchical Data

Hierarchical Data

List data type

List can contain lists

Rectangling

unnest_wider

unnest_longer

GOT Case Study

Rectangling GOT

Rectangling GOT

Create povBooks table

Create povBooks table

JSON Files

JSON Files

jsonlite

JSON Databases

Webscraping

HTML

Ethics: OK Cupid Case Study

Ethics: OK Cupid Case Study

Ethics: OK Cupid Case Study

Cambridge Analytica

Cambridge Analytica

Guidelines:

robotstxt

Meetup Reflection/One Minute Paper

`unnest_wider`

`unnest_longer`

Create `povBooks` table

Create `povBooks` table

`jsonlite`

`robotstxt`