6. Tidying Data (pd.melt())

Tidying data for analysis

• Principles of tidy data
  • Columns represent separate variables
  • Rows represent individual observations
  • Observational units form tables

Melting

• Turn columns into rows

airquality_melt = pd.melt(frame=airquality, id_vars='Date', var_name='measurement', value_name='reading') # id_vars is the column that do not melt

print(airquality_melt.head())

import pandas as pd

gapminder_melt = pd.melt(frame=gapminder, id_vars="Life expectancy") # Melt gapminder

gapminder_melt.columns = ['country','year','life_expectancy'] # Rename the columns

print(gapminder_melt.head())

Melting and Parsing

tb_melt = pd.melt(frame=tb, id_vars=['country', 'year']) # Melt tb

tb_melt['gender'] = tb_melt.variable.str[0] # Create the 'gender' column

tb_melt['age_group'] = tb_melt.variable.str[1:] # Create the 'age_group' column

print(tb_melt.head())

ebola_melt = pd.melt(ebola, id_vars=['Date', 'Day'], var_name='type_country', value_name='counts') # Melt ebola

ebola_melt['str_split'] = ebola_melt.type_country.str.split("_") # Create the 'str_split' column

ebola_melt['type'] = ebola_melt.str_split.str.get(0) # Create the 'type' column

ebola_melt['country'] = ebola_melt.str_split.str.get(1) # Create the 'country' column

print(ebola_melt.head())

String Parsing with Regular Expressions

import re # Import the regular expression module

prog = re.compile('\d{3}-\d{3}-\d{4}') # Compile the pattern: prog

result = prog.match('123-456-7890') # See if the pattern matches

print(bool(result))

result2 = prog.match('1123-456-7890') # See if the pattern matches

print(bool(result2))