Introduction to Data Science

Exam1 Python Commands Cheat Sheet

import numpy as np
import pandas as pd

import matplotlib.pyplot as plt
import plotly.express as px
from plotly.subplots import make_subplots
import plotly.io as pio
pio.renderers.default = 'notebook_connected'

Loading Data using Pandas:

• pd.read_csv()
• pd.read_excel()
• pd.read_json()
• pd.read_table()

Inside the parenthesis you put the exact name/location of the data.

Code Examples

DF = pd.read_csv('mapdataall.csv')

file_location = 'https://joannabieri.com/introdatascience/data/starwars.csv'
DF = pd.read_csv(file_location)

Data Frame Basics

• .shape - find the rows and columns
• .columns - get the column names
• .describe - do some basic statistics
• display() - print the data frame in a nice table

Assume our data is stored in the variable DF.

Code Examples

DF.shape

(87, 14)

DF.columns

Index(['name', 'height', 'mass', 'hair_color', 'skin_color', 'eye_color',
       'birth_year', 'sex', 'gender', 'homeworld', 'species', 'films',
       'vehicles', 'starships'],
      dtype='object')

DF.describe()

	height	mass	birth_year
count	81.000000	59.000000	43.000000
mean	174.604938	97.311864	87.565116
std	34.774157	169.457163	154.691439
min	66.000000	15.000000	8.000000
25%	167.000000	55.600000	35.000000
50%	180.000000	79.000000	52.000000
75%	191.000000	84.500000	72.000000
max	264.000000	1358.000000	896.000000

display(DF)

	name	height	mass	hair_color	skin_color	eye_color	birth_year	sex	gender	homeworld	species	films	vehicles	starships
0	Luke Skywalker	172.0	77.0	blond	fair	blue	19.0	male	masculine	Tatooine	Human	A New Hope, The Empire Strikes Back, Return of...	Snowspeeder, Imperial Speeder Bike	X-wing, Imperial shuttle
1	C-3PO	167.0	75.0	NaN	gold	yellow	112.0	none	masculine	Tatooine	Droid	A New Hope, The Empire Strikes Back, Return of...	NaN	NaN
2	R2-D2	96.0	32.0	NaN	white, blue	red	33.0	none	masculine	Naboo	Droid	A New Hope, The Empire Strikes Back, Return of...	NaN	NaN
3	Darth Vader	202.0	136.0	none	white	yellow	41.9	male	masculine	Tatooine	Human	A New Hope, The Empire Strikes Back, Return of...	NaN	TIE Advanced x1
4	Leia Organa	150.0	49.0	brown	light	brown	19.0	female	feminine	Alderaan	Human	A New Hope, The Empire Strikes Back, Return of...	Imperial Speeder Bike	NaN
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
82	Finn	NaN	NaN	black	dark	dark	NaN	male	masculine	NaN	Human	The Force Awakens	NaN	NaN
83	Rey	NaN	NaN	brown	light	hazel	NaN	female	feminine	NaN	Human	The Force Awakens	NaN	NaN
84	Poe Dameron	NaN	NaN	brown	light	brown	NaN	male	masculine	NaN	Human	The Force Awakens	NaN	X-wing
85	BB8	NaN	NaN	none	none	black	NaN	none	masculine	NaN	Droid	The Force Awakens	NaN	NaN
86	Captain Phasma	NaN	NaN	none	none	unknown	NaN	female	feminine	NaN	Human	The Force Awakens	NaN	NaN

87 rows × 14 columns

Selecting Columns

• Choose the columns you care about
• Select those columns using DF[ ]
• The list of columns goes between the square brackets

Code Examples

DF['name']

0     Luke Skywalker
1              C-3PO
2              R2-D2
3        Darth Vader
4        Leia Organa
           ...      
82              Finn
83               Rey
84       Poe Dameron
85               BB8
86    Captain Phasma
Name: name, Length: 87, dtype: object

my_columns = ['name','hair_color', 'skin_color', 'eye_color']
DF[my_columns]

	name	hair_color	skin_color	eye_color
0	Luke Skywalker	blond	fair	blue
1	C-3PO	NaN	gold	yellow
2	R2-D2	NaN	white, blue	red
3	Darth Vader	none	white	yellow
4	Leia Organa	brown	light	brown
...	...	...	...	...
82	Finn	black	dark	dark
83	Rey	brown	light	hazel
84	Poe Dameron	brown	light	brown
85	BB8	none	none	black
86	Captain Phasma	none	none	unknown

87 rows × 4 columns

Plots - Plotly Express

• px.histogram()
• px.scatter()
• px.box()
• px.density_heatmap()
• px.bar()

All plots have three basic parts:

---

• Define the Basic Data and Plot Type

fig = px.histogram(DF,
                    x='column name')

---

• Add some fancy stuff

fig.update_layout(title=' ',
                  title_x=0.5,
                  xaxis_title=" ",
                  yaxis_title=" ",
                  autosize=False,
                  width=800,
                  height=500)

---

• Show the figure

fig.show()

For the examples below we are using the Starwars Data

display(DF)

	name	height	mass	hair_color	skin_color	eye_color	birth_year	sex	gender	homeworld	species	films	vehicles	starships
0	Luke Skywalker	172.0	77.0	blond	fair	blue	19.0	male	masculine	Tatooine	Human	A New Hope, The Empire Strikes Back, Return of...	Snowspeeder, Imperial Speeder Bike	X-wing, Imperial shuttle
1	C-3PO	167.0	75.0	NaN	gold	yellow	112.0	none	masculine	Tatooine	Droid	A New Hope, The Empire Strikes Back, Return of...	NaN	NaN
2	R2-D2	96.0	32.0	NaN	white, blue	red	33.0	none	masculine	Naboo	Droid	A New Hope, The Empire Strikes Back, Return of...	NaN	NaN
3	Darth Vader	202.0	136.0	none	white	yellow	41.9	male	masculine	Tatooine	Human	A New Hope, The Empire Strikes Back, Return of...	NaN	TIE Advanced x1
4	Leia Organa	150.0	49.0	brown	light	brown	19.0	female	feminine	Alderaan	Human	A New Hope, The Empire Strikes Back, Return of...	Imperial Speeder Bike	NaN
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
82	Finn	NaN	NaN	black	dark	dark	NaN	male	masculine	NaN	Human	The Force Awakens	NaN	NaN
83	Rey	NaN	NaN	brown	light	hazel	NaN	female	feminine	NaN	Human	The Force Awakens	NaN	NaN
84	Poe Dameron	NaN	NaN	brown	light	brown	NaN	male	masculine	NaN	Human	The Force Awakens	NaN	X-wing
85	BB8	NaN	NaN	none	none	black	NaN	none	masculine	NaN	Droid	The Force Awakens	NaN	NaN
86	Captain Phasma	NaN	NaN	none	none	unknown	NaN	female	feminine	NaN	Human	The Force Awakens	NaN	NaN

87 rows × 14 columns

Code Examples

fig = px.scatter(DF,
                 x='height',
                 y='mass',
                 hover_data='name',
                 color = 'sex')

fig.update_layout(title='Mass vs. Height of Starwars Characters',
                  title_x=0.5,
                  xaxis_title="Height",
                  yaxis_title="Mass",
                  autosize=False,
                  width=800,
                  height=500)

fig.show()

fig = px.histogram(DF,
                 x='height',
                 nbins=10,
                 color = 'sex')

fig.update_layout(bargap=0.02,
                  title='Height of Starwars Characters',
                  title_x=0.5,
                  yaxis_title="Frequency",
                  xaxis_title="Height",
                  autosize=False,
                  width=800,
                  height=500)

fig.show()

fig = px.histogram(DF,
                 x='height',
                 nbins=10,
                 facet_col= 'sex',
                 facet_col_wrap=2,
                  color='eye_color')

fig.update_layout(bargap=0.02,
                  title='Height of Starwars Characters',
                  title_x=0.5,
                                    autosize=False,
                  width=800,
                  height=800)

fig.show()

Categorical Data - value_counts and unique entries

• DF[[columns of interest]].value_counts()
• DF['column of interest'].unique() - one column at a time
• DF[[columns of interest]].drop_duplicates()

All of these commands will give you information about how many categories are in a column (or columns).

Code Examples

DF['sex'].value_counts()

sex
male              60
female            16
none               6
hermaphroditic     1
Name: count, dtype: int64

DF['sex'].drop_duplicates()

0               male
1               none
4             female
15    hermaphroditic
17               NaN
Name: sex, dtype: object

my_columns = ['sex','eye_color']
DF[my_columns].drop_duplicates()

	sex	eye_color
0	male	blue
1	none	yellow
2	none	red
3	male	yellow
4	female	brown
6	female	blue
8	male	brown
9	male	blue-gray
14	male	black
15	hermaphroditic	orange
16	male	hazel
17	NaN	blue
22	male	red
25	male	orange
43	male	pink
44	female	hazel
45	male	unknown
58	NaN	brown
68	female	yellow
71	female	black
73	none	red, blue
75	male	gold
77	male	green, yellow
80	NaN	white
82	male	dark
85	none	black
86	female	unknown

Sorting

• DF.sort_values(by='sorting column name')
• DF.sort_values(by='sorting column name',ascending=False)

You can technically leave out the by= command, but this sometimes helps me to understand the command. If you wanted to send in a list of columns, it can sort more than one column at a time. It will sort in the order the columns are given.

Code Examples

DF.sort_values(by='height',ascending=False)

	name	height	mass	hair_color	skin_color	eye_color	birth_year	sex	gender	homeworld	species	films	vehicles	starships
55	Yarael Poof	264.0	NaN	none	white	yellow	NaN	male	masculine	Quermia	Quermian	The Phantom Menace	NaN	NaN
78	Tarfful	234.0	136.0	brown	brown	blue	NaN	male	masculine	Kashyyyk	Wookiee	Revenge of the Sith	NaN	NaN
70	Lama Su	229.0	88.0	none	grey	black	NaN	male	masculine	Kamino	Kaminoan	Attack of the Clones	NaN	NaN
12	Chewbacca	228.0	112.0	brown	unknown	blue	200.0	male	masculine	Kashyyyk	Wookiee	A New Hope, The Empire Strikes Back, Return of...	AT-ST	Millennium Falcon, Imperial shuttle
35	Roos Tarpals	224.0	82.0	none	grey	orange	NaN	male	masculine	Naboo	Gungan	The Phantom Menace	NaN	NaN
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
82	Finn	NaN	NaN	black	dark	dark	NaN	male	masculine	NaN	Human	The Force Awakens	NaN	NaN
83	Rey	NaN	NaN	brown	light	hazel	NaN	female	feminine	NaN	Human	The Force Awakens	NaN	NaN
84	Poe Dameron	NaN	NaN	brown	light	brown	NaN	male	masculine	NaN	Human	The Force Awakens	NaN	X-wing
85	BB8	NaN	NaN	none	none	black	NaN	none	masculine	NaN	Droid	The Force Awakens	NaN	NaN
86	Captain Phasma	NaN	NaN	none	none	unknown	NaN	female	feminine	NaN	Human	The Force Awakens	NaN	NaN

87 rows × 14 columns

Boolean Masks

Basic Operators

Operator	Definition
<	less than
>	greater than
<=	less than or equal to
>=	greater than or equal to
==	exactly equal to
!=	not equal to

Advanced Operators

Operator	Definition
and	check if two things are both true
or	check if one of two things is true
in	checks if something is in another thing
not	checks if something is false

Operators that Combine Masks

Operator	Definition
&	checks that two masks are both true (AND)
|	checks if one of the two masks is true (OR)

when masking more than one condition, separate the conditions with parenthesis:

• mask = (condition 1) & (condition 2) - both must be true
• mask = (condition 1) | (condition 2) - at least one must be true
• DF[mask]

Code Examples

mask = DF['homeworld'] != 'Tatooine'
DF[mask]

	name	height	mass	hair_color	skin_color	eye_color	birth_year	sex	gender	homeworld	species	films	vehicles	starships
2	R2-D2	96.0	32.0	NaN	white, blue	red	33.0	none	masculine	Naboo	Droid	A New Hope, The Empire Strikes Back, Return of...	NaN	NaN
4	Leia Organa	150.0	49.0	brown	light	brown	19.0	female	feminine	Alderaan	Human	A New Hope, The Empire Strikes Back, Return of...	Imperial Speeder Bike	NaN
9	Obi-Wan Kenobi	182.0	77.0	auburn, white	fair	blue-gray	57.0	male	masculine	Stewjon	Human	A New Hope, The Empire Strikes Back, Return of...	Tribubble bongo	Jedi starfighter, Trade Federation cruiser, Na...
11	Wilhuff Tarkin	180.0	NaN	auburn, grey	fair	blue	64.0	male	masculine	Eriadu	Human	A New Hope, Revenge of the Sith	NaN	NaN
12	Chewbacca	228.0	112.0	brown	unknown	blue	200.0	male	masculine	Kashyyyk	Wookiee	A New Hope, The Empire Strikes Back, Return of...	AT-ST	Millennium Falcon, Imperial shuttle
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
82	Finn	NaN	NaN	black	dark	dark	NaN	male	masculine	NaN	Human	The Force Awakens	NaN	NaN
83	Rey	NaN	NaN	brown	light	hazel	NaN	female	feminine	NaN	Human	The Force Awakens	NaN	NaN
84	Poe Dameron	NaN	NaN	brown	light	brown	NaN	male	masculine	NaN	Human	The Force Awakens	NaN	X-wing
85	BB8	NaN	NaN	none	none	black	NaN	none	masculine	NaN	Droid	The Force Awakens	NaN	NaN
86	Captain Phasma	NaN	NaN	none	none	unknown	NaN	female	feminine	NaN	Human	The Force Awakens	NaN	NaN

77 rows × 14 columns

mask = DF['homeworld'] == 'Tatooine'
DF[mask]

	name	height	mass	hair_color	skin_color	eye_color	birth_year	sex	gender	homeworld	species	films	vehicles	starships
0	Luke Skywalker	172.0	77.0	blond	fair	blue	19.0	male	masculine	Tatooine	Human	A New Hope, The Empire Strikes Back, Return of...	Snowspeeder, Imperial Speeder Bike	X-wing, Imperial shuttle
1	C-3PO	167.0	75.0	NaN	gold	yellow	112.0	none	masculine	Tatooine	Droid	A New Hope, The Empire Strikes Back, Return of...	NaN	NaN
3	Darth Vader	202.0	136.0	none	white	yellow	41.9	male	masculine	Tatooine	Human	A New Hope, The Empire Strikes Back, Return of...	NaN	TIE Advanced x1
5	Owen Lars	178.0	120.0	brown, grey	light	blue	52.0	male	masculine	Tatooine	Human	A New Hope, Attack of the Clones, Revenge of t...	NaN	NaN
6	Beru Whitesun Lars	165.0	75.0	brown	light	blue	47.0	female	feminine	Tatooine	Human	A New Hope, Attack of the Clones, Revenge of t...	NaN	NaN
7	R5-D4	97.0	32.0	NaN	white, red	red	NaN	none	masculine	Tatooine	Droid	A New Hope	NaN	NaN
8	Biggs Darklighter	183.0	84.0	black	light	brown	24.0	male	masculine	Tatooine	Human	A New Hope	NaN	X-wing
10	Anakin Skywalker	188.0	84.0	blond	fair	blue	41.9	male	masculine	Tatooine	Human	The Phantom Menace, Attack of the Clones, Reve...	Zephyr-G swoop bike, XJ-6 airspeeder	Naboo fighter, Trade Federation cruiser, Jedi ...
41	Shmi Skywalker	163.0	NaN	black	fair	brown	72.0	female	feminine	Tatooine	Human	The Phantom Menace, Attack of the Clones	NaN	NaN
60	Cliegg Lars	183.0	NaN	brown	fair	blue	82.0	male	masculine	Tatooine	Human	Attack of the Clones	NaN	NaN

mask = DF['height'] > 100
DF_tall = DF[mask].copy()

# A character can only have one homeworld, so we need OR (|) here, not AND (&)
mask = (DF['homeworld'] == 'Tatooine') | (DF['homeworld'] == 'Naboo') | (DF['homeworld'] == 'Alderaan')
DF_homeworld = DF[mask].copy()
DF_homeworld

	name	height	mass	hair_color	skin_color	eye_color	birth_year	sex	gender	homeworld	species	films	vehicles	starships
0	Luke Skywalker	172.0	77.0	blond	fair	blue	19.0	male	masculine	Tatooine	Human	A New Hope, The Empire Strikes Back, Return of...	Snowspeeder, Imperial Speeder Bike	X-wing, Imperial shuttle
1	C-3PO	167.0	75.0	NaN	gold	yellow	112.0	none	masculine	Tatooine	Droid	A New Hope, The Empire Strikes Back, Return of...	NaN	NaN
2	R2-D2	96.0	32.0	NaN	white, blue	red	33.0	none	masculine	Naboo	Droid	A New Hope, The Empire Strikes Back, Return of...	NaN	NaN
3	Darth Vader	202.0	136.0	none	white	yellow	41.9	male	masculine	Tatooine	Human	A New Hope, The Empire Strikes Back, Return of...	NaN	TIE Advanced x1
4	Leia Organa	150.0	49.0	brown	light	brown	19.0	female	feminine	Alderaan	Human	A New Hope, The Empire Strikes Back, Return of...	Imperial Speeder Bike	NaN
5	Owen Lars	178.0	120.0	brown, grey	light	blue	52.0	male	masculine	Tatooine	Human	A New Hope, Attack of the Clones, Revenge of t...	NaN	NaN
6	Beru Whitesun Lars	165.0	75.0	brown	light	blue	47.0	female	feminine	Tatooine	Human	A New Hope, Attack of the Clones, Revenge of t...	NaN	NaN
7	R5-D4	97.0	32.0	NaN	white, red	red	NaN	none	masculine	Tatooine	Droid	A New Hope	NaN	NaN
8	Biggs Darklighter	183.0	84.0	black	light	brown	24.0	male	masculine	Tatooine	Human	A New Hope	NaN	X-wing
10	Anakin Skywalker	188.0	84.0	blond	fair	blue	41.9	male	masculine	Tatooine	Human	The Phantom Menace, Attack of the Clones, Reve...	Zephyr-G swoop bike, XJ-6 airspeeder	Naboo fighter, Trade Federation cruiser, Jedi ...
19	Palpatine	170.0	75.0	grey	pale	yellow	82.0	male	masculine	Naboo	Human	The Empire Strikes Back, Return of the Jedi, T...	NaN	NaN
33	Padmé Amidala	185.0	45.0	brown	light	brown	46.0	female	feminine	Naboo	Human	The Phantom Menace, Attack of the Clones, Reve...	NaN	Naboo fighter, H-type Nubian yacht, Naboo star...
34	Jar Jar Binks	196.0	66.0	none	orange	orange	52.0	male	masculine	Naboo	Gungan	The Phantom Menace, Attack of the Clones	NaN	NaN
35	Roos Tarpals	224.0	82.0	none	grey	orange	NaN	male	masculine	Naboo	Gungan	The Phantom Menace	NaN	NaN
36	Rugor Nass	206.0	NaN	none	green	orange	NaN	male	masculine	Naboo	Gungan	The Phantom Menace	NaN	NaN
37	Ric Olié	183.0	NaN	brown	fair	blue	NaN	male	masculine	Naboo	Human	The Phantom Menace	NaN	Naboo Royal Starship
40	Quarsh Panaka	183.0	NaN	black	dark	brown	62.0	male	masculine	Naboo	Human	The Phantom Menace	NaN	NaN
41	Shmi Skywalker	163.0	NaN	black	fair	brown	72.0	female	feminine	Tatooine	Human	The Phantom Menace, Attack of the Clones	NaN	NaN
58	Gregar Typho	185.0	85.0	black	dark	brown	NaN	NaN	NaN	Naboo	NaN	Attack of the Clones	NaN	Naboo fighter
59	Cordé	157.0	NaN	brown	light	brown	NaN	NaN	NaN	Naboo	NaN	Attack of the Clones	NaN	NaN
60	Cliegg Lars	183.0	NaN	brown	fair	blue	82.0	male	masculine	Tatooine	Human	Attack of the Clones	NaN	NaN
64	Dormé	165.0	NaN	brown	light	brown	NaN	female	feminine	Naboo	Human	Attack of the Clones	NaN	NaN
66	Bail Prestor Organa	191.0	NaN	black	tan	brown	67.0	male	masculine	Alderaan	Human	Attack of the Clones, Revenge of the Sith	NaN	NaN
79	Raymus Antilles	188.0	79.0	brown	light	brown	NaN	male	masculine	Alderaan	Human	A New Hope, Revenge of the Sith	NaN	NaN

Adding a Column to a Data Frame

To add a column you just enter a new column name into DF[] followed by a calculation

• DF['new_column_name'] = enter your calculation here

Code Examples

DF['height_and_mass'] = DF['height'] + DF['mass']

DF['is_human_female'] = (DF['species']=='Human') & (DF['sex']=='female')

Grouping Data by a Categorical Column

• .groupby('column name').function()

The groupby command just separates the data in Pythons memory. Then you apply a function to see the results. In this class we have been doing

my_columns = [list of columns] # must include the column you are grouping by
DF[my_columns].groupby('grouping column').describe()

You can use all sorts of functions at the end:

• .min()
• .max()
• .mean()
• .median()
• .sum()
• .prod()
• .count()
• .describe()

if you just want to see one group you can use

• .get_group()

Code Examples

my_columns = ['sex','height']
DF[my_columns].groupby('sex').describe()

	height
	count	mean	std	min	25%	50%	75%	max
sex
female	14.0	171.571429	15.911845	150.0	165.0	167.5	178.0	213.0
hermaphroditic	1.0	175.000000	NaN	175.0	175.0	175.0	175.0	175.0
male	57.0	179.122807	36.009458	66.0	173.0	183.0	193.0	264.0
none	5.0	131.200000	49.149771	96.0	96.0	97.0	167.0	200.0

my_columns = ['sex','height']
DF[my_columns].groupby('sex').get_group('female')

	sex	height
4	female	150.0
6	female	165.0
26	female	150.0
33	female	185.0
41	female	163.0
44	female	178.0
53	female	184.0
62	female	170.0
63	female	166.0
64	female	165.0
68	female	168.0
71	female	213.0
72	female	167.0
76	female	178.0
83	female	NaN
86	female	NaN

my_columns = ['sex','height','mass']
DF[my_columns].groupby('sex').mean()

	height	mass
sex
female	171.571429	54.688889
hermaphroditic	175.000000	1358.000000
male	179.122807	80.219048
none	131.200000	69.750000

Joining Data Frames

If you have two or more data frames that share a column you can join them together

• pd.merge(DataFrame1, DataFrame2, on='shared column', how=???)
  • how='left' - DataFrame1 observations are used to set the shared column in the new DataFrame
  • how='right' - DataFrame2 observations are used to set the shared column in the new DataFrame
  • how='outer' - All possible observations are used to set the shared column in the new DataFrame
  • how='inner' - Only observations that are in BOTH DataFrames are used to set the shared column in the new DataFrame

Code Examples - using fake data

DF_fake1 = pd.DataFrame({'id':[1,2,3],'data1':['x1','x2','x3']})
DF_fake2 = pd.DataFrame({'id':[1,2,4],'data1':['y1','y2','y4']})

display(DF_fake1)
display(DF_fake2)

	id	data1
0	1	x1
1	2	x2
2	3	x3

	id	data1
0	1	y1
1	2	y2
2	4	y4

pd.merge(DF_fake1, DF_fake2, on='id',how='left')

	id	data1_x	data1_y
0	1	x1	y1
1	2	x2	y2
2	3	x3	NaN

pd.merge(DF_fake1, DF_fake2, on='id',how='right')

	id	data1_x	data1_y
0	1	x1	y1
1	2	x2	y2
2	4	NaN	y4

pd.merge(DF_fake1, DF_fake2, on='id',how='outer')

	id	data1_x	data1_y
0	1	x1	y1
1	2	x2	y2
2	3	x3	NaN
3	4	NaN	y4

pd.merge(DF_fake1, DF_fake2, on='id',how='inner')

	id	data1_x	data1_y
0	1	x1	y1
1	2	x2	y2

Using .apply and lambda x: to fix your data

• DF['column name'].apply(lambda x: operation on x)
  • The .apply() command lets you apply a function to every row along your data frame.
  • lambda is a special function you first write the word lambda and choose a variable followed by a colon.

In the example below we are going to create a column in our data frame that counts the total number of films that a Starwars character has been in

Code Examples

DF['number_films'] = DF['films'].apply(lambda x: len(x.split(',')))
''' 
Why does this work???

Here we are looking at each piece of data in the films column.
The data looks like this: 
"A New Hope, The Empire Strikes Back, Return of the Jedi, Revenge of the Sith"

When we do x.split(',') this returns a list: 
["A New Hope", "The Empire Strikes Back", "Return of the Jedi", "Revenge of the Sith"]

We find the length of this list to count up the movies!

'''
DF[['name','number_films']]

	name	number_films
0	Luke Skywalker	5
1	C-3PO	6
2	R2-D2	7
3	Darth Vader	4
4	Leia Organa	5
...	...	...
82	Finn	1
83	Rey	1
84	Poe Dameron	1
85	BB8	1
86	Captain Phasma	1

87 rows × 2 columns

DF['weight'] = DF['mass'].apply(lambda x: x*2.20462)

Renaming Columns

• DF.rename(columns={'old name':'new name'}, inplace=True)

List the old name first, then a colon, then the new name. You can rename as many columns as you want at once.

Code Examples - using fake data

DF_grades = pd.DataFrame({'Student Name':['Ana','Ben'],'Exam Score':[92,70]})
DF_grades.rename(columns={'Student Name':'name','Exam Score':'score'}, inplace=True)
DF_grades

	name	score
0	Ana	92
1	Ben	70

Melt and Pivot - Rearranging Data

• pd.melt(DF, id_vars=[columns to keep], var_name='name for the old column names', value_name='name for the values') - makes your data longer
• DF.pivot(index='rows column', columns='new columns column', values='values column') - makes your data wider

Code Examples - using fake data

DF_wide = pd.DataFrame({'name':['Ana','Ben'],'math':[85,90],'sci':[70,82]})
display(DF_wide)
DF_long = pd.melt(DF_wide, id_vars=['name'], var_name='subject', value_name='score')
DF_long

	name	math	sci
0	Ana	85	70
1	Ben	90	82

	name	subject	score
0	Ana	math	85
1	Ben	math	90
2	Ana	sci	70
3	Ben	sci	82

DF_long.pivot(index='name', columns='subject', values='score')

subject	math	sci
name
Ana	85	70
Ben	90	82

Filling in Missing Values

• DF.ffill() - fills each NaN with the last real value above it

Code Examples - using fake data

DF_brand = pd.DataFrame({'brand':['Brand 1',np.nan,np.nan,'Brand 2',np.nan]})
DF_brand.ffill()

	brand
0	Brand 1
1	Brand 1
2	Brand 1
3	Brand 2
4	Brand 2