{ "cells": [ { "cell_type": "markdown", "id": "a9215f5c-525b-4508-8b0d-700f55cbd252", "metadata": {}, "source": [ "## Introduction to Data Science\n", "\n", "#### University of Redlands - DATA 101\n", "#### Prof: Joanna Bieri [joanna_bieri@redlands.edu](mailto:joanna_bieri@redlands.edu)\n", "#### [Class Website: data101.joannabieri.com](https://data101.joannabieri.com)\n", "\n", "---------------------------------------\n", "# Homework Day 7\n", "---------------------------------------\n", "\n", "GOALS:\n", "\n", "1. Answer all the questions from the lecture - we will use the scientists data and the grocery data.\n", "\n", "----------------------------------------------------------\n", "\n", "This homework has **9 Questions**.\n", "\n", "The Practice Exam for Exam 1 is a separate notebook this time, Day7-PracticeExam.ipynb. Download it from the class website. It is not graded, but we will go over it on Review Day (Thursday 10/1)." ] }, { "cell_type": "code", "execution_count": 1, "id": "049c24b6-a6c1-4ccc-b2de-5ea0366d09f8", "metadata": { "execution": { "iopub.execute_input": "2026-09-16T22:25:27.672680Z", "iopub.status.busy": "2026-09-16T22:25:27.672514Z", "iopub.status.idle": "2026-09-16T22:25:29.419795Z", "shell.execute_reply": "2026-09-16T22:25:29.419178Z" } }, "outputs": [], "source": [ "import numpy as np\n", "import pandas as pd\n", "\n", "import matplotlib.pyplot as plt\n", "import plotly.express as px\n", "from plotly.subplots import make_subplots\n", "import plotly.io as pio\n", "pio.renderers.default = 'notebook_connected'" ] }, { "cell_type": "markdown", "id": "23773272-c059-4148-a5a8-91e9da594c46", "metadata": {}, "source": [ "## Lecture Questions\n", "\n", "First we will import the data about female scientists." ] }, { "cell_type": "code", "execution_count": 2, "id": "e90fdd5f-bd17-4c94-b3f4-7ddc998bd001", "metadata": { "execution": { "iopub.execute_input": "2026-09-16T22:25:29.421624Z", "iopub.status.busy": "2026-09-16T22:25:29.421429Z", "iopub.status.idle": "2026-09-16T22:25:29.988710Z", "shell.execute_reply": "2026-09-16T22:25:29.988059Z" } }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
namebirth_yeardeath_year
0Janaki Ammal18971984.0
1Chien-Shiung Wu19121997.0
2Katherine Johnson19182020.0
3Rosalind Franklin19201958.0
4Vera Rubin19282016.0
5Gladys West1930NaN
6Flossie Wong-Staal1947NaN
7Jennifer Doudna1964NaN
\n", "
" ], "text/plain": [ " name birth_year death_year\n", "0 Janaki Ammal 1897 1984.0\n", "1 Chien-Shiung Wu 1912 1997.0\n", "2 Katherine Johnson 1918 2020.0\n", "3 Rosalind Franklin 1920 1958.0\n", "4 Vera Rubin 1928 2016.0\n", "5 Gladys West 1930 NaN\n", "6 Flossie Wong-Staal 1947 NaN\n", "7 Jennifer Doudna 1964 NaN" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
nameprofession
0Ada LovelaceMathematician
1Marie CuriePhysicist and Chemist
2Janaki AmmalBotanist
3Chien-Shiung WuPhysicist
4Katherine JohnsonMathematician
5Rosalind FranklinChemist
6Vera RubinAstronomer
7Gladys WestMathematician
8Flossie Wong-StaalVirologist and Molecular Biologist
9Jennifer DoudnaBiochemist
\n", "
" ], "text/plain": [ " name profession\n", "0 Ada Lovelace Mathematician\n", "1 Marie Curie Physicist and Chemist\n", "2 Janaki Ammal Botanist\n", "3 Chien-Shiung Wu Physicist\n", "4 Katherine Johnson Mathematician\n", "5 Rosalind Franklin Chemist\n", "6 Vera Rubin Astronomer\n", "7 Gladys West Mathematician\n", "8 Flossie Wong-Staal Virologist and Molecular Biologist\n", "9 Jennifer Doudna Biochemist" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
nameknown_for
0Ada Lovelacefirst computer algorithm
1Marie Curietheory of radioactivity, discovery of element...
2Janaki Ammalhybrid species, biodiversity protection
3Chien-Shiung Wuconfim and refine theory of radioactive beta d...
4Katherine Johnsoncalculations of orbital mechanics critical to ...
5Vera Rubinexistence of dark matter
6Gladys Westmathematical modeling of the shape of the Eart...
7Flossie Wong-Staalfirst scientist to clone HIV and create a map ...
8Jennifer Doudnaone of the primary developers of CRISPR, a gro...
\n", "
" ], "text/plain": [ " name known_for\n", "0 Ada Lovelace first computer algorithm\n", "1 Marie Curie theory of radioactivity, discovery of element...\n", "2 Janaki Ammal hybrid species, biodiversity protection\n", "3 Chien-Shiung Wu confim and refine theory of radioactive beta d...\n", "4 Katherine Johnson calculations of orbital mechanics critical to ...\n", "5 Vera Rubin existence of dark matter\n", "6 Gladys West mathematical modeling of the shape of the Eart...\n", "7 Flossie Wong-Staal first scientist to clone HIV and create a map ...\n", "8 Jennifer Doudna one of the primary developers of CRISPR, a gro..." ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "file1 = 'https://joannabieri.com/introdatascience/data/dates.csv'\n", "DF_dates = pd.read_csv(file1)\n", "file2 = 'https://joannabieri.com/introdatascience/data/professions.csv'\n", "DF_professions = pd.read_csv(file2)\n", "file3 = 'https://joannabieri.com/introdatascience/data/works.csv'\n", "DF_works = pd.read_csv(file3)\n", "display(DF_dates)\n", "display(DF_professions)\n", "display(DF_works)" ] }, { "cell_type": "markdown", "id": "941e4499-edd1-4210-b3ff-292dd2eb4df7", "metadata": {}, "source": [ "**Q1** Are each of these data sets Tidy?\n", "\n", "**Q2** Do they all contain the same number of observations?\n", "\n", "**Q3** What are the five variables?\n", "\n", "**Q4** What variable do they have in common - what can we join **on**" ] }, { "cell_type": "markdown", "id": "0e1f769b-d87d-429a-9de8-c35af020525d", "metadata": {}, "source": [ "**(Click Here)**\n", "\n" ] }, { "cell_type": "markdown", "id": "fa09683e-ac6a-4cb5-a47d-6cbf580524d2", "metadata": {}, "source": [ "## Combining Three Data Sets\n", "\n", "You need to think about the order and whether or not you want to keep the maximum number of names or only keep names that are in all the data frames.\n", "\n", "In this case we will try to keep the maximal amount of data - the maximum number of names.\n", "\n", "1. Combine the professions and works data\n", "2. Then add the dates data" ] }, { "cell_type": "code", "execution_count": 3, "id": "1e06d293-4d62-49f6-86c5-568b22cee865", "metadata": { "execution": { "iopub.execute_input": "2026-09-16T22:25:29.990539Z", "iopub.status.busy": "2026-09-16T22:25:29.990350Z", "iopub.status.idle": "2026-09-16T22:25:30.001184Z", "shell.execute_reply": "2026-09-16T22:25:30.000814Z" } }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
nameprofessionknown_forbirth_yeardeath_year
0Ada LovelaceMathematicianfirst computer algorithmNaNNaN
1Marie CuriePhysicist and Chemisttheory of radioactivity, discovery of element...NaNNaN
2Janaki AmmalBotanisthybrid species, biodiversity protection1897.01984.0
3Chien-Shiung WuPhysicistconfim and refine theory of radioactive beta d...1912.01997.0
4Katherine JohnsonMathematiciancalculations of orbital mechanics critical to ...1918.02020.0
5Rosalind FranklinChemistNaN1920.01958.0
6Vera RubinAstronomerexistence of dark matter1928.02016.0
7Gladys WestMathematicianmathematical modeling of the shape of the Eart...1930.0NaN
8Flossie Wong-StaalVirologist and Molecular Biologistfirst scientist to clone HIV and create a map ...1947.0NaN
9Jennifer DoudnaBiochemistone of the primary developers of CRISPR, a gro...1964.0NaN
\n", "
" ], "text/plain": [ " name profession \\\n", "0 Ada Lovelace Mathematician \n", "1 Marie Curie Physicist and Chemist \n", "2 Janaki Ammal Botanist \n", "3 Chien-Shiung Wu Physicist \n", "4 Katherine Johnson Mathematician \n", "5 Rosalind Franklin Chemist \n", "6 Vera Rubin Astronomer \n", "7 Gladys West Mathematician \n", "8 Flossie Wong-Staal Virologist and Molecular Biologist \n", "9 Jennifer Doudna Biochemist \n", "\n", " known_for birth_year death_year \n", "0 first computer algorithm NaN NaN \n", "1 theory of radioactivity, discovery of element... NaN NaN \n", "2 hybrid species, biodiversity protection 1897.0 1984.0 \n", "3 confim and refine theory of radioactive beta d... 1912.0 1997.0 \n", "4 calculations of orbital mechanics critical to ... 1918.0 2020.0 \n", "5 NaN 1920.0 1958.0 \n", "6 existence of dark matter 1928.0 2016.0 \n", "7 mathematical modeling of the shape of the Eart... 1930.0 NaN \n", "8 first scientist to clone HIV and create a map ... 1947.0 NaN \n", "9 one of the primary developers of CRISPR, a gro... 1964.0 NaN " ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# 1. Combine the professions and works data\n", "DF_scientists = pd.merge(DF_professions,DF_works,on='name',how='left')\n", "DF_scientists\n", "# 2. Then add the dates data\n", "DF_scientists = pd.merge(DF_scientists,DF_dates,on='name',how='left')\n", "DF_scientists" ] }, { "cell_type": "markdown", "id": "e2bbc33c-6c0e-4ca8-ad72-de5018bddce8", "metadata": {}, "source": [ "**Q5** What, if anything, would change if you switched to how='right'?" ] }, { "cell_type": "markdown", "id": "def3bc37-ca8a-4782-a154-6b51e084b944", "metadata": {}, "source": [ "**(Click Here)**" ] }, { "cell_type": "markdown", "id": "26a9dee0-e533-4439-8972-f2e69b544ec9", "metadata": {}, "source": [ "## Combining Two Data Sets - You Try\n", "\n", "**Q6** Write code that would combine the professions data and the dates data, but drop any names that don't appear in both." ] }, { "cell_type": "code", "execution_count": 4, "id": "18d09352-c2dc-4c21-887d-7e54387c55a9", "metadata": { "execution": { "iopub.execute_input": "2026-09-16T22:25:30.002807Z", "iopub.status.busy": "2026-09-16T22:25:30.002641Z", "iopub.status.idle": "2026-09-16T22:25:30.004525Z", "shell.execute_reply": "2026-09-16T22:25:30.004169Z" } }, "outputs": [], "source": [ "# Your code here\n", "\n" ] }, { "cell_type": "markdown", "id": "81aaba84-81ab-4fc7-95cb-e6df13157e2c", "metadata": {}, "source": [ "## Combining Three Data Sets - You Try\n", "\n", "**Q7** Try to combine the three data sets together (professions, works, and dates) but instead of doing what we did above, see if you can keep only the names that appeared in all three data sets." ] }, { "cell_type": "code", "execution_count": 5, "id": "c2766e8b-f125-4bd6-9937-13cd92c8ba62", "metadata": { "execution": { "iopub.execute_input": "2026-09-16T22:25:30.005969Z", "iopub.status.busy": "2026-09-16T22:25:30.005813Z", "iopub.status.idle": "2026-09-16T22:25:30.007640Z", "shell.execute_reply": "2026-09-16T22:25:30.007292Z" } }, "outputs": [], "source": [ "# Your code here\n", "\n" ] }, { "cell_type": "markdown", "id": "e2a3bd38-8875-4133-9fb5-2080542349b6", "metadata": {}, "source": [ "## Case study - Grocery Sales\n", "\n", "Below you will load some data about grocery sales and see how joining or merging data can help us answer questions." ] }, { "cell_type": "code", "execution_count": 6, "id": "e73afe67-db9b-49da-b8db-fe0e8d7b4bde", "metadata": { "execution": { "iopub.execute_input": "2026-09-16T22:25:30.009141Z", "iopub.status.busy": "2026-09-16T22:25:30.008986Z", "iopub.status.idle": "2026-09-16T22:25:30.383782Z", "shell.execute_reply": "2026-09-16T22:25:30.383027Z" } }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
customer_iditem
01bread
11milk
21banana
32milk
42toilet paper
\n", "
" ], "text/plain": [ " customer_id item\n", "0 1 bread\n", "1 1 milk\n", "2 1 banana\n", "3 2 milk\n", "4 2 toilet paper" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
itemprice
0avocado0.50
1banana0.15
2bread1.00
3milk0.80
4toilet paper3.00
\n", "
" ], "text/plain": [ " item price\n", "0 avocado 0.50\n", "1 banana 0.15\n", "2 bread 1.00\n", "3 milk 0.80\n", "4 toilet paper 3.00" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "file1 = 'https://joannabieri.com/introdatascience/data/purchases.csv'\n", "DF_purchases = pd.read_csv(file1)\n", "file2 = 'https://joannabieri.com/introdatascience/data/prices.csv'\n", "DF_prices = pd.read_csv(file2)\n", "display(DF_purchases)\n", "display(DF_prices)" ] }, { "cell_type": "markdown", "id": "59cc40ab-4b37-4c68-8da7-fdb713a184e5", "metadata": {}, "source": [ "## Calculate the total revenue\n", "\n", "**Q8** Given the data above find the total revenue.\n", "\n", "We will have to join the data frames so that we can see the price of what was sold. So first we look for a common column. In this case both data frames have **item** as a column.\n", "\n", "Now, think about how to calculate revenue... we need to add up the total money we made, so we need to know the price of each thing sold.\n", "\n", "**PAUSE - see if you can write some of the code for this before looking at the cells below**" ] }, { "cell_type": "code", "execution_count": 7, "id": "0e354cf0-ce17-492c-b23f-129f01826d3f", "metadata": { "execution": { "iopub.execute_input": "2026-09-16T22:25:30.385996Z", "iopub.status.busy": "2026-09-16T22:25:30.385826Z", "iopub.status.idle": "2026-09-16T22:25:30.387747Z", "shell.execute_reply": "2026-09-16T22:25:30.387393Z" } }, "outputs": [], "source": [ "#" ] }, { "cell_type": "code", "execution_count": 8, "id": "d83267c8-fe75-4a94-85b8-e9e8e52fccc5", "metadata": { "execution": { "iopub.execute_input": "2026-09-16T22:25:30.389090Z", "iopub.status.busy": "2026-09-16T22:25:30.388933Z", "iopub.status.idle": "2026-09-16T22:25:30.390739Z", "shell.execute_reply": "2026-09-16T22:25:30.390396Z" } }, "outputs": [], "source": [ "#" ] }, { "cell_type": "code", "execution_count": 9, "id": "a6e9ef48-206c-45fd-96c4-751cba05b9b5", "metadata": { "execution": { "iopub.execute_input": "2026-09-16T22:25:30.392137Z", "iopub.status.busy": "2026-09-16T22:25:30.391982Z", "iopub.status.idle": "2026-09-16T22:25:30.393810Z", "shell.execute_reply": "2026-09-16T22:25:30.393472Z" } }, "outputs": [], "source": [ "#" ] }, { "cell_type": "code", "execution_count": 10, "id": "c42954be-51ba-48ba-a6de-e91b12964874", "metadata": { "execution": { "iopub.execute_input": "2026-09-16T22:25:30.395192Z", "iopub.status.busy": "2026-09-16T22:25:30.395039Z", "iopub.status.idle": "2026-09-16T22:25:30.397089Z", "shell.execute_reply": "2026-09-16T22:25:30.396598Z" } }, "outputs": [], "source": [ "#" ] }, { "cell_type": "markdown", "id": "e437831b-9e27-40c1-b7e1-893785439543", "metadata": {}, "source": [ "**See the lecture notes if you need help figuring this out!**\n", "\n", "**Q9** Calculate the revenue per customer? Hint - group by the customer id and then apply the sum()." ] }, { "cell_type": "code", "execution_count": 11, "id": "0863ba4f-65d0-4465-bd70-4557c581a15d", "metadata": { "execution": { "iopub.execute_input": "2026-09-16T22:25:30.398509Z", "iopub.status.busy": "2026-09-16T22:25:30.398344Z", "iopub.status.idle": "2026-09-16T22:25:30.400300Z", "shell.execute_reply": "2026-09-16T22:25:30.399880Z" } }, "outputs": [], "source": [ "# Your code here\n" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.13.5" } }, "nbformat": 4, "nbformat_minor": 5 }