{ "cells": [ { "cell_type": "markdown", "id": "b5a95eb3", "metadata": {}, "source": [ "## Introduction to Data Science\n", "\n", "#### University of Redlands - DATA 101\n", "#### Prof: Joanna Bieri [joanna_bieri@redlands.edu](mailto:joanna_bieri@redlands.edu)\n", "#### [Class Website: data101.joannabieri.com](https://data101.joannabieri.com)\n", "\n", "---------------------------------------\n", "# Day 4 - In Class Challenge - CDC Health Data\n", "---------------------------------------\n", "\n", "You have all of your homework handed in, nice work! So today you get to dig into some data instead of catching up.\n", "\n", "You saw this data set at the very end of the Day 4 homework challenge. There you just made one graph. Today we are going to actually ask it some questions.\n", "\n", "GOALS:\n", "\n", "1. Use everything from Days 1-4 to answer real questions about a data set.\n", "2. Practice deciding which plot (or which table) answers which question.\n", "3. Practice being skeptical about what the data can and can't tell you.\n", "\n", "----------------------------------------------------------\n", "\n", "**How this works:**\n", "\n", "* Work in your group. One person types, but everyone talks.\n", "* You have about **40 minutes**. Do not worry if you don't finish, the challenges get harder as you go.\n", "* You do not need to turn this in. If you want to keep it, save it in your Day4 folder and push it to GitHub.\n", "* Be ready to show the class **one plot** and tell us **one thing** you learned from the data.\n", "\n", "NOTE - Everything you need is in the Day 3 and Day 4 notes. If you get stuck, open those up and copy the code that is closest to what you want, then change it. A few places below give you one new line of code, and it is always written out for you." ] }, { "cell_type": "markdown", "id": "d3fffd3e", "metadata": {}, "source": [ "### Load the packages and the data\n", "\n", "This is a sample of 60 people from the Behavioral Risk Factor Surveillance System, a phone survey run by the CDC. Each row is one person.\n", "\n", "Here is a link to the variable descriptions. Look at it, you will need it for the warm up!" ] }, { "cell_type": "code", "execution_count": null, "id": "5401977d", "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "import pandas as pd\n", "\n", "import matplotlib.pyplot as plt\n", "import plotly.express as px\n", "from plotly.subplots import make_subplots\n", "import plotly.io as pio\n", "pio.renderers.default = 'notebook_connected'" ] }, { "cell_type": "code", "execution_count": null, "id": "633c009f", "metadata": {}, "outputs": [], "source": [ "file_location = 'https://joannabieri.com/introdatascience/data/cdc.samp.csv'\n", "DF = pd.read_csv(file_location)\n", "display(DF)" ] }, { "cell_type": "markdown", "id": "de1b1ad5", "metadata": {}, "source": [ "---------------------------------------\n", "## Warm Up - Get to know the data\n", "---------------------------------------\n", "\n", "**Q1** How many **observations** and how many **variables** are in this data set?\n", "\n", "**Q2** Sort the variables into numerical and categorical. Be careful here!\n", "\n", "* `exerany`, `hlthplan`, and `smoke100` are stored as numbers. Are they really numerical? What do the 0 and 1 mean? (Check the variable descriptions.)\n", "* Is `genhlth` ordinal or nominal? What about `gender`?\n", "* Is `age` discrete or continuous? What about `weight`?" ] }, { "cell_type": "code", "execution_count": null, "id": "4f8af4d9", "metadata": {}, "outputs": [], "source": [ "# Your code here:" ] }, { "cell_type": "markdown", "id": "bd7cb6bb", "metadata": {}, "source": [ "**(Double Click Here to answer the questions)**\n", "\n", "Numerical variables:\n", "\n", "Categorical variables:\n", "\n", "The tricky ones and why:" ] }, { "cell_type": "markdown", "id": "4aebc3ff", "metadata": {}, "source": [ "**Q3** How many people are in each `genhlth` category? Use `value_counts()` and then make a bar plot.\n", "\n", "Your bar plot probably puts the categories in a strange order. Since `genhlth` is ordinal, the bars should go from poor to excellent. Here is the one new thing you need, add this argument inside `px.bar( ... )`:\n", "\n", "```\n", "category_orders={'genhlth': ['poor', 'fair', 'good', 'very good', 'excellent']}\n", "```\n", "\n", "Once it is in order, what does the shape of the bar plot tell you about how people rate their own health?" ] }, { "cell_type": "code", "execution_count": null, "id": "8a8ca0d9", "metadata": {}, "outputs": [], "source": [ "# Your code here:" ] }, { "cell_type": "markdown", "id": "9c40c073", "metadata": {}, "source": [ "**(Double Click Here to answer Q3)**" ] }, { "cell_type": "markdown", "id": "a08f93a5", "metadata": {}, "source": [ "---------------------------------------\n", "## Challenge 1 - How much weight do people want to lose?\n", "---------------------------------------\n", "\n", "The survey asked each person their current weight (`weight`) and the weight they wish they were (`wtdesire`), both in pounds.\n", "\n", "**Q4** Make a scatter plot with `weight` on the x axis and `wtdesire` on the y axis. Color the points by `gender` and add `hover_data='age'`.\n", "\n", "**Q5** If a point sits exactly on the line where `wtdesire` equals `weight`, what does that person want? What about a point *above* that line? Hover around and find one person who wants to **gain** weight.\n", "\n", "**Q6** Now find those people with code instead of hovering. You can compare two columns in a filter, eg. `DF[DF['wtdesire'] > DF['weight']]`. How many people want to gain weight? Are they mostly men or women?\n", "\n", "**Q7** There is one person way off by themselves on the plot. Who is it? (Show their row.) What would happen to the mean weight in this data set if you removed them? Try it! Compare `DF['weight'].mean()` with the mean after you filter that person out. Which describes a \"typical\" person better here, the mean or the median?" ] }, { "cell_type": "code", "execution_count": null, "id": "593225d1", "metadata": {}, "outputs": [], "source": [ "# Your code here - scatter plot:" ] }, { "cell_type": "code", "execution_count": null, "id": "d9bea7a2", "metadata": {}, "outputs": [], "source": [ "# Your code here - who wants to gain weight?" ] }, { "cell_type": "code", "execution_count": null, "id": "4a91e7d2", "metadata": {}, "outputs": [], "source": [ "# Your code here - the outlier and the mean:" ] }, { "cell_type": "markdown", "id": "b2af037f", "metadata": {}, "source": [ "**(Double Click Here to answer Q5, Q6, and Q7)**" ] }, { "cell_type": "markdown", "id": "9a706712", "metadata": {}, "source": [ "**Q8** Make a new column that measures how much weight each person wants to lose. Here is how you make a new column from two others, it works just like the math you would expect:" ] }, { "cell_type": "code", "execution_count": null, "id": "e6f6d569", "metadata": {}, "outputs": [], "source": [ "DF['want_to_lose'] = DF['weight'] - DF['wtdesire']\n", "display(DF)" ] }, { "cell_type": "markdown", "id": "cad62d91", "metadata": {}, "source": [ "Now use a box plot (`px.box`) or a violin plot (`px.violin`) to compare `want_to_lose` between the genders. Who wants to lose more weight on average? Is the difference big or small compared to the spread inside each group?\n", "\n", "Then check your eyes against the numbers: filter down to just one gender and run `.describe()` on `want_to_lose`, then do the other gender." ] }, { "cell_type": "code", "execution_count": null, "id": "c0dfc55f", "metadata": {}, "outputs": [], "source": [ "# Your code here - box or violin plot:" ] }, { "cell_type": "code", "execution_count": null, "id": "ba415f06", "metadata": {}, "outputs": [], "source": [ "# Your code here - describe for each gender:" ] }, { "cell_type": "markdown", "id": "aa31619f", "metadata": {}, "source": [ "**(Double Click Here to answer Q8)**" ] }, { "cell_type": "markdown", "id": "0a7e830a", "metadata": {}, "source": [ "---------------------------------------\n", "## Challenge 2 - Does how people feel match the numbers?\n", "---------------------------------------\n", "\n", "`genhlth` is how each person **rated their own health**. Let's see if that lines up with a number we can calculate.\n", "\n", "**Body Mass Index (BMI)** is one (imperfect!) way to combine height and weight into a single number. With height in inches and weight in pounds:\n", "\n", "$$ \\text{BMI} = 703 \\times \\frac{\\text{weight}}{\\text{height}^2} $$\n", "\n", "**Q9** Make a new column called `bmi`. In Python `**` means \"to the power of\", so `DF['height']**2` squares the heights.\n", "\n", "**Q10** Make a box plot of `bmi` split by `genhlth`. Put the categories in order from poor to excellent like you did in Q3. Does self-rated health line up with BMI? Where does it not?\n", "\n", "**Q11** Make a histogram of `bmi` and use `color='exerany'` to split it by whether the person exercised in the last month. What is annoying about the legend on this plot? Here is a fix, it swaps the 0 and 1 for words:\n", "\n", "```\n", "DF['exercise'] = DF['exerany'].replace({0: 'no', 1: 'yes'})\n", "```\n", "\n", "Remake the histogram with your new column. Do the people who exercise look different from the people who don't?" ] }, { "cell_type": "code", "execution_count": null, "id": "deb3709f", "metadata": {}, "outputs": [], "source": [ "# Your code here - bmi column:" ] }, { "cell_type": "code", "execution_count": null, "id": "4e94e6a0", "metadata": {}, "outputs": [], "source": [ "# Your code here - box plot by genhlth:" ] }, { "cell_type": "code", "execution_count": null, "id": "000786e2", "metadata": {}, "outputs": [], "source": [ "# Your code here - histogram by exercise:" ] }, { "cell_type": "markdown", "id": "92d4b6de", "metadata": {}, "source": [ "**(Double Click Here to answer Q10 and Q11)**" ] }, { "cell_type": "markdown", "id": "b4979fa0", "metadata": {}, "source": [ "---------------------------------------\n", "## Challenge 3 - Smoking and health, and being a skeptic\n", "---------------------------------------\n", "\n", "`smoke100` is 1 if the person has smoked at least 100 cigarettes in their life and 0 if not.\n", "\n", "**Q12** Make a bar plot of `genhlth` colored by `smoke100` (or by a word version of it, like you did in Q11). Put the categories in order. Does it look like smokers rate their health differently than non smokers?\n", "\n", "**Q13** Bar plots can fool you when the groups are different sizes. Filter the data down to just the smokers and run `value_counts()` on `genhlth`. Then do the same for the non smokers. Now compare. Does your answer to Q12 change?\n", "\n", "**Q14** Being a skeptic. Suppose your group found that smokers rate their health worse. Write down **two reasons** you should not go tell the news that \"smoking makes you feel unhealthy\" based on this data set. Think about how many people are in the data, how the data was collected, and what else might be different between the two groups." ] }, { "cell_type": "code", "execution_count": null, "id": "f06a7670", "metadata": {}, "outputs": [], "source": [ "# Your code here - bar plot:" ] }, { "cell_type": "code", "execution_count": null, "id": "fea3118c", "metadata": {}, "outputs": [], "source": [ "# Your code here - value_counts for smokers and non smokers:" ] }, { "cell_type": "markdown", "id": "8f15f589", "metadata": {}, "source": [ "**(Double Click Here to answer Q12, Q13, and Q14)**" ] }, { "cell_type": "markdown", "id": "40983b86", "metadata": {}, "source": [ "---------------------------------------\n", "## Challenge (optional) - Age\n", "---------------------------------------\n", "\n", "We have not used `age` yet. Pick one of these and make a plot that answers it:\n", "\n", "* Do older people in this sample rate their health lower? (Hint - `age` on one axis, `genhlth` on the other, and a box plot.)\n", "* Do older people want to lose more weight, or less?\n", "* Who is more likely to have a health plan (`hlthplan`), younger people or older people?" ] }, { "cell_type": "code", "execution_count": null, "id": "df7fe6ee", "metadata": {}, "outputs": [], "source": [ "# Your code here:" ] }, { "cell_type": "markdown", "id": "7a36a728", "metadata": {}, "source": [ "---------------------------------------\n", "## FREE FORM!!!\n", "---------------------------------------\n", "\n", "Pick any question you have about this data and make the best plot you can to answer it. Give it a real title, label the axes (with units!), and pick colors on purpose.\n", "\n", "Be ready to show this one to the class and to say, in one sentence, what the plot tells you and one reason you might be wrong." ] }, { "cell_type": "code", "execution_count": null, "id": "499631d8", "metadata": {}, "outputs": [], "source": [ "# Your code here:" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "name": "python" } }, "nbformat": 4, "nbformat_minor": 5 }