Weekly Homework 1: Fuel Economy

Author:

Introduction to Machine Learning

University of Redlands - DATA 301

Prof: Joanna Bieri joanna_bieri@redlands.edu

Class Website

Due Sunday 9/6 at 11:59pm. This covers Day 1 and Day 2.

GOALS:

Dust off your DATA 101 skills on a real, slightly messy dataset.
Do a train/test split properly, and understand why it comes early.
Use everything from Day 2: polynomial features, RMSE, underfitting and overfitting.
Say what your numbers actually mean.

How to turn this in. In your own repository:

git checkout -b hw/week-01
git add .
git commit -m "Weekly homework 1"
git push origin hw/week-01

Then open a Pull Request into main. That Pull Request is your submission.

What I am grading. Working code with no writing is not a pass. Every question that asks “why” or “what does this mean” needs real sentences. That is the part I actually care about.

This is a full mini project, so unlike the daily notes there are no optional parts here. Answer everything, in order. Several questions ask you to commit to a guess before you run the code, and those are there on purpose, so do not skip ahead and backfill them.

The data

We are using the mpg dataset: 398 cars from the 1970s and early 80s with their fuel economy in miles per gallon. It is real data, which means it has some problems in it. Good.

The question: can we predict a car’s fuel economy from its horsepower?

Part 1: Get to know the data

This part is DATA 101 review. You should be able to do all of it already.

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

cars = sns.load_dataset("mpg")
cars.head()

1a. How many rows and columns? What does a single row represent in the real world?

1b. Print the column names and their data types. Which are numeric and which are not?

1c. Run .describe(). Pick two columns and say in words what their ranges tell you about these cars.

# your code here

Your answers here.

Missing data

1d. Check for missing values. Which column has them, and how many?

1e. Print the actual rows that are missing something.

1f. Decide what to do about them: drop, fill, or keep. Do it, and defend it in two or three sentences. There is no single right answer, but there is a wrong one, which is doing something without saying why.

# your code here

Your answer here.

1g. Before you write any more code, write down what you expect the relationship between horsepower and mpg to look like. Going up? Down? Straight? Curved?

You will check this in Part 3. Being wrong is fine and interesting, and I would rather see a wrong guess than no guess.

Your answer here.

Part 2: Split before you look too hard

Here is what is new since DATA 101.

We are about to explore this data, plot it, and get ideas from it. Every one of those ideas is a decision. If we make those decisions while looking at all the data, we have quietly used the test set to build the model, and then it is not a fair test any more.

So we split now, before the interesting exploration, and from here on we only look at the training set.

from sklearn.model_selection import train_test_split

# Split your cleaned data 80/20. Use random_state=42 so your results are reproducible.
# Predict mpg from horsepower.

# your code here

2a. How many rows in each set?

2b. Why random_state=42? What would happen if you left it out and reran the notebook a few times?

2c. In your own words, why did we split before exploring rather than after?

Your answers here.

Part 3: Explore the training set

Now look properly. Training set only.

3a. Scatter plot horsepower against mpg. Compare it to what you predicted in 1g. Were you right?

3b. Is this a straight line relationship? Say what you see.

3c. Make one more plot of your own choosing and say what it tells you.

# your code here

Your answers here.

Part 4: Fit some models

Training set only.

4a. Fit a straight line and plot it on the training data.

4b. Report the training RMSE. What are the units of that number? A student says “the RMSE is 4.9.” What does that mean about a car?

from sklearn.linear_model import LinearRegression
from sklearn.metrics import root_mean_squared_error

# your code here

Your answers here.

4c. Fit polynomial models of degree 2, 5, and 15. Plot them all on one figure with the training data.

4d. Make a small table of training RMSE for each degree, including the straight line.

4e. Look at your table. Is degree 15 much better than degree 2? Say what you expected and what actually happened.

from sklearn.preprocessing import PolynomialFeatures, StandardScaler

# your code here

Your answers here.

4f. You probably expected degree 15 to fit the training data much better than degree 2, and it barely did.

Before you read Part 5, write down your best guess as to why. What is it about this dataset that stops a very flexible model from running away?

Answer this before moving on. You will come back to it in 5d, and I want to see what you thought first.

Your answer here.

Part 5: The same models, much less data

Let’s test the theory you just wrote down.

Take 40 cars from your training set and pretend that is all the data you have. Nothing else changes.

# use exactly this so everyone gets the same 40 cars
rng = np.random.default_rng(42)
small_rows = rng.choice(len(X_train), 40, replace=False)

# your code here: build X_small and y_small from those rows

5a. Refit the straight line and degrees 2, 5, and 15 on just those 40 cars, and plot them on top of the 40 points.

5b. Report two numbers for each degree: RMSE on the 40 training cars, and RMSE on the test set.

5c. What happened to degree 15? Describe the size of the difference between its two numbers.

5d. Go back to what you wrote in 4f. Were you right? Now answer it properly: overfitting is not really about the degree. What is it about?

# your code here

Your answers here.

Part 6: Learning curves explain it

6a. Plot learning curves for the degree 15 model on the full training set. You can reuse plot_learning_curve from the Day 2 notes.

6b. Look at the left edge, where the training set is small. Does it match what you found in Part 5?

6c. Look at the right edge. Have the curves flattened out, or is the validation curve still coming down?

6d. Based only on that plot: if someone offered you data on another 400 cars, would it be worth having? Why?

from sklearn.model_selection import learning_curve
from sklearn.pipeline import make_pipeline

# your code here

Your answers here.

Part 7: The test set, once

Every decision so far was made on the training set. Time to commit.

7a. Pick the degree you think is best for the full training set and say why, before you touch the test set.

7b. Now evaluate that one model on the test set. Report the test RMSE.

7c. How does it compare to the training RMSE for the same model? Is that what you expected?

7d. Suppose the test RMSE came out worse than you hoped. Are you allowed to go back, pick a different degree, and report that instead? Why or why not?

# your code here

Your answers here.

Part 8: What does it actually mean

8a. In two or three sentences with no equations: how well can you predict fuel economy from horsepower alone? Would you trust this model?

8b. Your best RMSE is probably somewhere around 4. Is that good? What would you need to know to answer that properly?

8c. This data is cars from 1970 to 1982. Someone wants to use your model on a 2026 electric vehicle. What do you tell them?

8d. Suppose a manufacturer used a model like this to advertise fuel economy to customers. What would it mean for the model to be wrong, and who would pay for it?

Your answers here.

8e. What is the one thing in this assignment you did not expect? A short paragraph.

Your answer here.