Introduction to Machine Learning
Final Projects
Important Information
- Email: joanna_bieri@redlands.edu
- Office Hours take place in Duke 209 - Office Hours Schedule
- Class Website
- Syllabus
Final Projects
The final project is your chance to take a machine learning problem all the way from a question to an honestly evaluated answer, in a subject area that your team cares about. It is 20% of your course grade.
You can use any model from this class: linear and logistic regression, trees and ensembles, XGBoost, neural networks, CNNs, or the transformer and LLM tools we see at the end of the semester. You do NOT have to use an LLM. Pick the model that fits your question, and then show me how well it really works.
This is the one piece of team work in this course. Teams are 3-4 students, and you choose your own team.
Due Dates:
- Tuesday 11/3 - Idea time at the end of class (15 min). Come with a topic you are curious about!
- Tuesday 11/10 11:59pm - Final Project Teams due (on Canvas, not graded)
- Tuesday 11/17 11:59pm - Final Project Proposal due (one per team, 20 points)
- Thursday 12/3 - In class work day and team check-in with me
- Wednesday 12/9 11:59pm - Final Project Report and Code due (100 points)
- Thursday 12/10 3:00pm - Final Project Presentations, our final exam time (30 points)
Getting Started - Ideas and Teams
On Tuesday 11/3 we will take the last 15 minutes of class to talk about ideas. Come with one or two subjects you are curious about (eg. your major, a sport, health, music, the environment, a job you would like to have). Talk with your classmates and find people who are excited about the same kind of question.
By Tuesday 11/10 at 11:59pm one person from each team submits the Final Project Teams assignment on Canvas with:
- The names of everyone on your team (3-4 people).
- Your subject area, in a sentence or two.
If you can’t find a team, that is ok! Tell me on Canvas and I will help place you.
Some places to look for data:
- Kaggle Datasets
- UC Irvine Machine Learning Repository
- Hugging Face Datasets
- data.gov and Our World in Data
Good data for this project has enough observations to make a real train/validation/test split (a few thousand is great, a few hundred can work for a simple model) and a clear thing you want to predict or classify.
Final Project Proposal (one per team, due Tuesday 11/17 11:59pm, 20 points)
The proposal is a short notebook. It does not have to be fancy, I want to see that your team has a real question, real data, and a plan. Include these sections:
- Question. What are you trying to predict or classify? Why does your team care about it?
- Data. Where is it from (give the link)? Load it and show the number of observations and variables and what your target looks like. Are there any ethical concerns with the data or how it was collected?
- Baseline. What is the simplest thing you could compare against? (eg. always predict the most common class, predict the mean, a plain linear or logistic regression). Every model you build later gets compared to this.
- Plan. Which model(s) do you want to try and why? What metric will you use to decide if a model is good, and why does that metric fit your question?
- Team plan. Who is doing what, and when will you meet?
Submit the notebook on Canvas and push it to your team’s repo.
Working as a Team
- Work in your team repo and use the git habits you already have: your own branch, Pull Requests, and a teammate looks at your code before you merge.
- Have at least one organizational meeting where you divide up the work. Every person should contribute to the modeling AND the writing.
- If your team needs more compute than a laptop can give, talk to me early and we will arrange access to a cloud GPU.
On Thursday 12/3 we have a work day in class and I will check in with each team. Come with your results so far and your questions.
Final Project Report and Code (due Wednesday 12/9 11:59pm, 100 points)
Your report is one JupyterLab notebook per team, pushed to main in your team repo. Whatever is on main at 11:59pm on Wednesday 12/9 is what gets graded. It should have these sections:
- Title, Authors and Abstract
- Introduction and Background
- Data
- Baseline
- Models and Results
- Final Evaluation on the Test Set
- Conclusion and Limitations
- Who Did What
Important requirements:
- Your code must run from start to finish without errors. Set your random seeds.
- Your project should load the data directly from an online source, or the data file must be in your team repo (if it is too big for git, tell us exactly how to download it).
- Split your data into train, validation, and test sets at the start. Make all of your choices (features, model, parameters) using the training and validation data. You only get to use your test set once, at the very end, on your one final model.
- Compare your final model to your baseline. If your fancy model barely beats the baseline, say so! That is a real and useful result.
- Be honest about your results. How sure are you of your numbers? Where does your model make mistakes? Who might be hurt if someone used it?
- Follow the AI policy in the syllabus. AI is welcome for the mechanics of code, and you are responsible for every number you report. If you copy and paste code an AI wrote, say so in a short note.
Presentations (Thursday 12/10 3:00pm, 30 points)
Each team presents during our final exam time. Tell us your question, show us your data, compare your model to your baseline, and tell us what your model can and can’t do. Plan for 10-15 minutes. Everyone on the team should speak.
Rubrics
Proposal (20 points)
| Part | Points |
|---|---|
| Question is clear and it is something you can predict or classify | 5 |
| Data is found, loaded, and described, with an ethics check | 5 |
| Baseline and metric are chosen, with a reason | 5 |
| Model plan and team plan | 5 |
Report and Code (100 points)
| Part | Points |
|---|---|
| Introduction and question: why it matters, what you are trying to do | 10 |
| Data: source, description, cleaning, ethics | 15 |
| Baseline: a simple model and its score | 10 |
| Models: at least one model from class, choices explained, tuned on validation data | 25 |
| Honest evaluation: test set used once, compared to baseline, mistakes and limits discussed | 25 |
| Writing: clear explanations, conclusions supported by your results | 10 |
| Code runs from start to finish, team repo used with PRs, Who Did What section | 5 |
Presentation (30 points)
| Part | Points |
|---|---|
| Question and data are clear to someone outside the class | 10 |
| Results compared to the baseline, with honest limits | 10 |
| Organized, on time, and everyone speaks | 10 |
Final Project Outline
Below are some notebook cells that outline how you might write up your final project.
FINAL PROJECT TITLE
By: Your Names
Abstract
Write a 3-4 sentence summary of: - The question you worked on - The data you used - The model you chose - How well it worked compared to your baseline
Introduction and Background
- What is your question, and why does your team care about it?
- Who would use a model like this?
- What would “good enough” look like for this problem?
Data
- Where did the data come from and how was it collected?
- How many observations and variables? What is your target?
- What cleaning did you do?
- Any ethical concerns (privacy, bias, who is missing from the data)?
Show a few of the most useful plots or tables, not every plot you made.
Make your train, validation, and test split here, at the start.
Baseline
Build the simplest reasonable model and report its score on the validation data. Everything after this gets compared to it.
Models and Results
For each model you try:
- Show the code (clean, runnable, commented).
- Show the validation score and any plots that help (eg. learning curves, confusion matrix).
- Explain in words what the result means and why you made your next choice.
End this section by saying which ONE model you are choosing and why.
Final Evaluation on the Test Set
Run your one chosen model on the test set, once.
- How does the test score compare to your validation score and to your baseline?
- Where does the model make mistakes? Show some examples.
Conclusion and Limitations
- What did you find? Does it answer your question?
- What are the limits of your model? When would you NOT trust it?
- What would you do next with more time, data, or compute?
Who Did What
A short list of what each team member worked on.