Introduction to Machine Learning

Course Setup, Git, and the ML Landscape

Author

Joanna Bieri
DATA301

Welcome to Machine Learning!

Important Information

How to use these notes

Two kinds of box show up in these notes.

Blue Q boxes are questions for you to answer by hand, in a notebook, with a pen. Not because I am old fashioned. Writing something down by hand is slow, and slow is the point: it is very hard to write an explanation you do not actually understand. You are welcome to use AI in this class for the mechanics of code, but these boxes are the part where you do the thinking yourself. Bring your written notes to class, I will ask to see them.

Green You Try boxes are optional code for you to work through. Nothing is collected and nothing is graded. They are there because you will learn more from changing a number and rerunning than from watching me do it.

Today

Three things, in this order:

  1. Get your computer set up so you can train models on it.
  2. Get your GitHub repository working, since that is how you will turn in every assignment.
  3. Talk about what machine learning actually is, and what we are going to do for fourteen weeks.

A good set up today saves a lot of heartache later in the semester!

Computer Set Up

You need a computer that can run Anaconda Python and JupyterLab. If your machine cannot handle it, talk to me: the Data Science program has a laptop lending program and we will get you sorted out.

Step 1: Check what you already have

Run each of these. You are looking for Python 3.11 or 3.12, any recent conda, and any git at all.

!python --version
Python 3.12.14
!conda --version
conda 24.11.3
!git --version
git version 2.34.1

If git --version fails, install git first:

Step 2: Install the packages

Once your repository is cloned (we do that in the next section), open a terminal, move into your repository folder, and run:

pip install -r requirements.txt

This installs everything into your regular Anaconda Python, so there is nothing special to turn on later.

Because this goes into the same Python your other classes use, requirements.txt mostly does not lock packages to an exact version. If you already have a working numpy or pandas from DATA 101 or DATA 201, this leaves it alone.

The exception is the deep learning packages, which have an upper limit on the major version, like transformers<6.0. Those libraries change fast, and a new major release in the middle of the semester can break notebooks that worked the week before. The limit lets you pick up bug fixes without getting a breaking change.

Finding a terminal

One of the packages you are installing is jupyterlab-git, and it is the easiest way to get to a terminal, especially on Windows. Once it is installed, restart JupyterLab and you will get two things:

  • A Git panel in the left sidebar, where you can commit, push, and pull by clicking.
  • A Terminal option in the Launcher, so you never have to hunt for a terminal in Windows.

For the very first install you do need a terminal from somewhere. On Windows use Anaconda Prompt from the Start menu. On Mac use Terminal. After that, JupyterLab can be your home base.

Read this before you install, especially on Linux

On Linux, a plain pip install torch downloads the CUDA build, which is roughly 2.5 GB of GPU libraries that your laptop probably cannot use. Everything in this course runs on CPU. So on Linux, install the small build first:

pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu

and then run pip install -r requirements.txt as normal.

On Mac and Windows the default install is already the right one.

Step 3: Check that it worked

Restart your kernel, then run this. If every line prints a version number then you are all set, and you should help your neighbor!

import sys
print("python      ", sys.version.split()[0])

import numpy, pandas, sklearn
print("numpy       ", numpy.__version__)
print("pandas      ", pandas.__version__)
print("scikit-learn", sklearn.__version__)

import torch
print("torch       ", torch.__version__)
print("  CPU is all we need:", torch.tensor([1.0, 2.0]).sum().item(), "should be 3.0")

import transformers
print("transformers", transformers.__version__)
python       3.12.14
numpy        2.5.2
pandas       3.0.5
scikit-learn 1.9.0
torch        2.13.0+cu130
  CPU is all we need: 3.0 should be 3.0
transformers 5.16.1

If something fails here, it is almost always one of these:

  1. JupyterLab is running an old kernel. Restart the kernel, or quit JupyterLab and open it again.
  2. The install quietly failed partway through. Scroll back through the pip install output and look for red text.
  3. You have more than one Python installed and JupyterLab is using the other one. Run the cell below to check which Python you are actually on.

Let me know which one it was. That tells me what to fix for next year.

import sys
print("JupyterLab is using this Python:")
print(sys.executable)
JupyterLab is using this Python:
/home/bellajagu/anaconda3/envs/machinelearning/bin/python3.12

Your Git Workflow

Every assignment in this course is turned in through GitHub. There is a good reason for this. In machine learning you will run a LOT of experiments, and they are very easy to run and very hard to remember. Your commit history ends up being the only reliable record of what you actually tried.

You are working with two repositories

Repository What it is Can you write to it?
MachineLearningUoR/FALL26 The course materials. New notes and assignments appear here all semester. No, read only
MachineLearningUoR/DATA301-F26-<your-username> Yours. Private. All of your work lives here. Yes

In git terms, FALL26 is your upstream and your own repo is your origin. Material flows one direction: from upstream into your repo. Your work never flows back to upstream.

One-time setup

Do this once, today. I have already created your repository for you.

git clone https://github.com/MachineLearningUoR/DATA301-F26-<your-username>.git
cd DATA301-F26-<your-username>
git remote add upstream https://github.com/MachineLearningUoR/FALL26.git
pip install -r requirements.txt
nbstripout --install

After this, restart JupyterLab and open your repository folder in it. The Git panel in the left sidebar will be pointed at your repo, and you can do most of the day to day git work from there instead of typing commands.

Don’t skip that last line. nbstripout strips the output cells out of your notebooks before they get committed, so your diffs show what you actually changed instead of the fact that you re-ran a cell and the plot came out with slightly different pixels. Without it, notebook merge conflicts are miserable.

Check that both remotes are there:

git remote -v

You should see origin pointing at your repo and upstream pointing at FALL26.

The weekly sync: getting new material

Whenever I announce new content, run this in your repo:

git fetch upstream
git checkout main
git merge upstream/main
git push origin main

Turning in work: branch, then Pull Request

Never do your homework directly on main. For each assignment:

Why bother with a branch? Three reasons.

  • Your main branch always holds a version that works. If you try something on a branch and it goes badly, you throw the branch away and main is still fine.
  • The Pull Request shows exactly what changed, so my comments can land on the specific lines I am reacting to instead of somewhere in a 300 line notebook.
  • This is how almost every software team works, so it is worth getting used to now.
git checkout -b hw/week-01          # start a branch
# ... do the work, committing as you go ...
git add .
git commit -m "Finish Day 1 setup checks"
git push origin hw/week-01

Then go to your repository on GitHub and open a Pull Request from your branch into main. That Pull Request is your submission. I read it, leave comments inline on the specific lines I am reacting to, and that is your feedback. When it is done, you merge it.

You are welcome to add a classmate as a reviewer, and I hope you do. One rule, which is also in the syllabus: reviewers comment, reviewers don’t commit. Reading someone else’s code and asking a good question about it is how you learn to read code. Writing it for them is not.

If git breaks today, you are not stuck

Getting fourteen people through git setup in one class period never goes perfectly. If yours will not cooperate, submit this week’s work on Canvas instead and come see me in office hours. Nobody loses points for a setup problem in week one!

What Is Machine Learning?

Machine learning is the science (and art) of programming computers so they can learn from data.

Aurelien Geron, chapter 1

Compare two ways of writing a spam filter.

The way you already know how to do it. You look at spam, notice that it says things like “free money” and comes from strange addresses, and you write rules. It works for a while. Then spammers write “fr33 m0ney” and you write more rules. Forever. The program only knows what you personally thought to tell it.

The machine learning way. You collect thousands of emails already labeled spam or not spam, and you write a program that finds the patterns itself. When spammers change tactics you do not rewrite the rules; you retrain on newer email.

The second program is not smarter than you! It is doing something you could do by hand, just across way more examples than you have patience for, and it keeps doing it after you stop paying attention.

An engineering way to think about machine learning is

A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E.

Tom Mitchell

So what does this mean in the SPAM/HAM example? The task, T = classify if something is spam or not (yes/no). The experience, E = trying to classify thousands of emails that have answers or labels. The performance measure, P = probably accuracy, how many did we get right. Well, the machine will start and not do very well but as it keeps trying it learns from the experience. This is an example of supervised learning.

The kinds of learning

Kind What the data looks like Example
Supervised Examples with correct answers attached Photos labeled cat or dog
Unsupervised Examples with no answers Customers, grouped by whatever structure exists
Self-supervised Answers generated from the data itself Hide a word in a sentence, predict the hidden word
Reinforcement No answers, just rewards and penalties An agent learning to play a game

Most of this course is supervised learning. Keep self-supervised in the back of your mind though, because it is the trick that makes large language models possible. We get there in November.

Two Flavors of Supervised Learning

Inside supervised learning there is one more split you need, and it comes up in almost every problem we look at.

  • Classification predicts a category. Spam or not spam. Which of ten digits this image shows. Whether a patient gets readmitted.
  • Regression predicts a number. The price of a house. Tomorrow’s temperature. How many minutes until you arrive.

The quick test: if the answer is a label, it is classification. If the answer is a quantity you could do arithmetic on, it is regression. Sometimes the same situation can be set up either way, and choosing which is part of framing the problem. “Will this customer leave?” is classification. “How many months until this customer leaves?” is regression.

Q1. Write this one out by hand

Say whether each of these is classification or regression:

Example 1: Real Estate

  • Scenario A: Predicting the exact selling price of a house in dollars based on its square footage and number of bedrooms.
  • Scenario B: Predicting whether a house will sell in under 30 days or take longer than 30 days.

Example 2: Email Management

  • Scenario A: Sorting incoming emails into folders labeled “Work,” “Personal,” or “Spam.”
  • Scenario B: Estimating how many minutes a user will spend reading a specific email based on its length and sender.

Example 3: Agriculture and Farming

  • Scenario A: Predicting the total weight of tomatoes a single plant will harvest in pounds.
  • Scenario B: Labeling a tomato plant’s health status as “Healthy,” “Nutrient Deficient,” or “Diseased.”

What this course is not

If you took or are taking DATA 201, you have/will already met k-nearest neighbors, logistic regression, naive Bayes, decision trees, support vector machines, PCA, and k-means. We also saw linear and logistic regressoin in DATA 101. We are not going to re-teach those.

Instead we start with a harder and more useful question: how do you know whether a model is any good? We spend the first three weeks on this. It is the part that most people get wrong, including a lot of working professionals.

If you have not taken DATA 201, you are fine. We start from regression, which you saw in DATA 101, and build up from there.

The cycle that runs the entire course

Almost everything we do fits this loop:

Frame the question is first for a reason. “Recommend good shows” is not something you can train or measure. “Given a user and a show, predict the probability that the user watches more than five minutes” is. Being specific about what goes in, what comes out, and how you would score it is most of the work of getting a project started.

Three piles of data

I am assuming that you are comfortable reading data into Python (pd.read_csv() or pd.read_excel() for example). Before you train anything, you split your data into three piles.

  • Training set. The model learns from this one. Usually most of your data around 80%.
  • Validation set. You use this while you are still making choices: which model, how many layers, what settings. Check it as often as you want. Around 10% of the data.
  • Test set. You use this once, at the very end, to report how well you did. Around 10% of the data.

The validation set is the one that saves you. If you want to try fifteen different models, try them all against validation. Then when you have picked one, you check the test set a single time and report that number.

The most important idea in this course is in that diagram: you only get to use your test set once. Every time you check your test performance and then go change something, a little information about the test set leaks into your model, and your reported accuracy gets a little less honest.

Now, in the real world, you sometimes have to look more than once to debug broken code, fix data pipeline errors, or verify that the model is actually making predictions. However, the golden rule remains: every time you peek at the test set to make a design choice, you are allowing data leakage!

Being wrong has a cost, and somebody pays it

Every model is wrong sometimes. The interesting question is what happens when it is wrong and who is hurt.

A navigation app that says 12 minutes when the real answer is 14 costs you almost nothing. A model that flags a credit card charge as fraud is a different story. Get it wrong one way and somebody’s real purchase gets declined at the grocery store. Get it wrong the other way and a stolen card keeps working. Those two mistakes are not the same!

We come back to this on Day 5 when we look at metrics, because plain “accuracy” hides exactly this. For now, get in the habit of asking two questions about any model you meet: what happens if it is wrong?

Where large language models fit

By the end of the semester you will build a small language model from scratch and train it on your own laptop! This will not be a demo but an actual attention model that you code, a transformer block that you put together, and weights that are trained until it produces text.

Your model will be small and it won’t be very good, and that is ok! Once you have built one, Chat GPT or Claude stop being magic. Then you can actually reason about what they can and can’t do. This is important for engineeing prompts and becomming AI power users.

The importance of DATA!

Since most (all?) of you are data science majors or minors, you know how hard it is to find, wrangle, and clean data. Getting good data is the first, and likely most important, part of building a good machine learning model. Data is the biggest challenge in machine learning. Why?

  1. Not enough data: To train a good model you need a lot of data. Modern Large Language Models (LLMs) were trained on the whole internet (pretty much). In a 2001 paper, Banko and Brill found that almost any model (simple or complicated) can have comparably good performance if they are just given enough data!
  2. Training data does not represent the general trend: If you train your model on data that is different from the data it will be using in real life the model will never perform well. An example of this would be having a huge data set of fruit and building a fruit classifier, but in your data only .001% of the data has pictures of figs. Well it is pretty obvious that your model will not be good at identifying figs. And to make matters worse, when you split your data into three chunks, there is a real chance there will be no examples of figs in the training data.
  3. Poor quality data: Obviously if your data is just completely wrong your model will also be wrong! Sometimes this is missing features, sometimes this is outliers, sometimes this is just a bad save.
  4. Irrelevant features: It is not always true that more features make a model better. Sometimes sending in variables that have nothing to do with the task will make a model behave strangely. For example, what if you were training a model to detect breast cancer and sent in the initials of all the patients. It is very unlikely that this will make your model better! In fact it could make your model worse. Feature selection, extraction, and engineering are important data science topics that flow into the machine learning model making work.

What You Should Already Know

This class assumes introductory programming in Python and an introductory data science course. Concretely, you should be comfortable with:

  • Importing modules and calling functions
  • Loading a CSV into a pandas DataFrame and looking at it
  • Basic plotting with matplotlib
  • Writing a loop and a function without looking it up

You do not need calculus or linear algebra to succeed here. We will use ideas from both, and I will teach them when they come up. If you are a math or physics major, you will recognize a lot of the math, even though we use it differently here. If you are a business or biology major, the ideas are a lot more approachable than the notation makes them look.

Before Next Class

  1. Finish the setup above. Everything in the check cell should print a version.
  2. Clone your repository, add the upstream remote, and run nbstripout --install.
  3. Read Geron chapter 1. It is short and it is worth reading properly.
  4. Do HW_day1.ipynb, in your repo under Day1/.

Bring your laptop on to class every day! We will be training models in class!