What You Will Be Able To Do

• Describe data as examples, features, labels
• Explain training in simple steps
• Split data into train, validation, test
• Recognize overfitting and reduce it
• Train and evaluate a small classifier

Three Kinds Of Learning

• Supervised: examples come with answers
• Unsupervised: find groups without answers
• Reinforcement: learn from rewards
• This course mostly uses supervised learning

Examples, Features, Labels

• Each row is one example
• Columns on the left are features
• The last column is the label

area  rooms  age  | price
 80     3    20   | 150000
 55     2     5   | 130000
120     4    35   | 210000

Parameters: What The Model Stores

• Parameters are internal numbers
• Training adjusts them
• Large models have billions of parameters

price = w * area + b

w and b are parameters
training finds good values

Loss: How Wrong Are We?

• One number that sums up the errors
• Large when predictions are poor
• Small when predictions are good
• Training tries to make it small

Gradient Descent

• Small steps downhill in fog
• Learning rate sets the step size
• One full pass is an epoch

repeat:
  predict on training data
  compute loss
  nudge each parameter
    to lower the loss

Splitting Data Honestly

• Training set: learn from it
• Validation set: compare settings
• Test set: use once at the end
• Never tune on the test set

Overfitting

• Memorizes quirks of the training data
• High training score, low new-data score
• Like memorizing last year's exam answers
• Watch the gap between the two scores

Underfitting And Remedies

• Underfitting: too simple, poor on both sets
• Overfitting fixes: more data, simpler model
• Stop training earlier, add regularization
• Underfitting fixes: richer model, better features

Accuracy Can Mislead

• 1 spam email in 100, always answer not spam
• That model is 99 percent accurate
• It finds no spam at all
• Use measures that match the task

Precision And Recall

• Precision: of flagged items, how many were right?
• Recall: of all real positives, how many found?
• Cancer screening favors recall
• Spam filtering favors precision

Confusion Matrix

• TP and TN are correct answers
• FP and FN are the two kinds of mistake

                 predicted
              spam     ok
actual spam    TP      FN
actual ok      FP      TN

A Complete Small Example

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
X, y = load_iris(return_X_y=True)
Xtr, Xte, ytr, yte = train_test_split(
    X, y, test_size=0.25, random_state=0)
m = DecisionTreeClassifier(max_depth=3).fit(Xtr, ytr)
print(m.score(Xte, yte))

Data Quality And Bias

• A model reflects its data
• Missing groups lead to poor results for them
• Leakage: answer hidden inside a feature
• Document where the data came from

Recap

• Data is examples with features and labels
• Training lowers the loss by small steps
• Split data and keep the test set clean
• Overfitting shows as a gap in scores
• Pick metrics that fit the task

Next Week: Neural Networks

• Neurons, layers, and activations
• How backpropagation assigns blame
• Training a tiny network by hand

Questions And Exercise

• Exercise: train and test a classifier
• Compare tree depth 3 with unlimited depth
• Read the Google Crash Course chapter on data
• Bring one question to week 3