K-nearest neighbours
One whole algorithm: measure the distances to the point to classify, keep the k nearest, vote or average; the choice of k, the effect of scales and the cost of a prediction.
- 22 cards
- 6 topics
- Free
The outline, topic by topic
Each topic is weighted the way the exam weighs it.
- Concepts
- Questions
- Sort and link
- The numbers
- The curves
- Scenarios
A few cards from the deck
k-nearest neighbours: what is it?
Show the answer
An algorithm that predicts a point's target from the k training examples nearest to it.
It measures the distance from the point to classify to every example, keeps the k smallest, then lets those neighbours vote. The idea: close points are alike.
Six training examples are given (coordinates, class). Which class is predicted for the point (2, 3) with k = 3?
```text
(1, 1) rouge
(2, 1) rouge
(2, 2) bleu
(5, 5) bleu
(6, 5) bleu
(6, 6) rouge`- Red: two of its three neighbours are red
- Blue: its nearest neighbour, (2, 2), is blue
- Blue: three examples out of six in the set are blue
- None: the three neighbours do not all share the same class
Show the answer
Red: two of its three neighbours are red
The distances to (2, 3) are 1 for (2, 2), 2 for (2, 1), √5 ≈ 2.24 for (1, 1), then much more for the rest. The three nearest are blue, red, red: the vote gives red. With k = 1, the answer would have been blue.
To classify a point, the k neighbours …; in regression, the … of their targets is taken instead.
Show the answer
To classify a point, the k neighbours vote; in regression, the average of their targets is taken instead.
A classification target is a label, which is counted; a regression target is a number, which is averaged.