Least squares method
Fitting a model to measurements by minimising the sum of squared vertical gaps: residuals, the least-squares line, normal equations, parabola fits and linearising an exponential or power model.
- 29 cards
- 5 topics
- Free
The outline, topic by topic
Each topic is weighted the way the exam weighs it.
- Concepts
- Questions
- Sort and link
- The numbers
- Understand
A few cards from the deck
Least squares method: what is it?
Show the answer
A way of choosing a model's parameters by making the sum of the squared vertical gaps between the measurements and the model's values as small as possible.
Squaring keeps positive and negative gaps from cancelling and lets the minimum be found by differentiating: for a straight-line model, you get an explicit formula.
For four points, n = 4, Σx = 6, Σy = 14, Σxy = 30 and Σx² = 14. What is the slope of the least-squares line?
- 1.8
- 2.1
- 2.5
- 2.3
Show the answer
1.8
a = (nΣxy − ΣxΣy)/(nΣx² − (Σx)²) = (120 − 84)/(56 − 36) = 36/20 = 1.8. The other values come from wrong formulas: Σxy/Σx² ≈ 2.1, (Σxy − ΣxΣy)/(Σx² − (Σx)²) ≈ 2.5 (without the factors n) or Σy/Σx ≈ 2.3.
The least-squares line always passes through the … with coordinates (…; …).
Show the answer
The least-squares line always passes through the mean point with coordinates (x̄; ȳ).
The second normal equation, a Σx + b n = Σy, divided by n, reads a·x̄ + b = ȳ: the point (x̄; ȳ) satisfies y = ax + b.
Residual: what is it?
Show the answer
The vertical gap between a measurement yᵢ and the value the model gives at xᵢ: eᵢ = yᵢ − ŷᵢ.
Positive if the point is above the line, negative if it is below. For the least-squares line with an intercept, they sum to zero.