Digital tools for statistics
Doing the path's calculations in a spreadsheet and in Python: spreadsheet functions, pivot table, numpy, pandas, matplotlib, and the n versus n − 1 divisor trap.
- 24 cards
- 3 topics
- Free
The outline, topic by topic
Each topic is weighted the way the exam weighs it.
- Concepts
- Questions
- Sort and link
A few cards from the deck
STDEV.P: what is it?
Show the answer
The spreadsheet function that computes the standard deviation of a series taken as a whole population, with divisor n.
Use it when the cells hold every individual studied, without exception: it is the calculation done by hand in class, and the equivalent of np.std(x) in numpy.
A script computes the standard deviation of the same column with np.std(x) and with pd.Series(x).std(): 2.0 on one side, 2.138 on the other. What happened?
- numpy divides by n by default, pandas by n − 1
- pandas rounds the values before adding them
- numpy ignores repeated values in the column
- pandas computes the variance instead of the standard deviation
Show the answer
numpy divides by n by default, pandas by n − 1
For the series 2, 4, 4, 4, 5, 5, 7, 9: the sum of squared deviations is 32. 32 / 8 = 4 gives 2.0 (numpy), 32 / 7 ≈ 4.571 gives 2.138 (pandas). Both tools are right: they answer two different questions.
In Python, numpy's np.std function divides by … by default, whereas pandas' std method divides by ….
Show the answer
In Python, numpy's np.std function divides by n by default, whereas pandas' std method divides by n − 1.
It is the classic source of a discrepancy between two tools on the same data. The ddof parameter (“delta degrees of freedom”) sets the divisor: n − ddof.
STDEV.S: what is it?
Show the answer
The spreadsheet function that computes the standard deviation of a sample, with divisor n − 1.
Use it when the series is only a sample of a larger population. It gives a slightly larger result than the divisor-n version, especially for a small series.
A group's marks are in B2:B31. Which formula gives the proportion of marks of at least 10?
=COUNTIF(B2:B31,">=10")=COUNTIF(B2:B31,">=10")/COUNT(B2:B31)=COUNTIF(B2:B31,10)/COUNT(B2:B31)=SUMIF(B2:B31,">=10")/SUM(B2:B31)
Show the answer
=COUNTIF(B2:B31,">=10")/COUNT(B2:B31)COUNTIF counts the marks that meet the criterion, COUNT counts all the marks: their quotient is the frequency. The first wrong answer gives a count, the second only counts marks equal to 10, the third gives the share of points, not of students.