Orientation — how many makes a law

Orientation: how many makes a law¶

This module has two jobs. The first is to state the question the whole course is an answer to, in the smallest setting where it can be asked honestly — no particles, no energy, no container. The second is to find out which tools you will need to sharpen, before a physical problem is riding on them.

The puzzle: one die is lawless, a thousand dice are a law¶

Roll one fair die. No theory in physics will tell you what comes up. Newtonian mechanics governs the tumble completely, and is useless to you: the outcome depends on the throw so sensitively that predicting it would require knowing the initial conditions to a precision nobody can achieve. The single roll is, for practical purposes, lawless.

Now roll a thousand dice and average them. You will get 3.50, give or take about 0.05. Do it again — 3.50 again, within about the same margin. Do it in another room, with another set of dice, next year. Still 3.50. Roll a million and average: now the answer is 3.500 to within about 0.002, and you could calibrate an instrument against it.

Nothing about any individual die changed. Each one is exactly as unpredictable as before. Yet a quantity built entirely out of unpredictable things has become reproducible enough to be treated as a property of the collection rather than as an accident of the throw.

Every topic in this course gets looked at from four directions at once, and it is worth seeing all four on something as simple as dice before they arrive attached to thermodynamics:

View	For dice	Later, for a gas
Macroscopic	the average, 3.50	pressure, temperature, entropy
Microscopic	the individual rolls	the positions and velocities of molecules
Mathematical	means, variances, independence	the same algebra, plus differentials and asymptotics
Computational	roll many, measure the scatter	simulate the gas, measure the fluctuation

This module is also a diagnostic. Thermal physics leans hard on four skills — units, conventions, derivatives, and probability — and it leans on them from the first week. By the end of this page you should know precisely which of the four to spend a weekend on.

Predict before you calculate¶

Commit to an answer for each of these before reading on or running anything. Write them down. The point is not to be right; it is to find out which of your intuitions is about to be overturned, and which of your tools is rusty.

You average 100 fair dice. What number do you expect, and how far from it would you bet a single such experiment lands?
You now average 400 dice instead of 100. Does the typical distance from 3.5 halve, quarter, or stay about the same?
You look at the sum of the 400 dice rather than their average. Does its scatter also shrink?
Pressure is measured in pascals; energy density in joules per cubic metre. Do these have the same dimensions? If they do, is that a coincidence?
Let $f(x,y) = x^2 y$ . Is $(\partial f/\partial x)_y$ at the point $(1,1)$ the same number as $\mathrm{d}f/\mathrm{d}x$ at $x=1$ along the line $y = x$ ?

Explore the model¶

Here is the phenomenon in one picture. On the left, four hundred rolls of one die, in order. On the right, the running average of that very same sequence, settling onto the dashed line at 3.5 without anything on the left ever becoming orderly.

Left: individual die rolls, scattered between one and six with no pattern that ever emerges. Right: the running average of those same rolls, converging on the dashed line at 3.5. The left panel is the raw randomness; the right panel is the same data, accumulated. Nothing was smoothed, filtered or corrected — the only operation applied was division by the number of rolls so far.

Now watch what changes when each experiment gets bigger. Below, the experiment “average $N$ dice” is repeated 160 times at three values of $N$ , with one dot per completed experiment:

Three panels at $N = 10$ , 100 and 1000, sharing a vertical scale, with one dot per completed experiment. The vertical spread of a column of dots is the reproducibility of a single experiment, and it narrows sharply as $N$ grows. The rest of this page is about how fast.

Model specification

System: $N$ dice, each showing one of six equally likely faces; the observable is their sum or their average.
Dynamics: none — each draw is independent of every other, with no memory between them and nothing evolving in time.
Boundary: closed trivially; $N$ is fixed for one experiment and nothing is exchanged with anything.
Ensemble: every face equally probable, by assumption rather than by proof.
Ignored: all physics of real dice — the tumble, the bounce, pip asymmetry, air drag, and the deterministic chaos that is the actual reason a throw looks random.
Valid when: the randomness is well described by independent uniform draws; a seeded pseudo-random generator is, by construction.
Failure modes: correlated draws, loaded dice, and any question about a single roll beyond its distribution.

Every simulation in this course carries a block like that one, and it is worth knowing why before you meet a harder example. Two of its seven fields are doing unusual work here. Dynamics: none is unusual — most models in this course evolve something — and it is the reason dice are the right starting point: whatever makes the average settle down, it cannot be a dynamical mechanism, because there is no dynamics. And Ensemble looks like an odd word for dice, but the entry is the honest one: “every face equally probable” is a postulate we are choosing, not a fact we established. Module 8 makes exactly this kind of postulate about the microstates of a gas, and the whole of statistical mechanics rests on it.

Run the laboratory now, then come back. Three things are worth doing before you read on:

Watch the running average for $N = 10$ , then for $N = 10{,}000$ . Note that the curve is just as jagged at the start in both cases.
Re-run the same experiment with a different seed, several times, and watch how much the final answer moves.
Load the die — make a six twice as likely as any other face — and see which parts of the picture change and which do not.

Derive the result¶

We now do the calculation the simulation is imitating. It needs no calculus and no central limit theorem — only the definitions of mean and variance, plus one property of independent quantities.

System. $N$ dice. Independent variables. $N$ , and the distribution of one die. Assumption. The draws are independent — the outcome of one tells you nothing about any other.

Definitions, and one die¶

For one fair six-sided die, $p_k = 1/6$ for $k = 1,\dots,6$ , so

\mu_1 = \frac{1+2+3+4+5+6}{6} = \frac{7}{2} = 3.5 , \qquad \langle X^2 \rangle = \frac{1+4+9+16+25+36}{6} = \frac{91}{6} ,

(2)

\sigma_1^2 = \frac{91}{6} - \left(\frac{7}{2}\right)^2 = \frac{35}{12} \approx 2.917 , \qquad \sigma_1 \approx 1.708 .

(3)

No approximation anywhere: those are the exact properties of one die.

The one property that does all the work¶

Means always add; variances add for independent quantities

For any two random variables, $\langle X + Y \rangle = \langle X \rangle + \langle Y \rangle$ . For the variance, expanding the definition gives

\operatorname{Var}(X+Y) = \operatorname{Var}(X) + \operatorname{Var}(Y) + 2\big( \langle XY \rangle - \langle X \rangle \langle Y \rangle \big) .

(4)

The last bracket is the covariance. It vanishes exactly when $X$ and $Y$ are independent, and then — and only then —

\operatorname{Var}(X+Y) = \operatorname{Var}(X) + \operatorname{Var}(Y) .

(5)

Repeating the argument $N-1$ times extends it to any number of mutually independent terms.

Notice what the proof did not use: nothing about dice, nothing about six faces, nothing about the shape of the distribution. It needs only that each quantity has a finite variance and that they are mutually independent. That generality is why this one calculation will still be doing the work when the random quantities are molecular velocities (module 4) or occupation numbers (module 8) — and the independence hypothesis is exactly what fails in the systems where thermal physics gets hard.

The consequence¶

Let $S_N = X_1 + \dots + X_N$ be the sum of $N$ independent dice, and $\bar{A}_N = S_N / N$ their average. Means add, so both have the expected value you would guess. Variances add, so $\operatorname{Var}(S_N) = N \sigma_1^2$ ; and since scaling a quantity by $1/N$ scales its variance by $1/N^2$ ,

\boxed{\; \sigma_{S_N} = \sigma_1 \sqrt{N} , \qquad \sigma_{\bar{A}_N} = \frac{\sigma_1}{\sqrt{N}} \; }

(6)

Read those two together, because they are the answer to prediction 3 and they point in opposite directions. The sum gets noisier in absolute terms as $N$ grows; the average gets steadier. Both statements come from the same $N\sigma_1^2$ , and confusing them is the usual error.

Why the honest statement is a ratio¶

The scatter $\sigma$ carries the units of whatever was measured. That makes it useless as a measure of “steadiness” on its own: a scatter of 0.05 is enormous for a quantity of size 0.01 and invisible for a quantity of size 10⁶. The quantity that means something is the dimensionless ratio

\frac{\sigma_{\bar{A}_N}}{\mu_1} = \frac{\sigma_1}{\mu_1} \, N^{-1/2} \qquad\text{with}\qquad \frac{\sigma_1}{\mu_1} = \frac{\sqrt{35/12}}{7/2} \approx 0.488 \ \text{for a die.}

(7)

That is the $N^{-1/2}$ law, with its coefficient. For $N = 100$ it gives about $4.9\%$ ; for $N = 10^4$ , about $0.49\%$ ; for $N = 10^{23}$ , about 10^-12 — which is why a gas has a pressure and a single die does not have an average.

This habit — comparing fractions rather than absolute differences — is not a stylistic preference. Energies in this course are routinely $10^{-18}\,\mathrm{J}$ , and any test that asks whether two of them agree “to within 10^-12” will answer yes no matter what the physics did.

Two things have deliberately not been shown. We have not shown what shape the distribution of $\bar{A}_N$ takes — that it becomes a bell curve is the central limit theorem, and it is module 3’s result, not ours. And we have not shown that the average is guaranteed to be close, only that its typical distance shrinks; turning that into a genuine probability bound is the advanced section below.

Verify computationally¶

Deriving a formula and trusting it are different things. The laboratory measures the law rather than assuming it, and every check below is also a test in the project’s test suite, so the claims on this page cannot silently rot.

1. The exponent. Measure the relative spread of $\bar{A}_N$ at $N \in \{4, 16, 64, 256, 1024\}$ , fit a straight line to $\log \sigma$ against $\log N$ , and read off the slope. It comes out at -0.5.

2. The coefficient. A fitted exponent alone is weak evidence — many wrong models produce a slope near $-\tfrac{1}{2}$ . So we also check the prefactor: measured spread times $\sqrt{N}$ must return $\sigma_1/\mu_1 \approx 0.488$ , and it does, to a few percent.

3. Sum versus average. The same data, read two ways: the absolute scatter of the sum is fitted at +0.5 while the relative scatter of the average is fitted at -0.5.

4. The mechanism. This is the one worth running yourself.

The three habits that make those numbers trustworthy are worth naming, because every later module uses them:

Every random function takes its generator as an argument. Nothing in thermolab reaches for global random state, so any result you see can be reproduced exactly.
Stochastic results are compared with an error bar, never bare. A measurement that “agrees with theory” without an uncertainty attached has not been compared with anything.
Agreement is judged as a fraction. See the previous section for what goes wrong otherwise.

Transfer the idea¶

The algebra above is three lines long and it is load-bearing for the rest of the course.

Module 3 — probability and emergence. The same sum of independent variables, asking the question we deferred: not how wide the distribution of an average is, but what shape it takes. The answer is the central limit theorem.
Module 4 — kinetic theory. The random variables become molecular velocities and the average becomes the pressure on a wall. The relative fluctuation comes out as $N^{-1/2}$ , from this same argument, and you can watch it on a simulated pressure gauge: the microscopic origin of pressure.
Module 8 — entropy. The objects being counted become microstates. The peak of the multiplicity narrows as $N^{-1/2}$ , which is why an isolated gas never gathers in one corner: entropy and multiplicity.

The road ahead¶

#	Module	The question it answers
0	Orientation	How can many unpredictable things add up to a reliable one?
1	Thermal equilibrium	What does it mean for two bodies to be “at the same temperature”?
2	Equations of state	How are pressure, volume and temperature tied together?
3	Probability and emergence	What shape does an average of many random quantities take?
4	Kinetic theory	Where does pressure come from, mechanically?
5	First law	Why does the energy delivered depend on the route taken?
6	Thermodynamic processes	What distinguishes the routes from one another?
7	Second law	Why can no engine do better than a certain efficiency?
8	Entropy	Why does anything happen in one direction and not the other?
9–12	Fundamental relation, potentials, ensembles, partition functions	How is all of the above one theory rather than a list of identities?
13–18	Advanced	Chemical potential, phases, transitions, radiation, quantum statistics, transport

Modules 4, 5 and 8 are written; the rest are on their way. Nothing on this page depends on reading them in order.

Your readiness map¶

Match your five predictions against the answers, and use the quiz below as the real test. Then spend your time where this table sends you.

Skill	Predicted in	Tested by	Refresh at
Mean and variance of a random variable	1	Q2	Derive, definitions box
The $N^{-1/2}$ law, and sum versus average	2, 3	Q1, Q3, Q9	Derive, boxed result
Dimensional analysis and SI units	4	Q4	Refresher — dimensional analysis
Partial derivatives and the total differential	5	Q5, Q6	Refresher — partial derivatives
Exact versus inexact differentials	—	Q7	Refresher — exact and inexact
The course’s conventions	—	Q8, Q10	Conventions

Two reference pages sit outside the module sequence and are meant to be returned to rather than read once: conventions, which lists the choices this course makes and sticks to, and the mathematics refresher, which is the toolkit it assumes.

Check your understanding¶

Score yourself honestly — this is a diagnostic, not an exam, and every item’s feedback names the section to revisit. The readiness map above turns your wrong answers into a study plan.

Averaging 100 fair dice gives an answer that typically lands within about 0.17 of 3.5. You now average 10,000 dice instead. Which statement is correct?
A. About 0.017 — the typical distance falls as N^(-1/2), so a hundredfold larger N shrinks it tenfold. B. About 0.017 — the later rolls compensate for the early deviations and cancel them out. C. About 0.0017 — the typical distance falls as 1/N. D. About 0.17 — averaging more dice does not change how far a single experiment strays.
Check your answer
Correct answer: A — About 0.017 — the typical distance falls as N^(-1/2), so a hundredfold larger N shrinks it tenfold.
- A. Correct, in both the number and the reason. sigma of the average is sigma_1/sqrt(N), so multiplying N by 100 divides the scatter by 10.
- B. The number is right and the mechanism is wrong, which is worth more than half the marks in the wrong direction. Dice have no memory: after a run of sixes the next roll still averages 3.5, and the laboratory measures exactly this. The early excess is never cancelled — it is divided by an ever-larger N until it stops mattering. Convergence is dilution, not compensation.
- C. This is the most common wrong scaling. Variances add, so the variance of the average falls as 1/N — and the scatter is the square root of the variance, hence N^(-1/2). Re-read the boxed result in the derivation section.
- D. If this were true no macroscopic quantity would ever be reproducible. Run the laboratory’s second animation: the scatter of the dots visibly narrows as N grows.
Compute the variance of a single roll of a fair six-sided die. Give a decimal number.
Check your answer
Correct answer: 2.917 (± 0.05)
= 7/2 and <X^2> = 91/6, so the variance is 91/6 - 49/4 = 35/12 = 2.9167. The standard deviation is its square root, about 1.708 — and the dimensionless ratio sigma_1/mu_1 is about 0.488, the coefficient in front of N^(-1/2). If you got 1.708 you computed the standard deviation instead; if you got 17.5 you forgot to subtract the square of the mean.
As N grows, what happens to the absolute scatter of the SUM of N dice, and to the relative scatter of their AVERAGE?
A. The sum’s absolute scatter grows as sqrt(N); the average’s relative scatter falls as N^(-1/2). B. Both shrink, since averaging always suppresses randomness. C. Both grow, since more dice means more opportunities for an extreme result. D. The sum’s scatter stays constant; the average’s falls as 1/N.
Check your answer
Correct answer: A — The sum’s absolute scatter grows as sqrt(N); the average’s relative scatter falls as N^(-1/2).
- A. Right, and both come from the single statement that variances of independent quantities add. The sum becomes less predictable in absolute terms at the same time as the average becomes more predictable in relative terms; there is no contradiction, because they are different questions about the same data.
- B. The average shrinks, but the sum does not. Var(S_N) = N sigma_1^2, so the sum’s standard deviation is sigma_1 sqrt(N), which grows without bound. Only after dividing by N does the scatter shrink.
- C. True of the sum, false of the average. More dice does make an extreme total larger in absolute terms — but the total it is compared against grows faster (as N rather than sqrt(N)), so the fractional deviation of the average falls.
- D. Neither half is right. Both exponents follow from Var(S_N) = N sigma_1^2: take the square root for the sum, then divide by N for the average.
Which of these expressions could be a molecular speed? (k_B is the Boltzmann constant, T a temperature, m a particle mass.)
A. sqrt(3 k_B T / m) B. 3 k_B T / m C. sqrt(3 k_B T m) D. sqrt(3 k_B T / m^2)
Check your answer
Correct answer: A — sqrt(3 k_B T / m)
- A. Correct. k_B T is an energy, so k_B T / m has dimensions of energy per mass, which is (length/time)^2; the square root is a speed. Note what this check cannot tell you: whether the numerical factor is 1, 2 or 3. Dimensions reject wrong formulas; they never produce right ones.
- B. This has dimensions of speed squared, not speed — energy per unit mass. You are one square root away.
- C. Multiplying by the mass rather than dividing gives energy times mass, whose square root is a momentum, not a speed. Check by writing out kg m^2 s^-2 times kg.
- D. This gives speed divided by the square root of a mass. A useful habit: reduce every candidate to powers of kg, m, s and K before deciding.
For f(x, y) = x^2 y, evaluate the partial derivative of f with respect to x at constant y, at the point (x, y) = (2, 3).
Check your answer
Correct answer: 12 (± 0.01)
Holding y fixed, d/dx of x^2 y is 2xy, which is 223 = 12. If you got 4 you differentiated x^2 and dropped the factor y; if you got 4 for the other partial you computed the derivative with respect to y instead, which is x^2.
What is the total differential df of f(x, y) = x^2 y?
A. df = 2xy dx + x^2 dy B. df = x^2 dx + 2xy dy C. df = 2xy dx D. df = 2xy + x^2
Check your answer
Correct answer: A — df = 2xy dx + x^2 dy
- A. Correct — each partial derivative multiplies the differential of its own variable. This is the object thermodynamics manipulates constantly; every Maxwell relation in module 10 is a statement about differentials of exactly this kind.
- B. The two coefficients have been swapped. The coefficient of dx must be the derivative with respect to x, at constant y, which is 2xy.
- C. This is the response to changing x alone. The total differential accounts for both variables changing; dropping the dy term is exactly the error that makes a partial derivative and a total derivative disagree.
- D. A differential form must carry the differentials dx and dy. Without them this is just a sum of two derivatives, and it cannot be integrated along a path.
Is the differential form w = y dx (that is, M = y and N = 0) exact?
A. No — dM/dy = 1 but dN/dx = 0, so no potential function exists and the integral depends on the path. B. Yes — it is the differential of xy. C. Yes — every form built from smooth functions is exact. D. It cannot be decided without first choosing a path.
Check your answer
Correct answer: A — No — dM/dy = 1 but dN/dx = 0, so no potential function exists and the integral depends on the path.
- A. Correct. The mixed-partials criterion settles it without integrating anything, and the integrals confirm it: from (0,0) to (1,1) this form gives 0, 1/2 or 1 depending on the route. In module 5 this is the mathematics behind heat and work being path functions.
- B. d(xy) = y dx + x dy, which has an extra term. Differentiate your candidate potential and compare with the form you were given; the x dy term has nowhere to come from.
- C. Smoothness is not the issue. Exactness is a condition on how M and N are related to each other, and most smooth forms fail it.
- D. This has it backwards: the criterion is precisely what lets you decide before choosing any path, and its whole content is whether the choice of path will matter.
A gas is compressed: the surroundings push a piston inward and the volume decreases. In the sign convention this course uses everywhere, the work term appearing in the first law is which of the following?
A. Positive, because energy was transferred into the gas. B. Negative, because the volume of the gas decreased. C. Negative, because work always enters the first law with a minus sign. D. Zero, because compression transfers heat rather than work.
Check your answer
Correct answer: A — Positive, because energy was transferred into the gas.
- A. Correct. This course writes the first law with the work done on the system, so everything that enters the system adds to its internal energy and there is no minus sign to remember. Compression puts energy in, so the term is positive.
- B. The sign tracks the direction of energy flow, not the direction of the volume change. Compressing the gas does work on it, so energy went in.
- C. That is the other common convention, in which the work counted is the work done by the system. Both conventions are correct physics and this course uses the other one — see the conventions page, which puts the two side by side.
- D. Pushing a piston is mechanical energy transfer, which is work by definition. Heat may also flow, but that is a separate term in the same equation.
Before working anything out: which is more predictable — the result of one die roll, or the average of ten thousand? Say by roughly how much, and name the mechanism responsible in your own words.
Check your answer
One roll strays from 3.5 by about 1.7 typically; the average of 10,000 strays by about 0.017, a hundredfold improvement. The mechanism is the point. A good answer says that each new roll contributes a fixed amount of scatter to the total while contributing a full unit to the count, so the total’s scatter grows only as sqrt(N) while the total itself grows as N. An answer that says “the deviations cancel out” is describing compensation, which does not happen: nothing about the dice changes after a run of sixes. If you wrote that, run the laboratory’s conditioning experiment before moving on.
In two or three sentences: what does the slashed differential in “delta Q” mark, and why does this course label some boxed statements “theorem” and others “numerical observation”?

Exam-style problems for this module — the ones worth doing with a pen — are collected in the problem set.

Explain it in your own words¶

Answer in a few sentences each. These are the questions that reveal whether the ideas landed; no number will save you.

A casino’s profit on any single spin is unpredictable, yet its annual revenue is budgeted to within a fraction of a percent. Explain why, without letting the word “average” do unexamined work — your answer must say what role the number of spins plays.
A student says: “The average settles down because a run of high rolls gets cancelled by a run of low ones later.” Repair this sentence, and say precisely what is wrong with the version given.
Why is a dimensionless ratio the only honest way to say that something is “steady”? Give an example where an absolute scatter would mislead you.
Look back at your five predictions. Which one was wrong, or which took you longest? Name the specific tool involved, and where you will go to sharpen it.

Advanced: from “typically close” to “almost certainly close”¶

The derivation showed that the typical distance of $\bar{A}_N$ from $\mu$ shrinks as $N^{-1/2}$ . That is not the same as a guarantee. Turning it into one takes two short steps.

Markov’s inequality. For a random variable $Y \ge 0$ and any $a > 0$ , $\langle Y \rangle \ge a \, P(Y \ge a)$ , because the sum defining $\langle Y\rangle$ can only lose by discarding every outcome below $a$ and replacing the rest by $a$ . Hence

P(Y \ge a) \le \frac{\langle Y \rangle}{a} .

(8)

Chebyshev’s inequality. Apply Markov to $Y = (X - \mu)^2$ and $a = \varepsilon^2$ :

P\big(|X - \mu| \ge \varepsilon\big) \le \frac{\sigma^2}{\varepsilon^2} .

(9)

Applied to the average of $N$ independent dice, whose variance is $\sigma_1^2/N$ , this gives

P\big(|\bar{A}_N - \mu_1| \ge \varepsilon\big) \le \frac{\sigma_1^2}{N \varepsilon^2} \;\xrightarrow[N \to \infty]{}\; 0 \qquad\text{for every fixed } \varepsilon > 0 .

(10)

That is the weak law of large numbers, and it is now a genuine probability statement: fix any tolerance you like, however tight, and the chance of missing it can be driven below any bound by taking $N$ large enough. Note what carried the argument — only the existence of a finite variance. No bell curve, no shape assumption, nothing about dice in particular.

Why variance, and not some other measure of spread? The mean absolute deviation $\langle |X - \mu| \rangle$ is arguably a more natural measure of typical distance, and it is the wrong tool here for one decisive reason: it does not add. Variances of independent quantities add exactly, which is what let us go from one die to $N$ in a single line. That additivity is the whole reason the second moment, rather than the first, sits at the centre of statistical physics — and it is why you will meet $\sigma^2$ again as an energy fluctuation tied to a heat capacity in module 18.