Recording readings

How many digits

Record the digits the instrument gives, no more and no fewer. A digital caliper with 0.01 mm resolution that shows 12.00 is recorded as 12.00 mm, not 12 mm: the two zeros say that the instrument resolved hundredths. A 0.02 mm vernier caliper gives readings whose last digit is even, such as 24.98 mm.

Do not add digits the instrument cannot give. Most physics lab handouts ask for one estimated digit on a micrometer screw gauge (0.001 mm); follow yours. A vernier caliper is read only to its least count (see the physics lesson).

Rounding rules (GB/T 8170-2008)

When a value must lose digits, for example a mean or a converted unit, look at the leftmost digit to be dropped:

• less than 5: drop it (24.9849 → 24.98);
• more than 5: add one to the last kept digit (24.9861 → 24.99);
• 5 followed by non-zero digits: add one (24.98501 → 24.99);
• 5 with nothing or only zeros after it: make the last kept digit even (24.985 → 24.98, 24.975 → 24.98).

Round once, from the original value. 15.4546 rounded to a whole number is 15; rounding step by step (15.455 → 15.46 → 15.5 → 16) gives the wrong answer, which the standard forbids.

Some texts and lab manuals round a 5 up in every case; the NCERT Class XI physics lab manual does so in its logarithm-table example. Use the rule your course or your standard names; in Chinese standards and test reports it is GB/T 8170.

Rounding to half a unit

GB/T 8170 also rounds to steps of 0.5 of a unit: double the value, round it, then halve it. Example, to steps of 0.0005 in: 12 mm = 0.472 44 in, doubled 0.944 88, rounded to 0.945, halved 0.4725 in. A digital caliper set to inches also shows steps of 0.0005 in; how it treats an exact half step is up to its maker.

Comparing with a limit

Unless the specification says otherwise, GB/T 8170 compares the full value with the limit (unrounded, or rounded and marked as rounded up or down); any excess, however small, does not conform. A value rounded first can hide a small excess.

A result with its uncertainty

The GUM (JCGM 100:2008, 7.2.6) says it usually suffices to quote an uncertainty to at most two significant digits, and that the result is rounded to be consistent with it. Example: a mean of 24.9867 mm with an expanded uncertainty of 0.023 mm is written 24.987 mm, U = 0.023 mm (k = 2).

What goes in the record

1. The value with its unit: 12.00 mm.
2. What was measured: the part and the feature, such as “pin, outside diameter, middle”.
3. The instrument, by its ID, so the reading can be traced to that instrument’s calibration.
4. The date and who measured.

In KELIBRON

The lab’s displays show what the instrument resolves: the digital caliper shows two decimals in mm and steps of 0.0005 in when switched to inches; the vernier caliper reads in steps of 0.02 mm. Its readings come from a model, so they are for practice, not for a real record.