Guide
How do you calculate measurement uncertainty for a machined dimension with a simple uncertainty budget?
Measurement uncertainty is the range around a reading within which the true size of the feature can be expected to lie, at a stated confidence. You calculate it by listing every source of doubt in the measurement, converting each to a standard uncertainty, combining them as a root sum of squares and multiplying by a coverage factor, usually k = 2, to get the expanded uncertainty U.
By Saola Manufacturing engineering team · Published · Updated
The usual reason an engineer goes looking for this calculation is a dispute. A bore passes on the supplier’s inspection report, fails at receiving inspection, and both sides are sure their gauge is right. An uncertainty budget settles that argument with arithmetic instead of opinion: it tells you how far apart two honest measurements of the same part can be, whether a reading near a limit proves anything, and which term to attack if the measurement is not good enough for the tolerance. This guide works one budget for one bore, end to end. How fine a gauge needs to be for a given tolerance, the 10:1 and 4:1 ratios, is covered in the 10:1 rule guide and is not repeated here.
On this page
Why does the same dimension pass at the supplier and fail at receiving inspection?
Because every measurement carries uncertainty, and when the true size sits near a tolerance limit, two correct measurements can land on opposite sides of it. The part has not changed; the two readings simply overlap the limit from different directions.
The pattern is easy to recognize. The disputed readings sit close to one limit rather than far outside it. The supplier and the customer use different instruments, different masters or different rooms. Re-measuring the same part at receiving gives a spread of results, some in and some out. And the dispute is concentrated on a few tight features, usually bores, fit diameters and thin-wall sizes, while the general-tolerance dimensions agree.
None of that means anyone measured carelessly. It means the drawing tolerance and the measurement uncertainty are of a similar order, and nobody agreed in advance how to treat a reading that falls in the zone where the uncertainty straddles the limit. The international method for quantifying that zone is the Guide to the Expression of Uncertainty in Measurement, JCGM 100:2008, and the simplified version written for shop-floor dimensional work is the ASME technical report B89.7.3.2, which works directly in units of length and skips the partial derivatives of the full method. The worked example below follows that simplified approach.
Which sources of uncertainty matter most on a machined dimension?
On a close-tolerance machined feature the largest terms are usually repeatability, the form of the part itself, temperature and the calibration of the master the gauge is set to. The resolution of the display, the figure most people check first, is usually one of the smallest.
Ranked roughly from most to least often decisive, these are the inputs to list for a measured size:
- Repeatability. The scatter when one person measures the same feature several times. It is a Type A term, estimated from the standard deviation of repeat readings, and it captures probe seating, rocking a bore gauge through center and reading the indicator.
- Part form. A two-point gauge on an out-of-round or tapered bore reports one diameter out of many. The supplier may measure across the lobes and the customer between them, and both readings are true.
- Temperature. Per ISO 1:2022, drawing sizes apply at 20 °C, so a warm part, or a part and master at different temperatures, reads a different size. Aluminum moves roughly twice as much as steel per degree, which is why aluminum parts are the usual victims.
- Master or reference calibration. A bore gauge is a comparator; it is only as good as the setting ring and the uncertainty printed on that ring’s calibration certificate.
- Reproducibility between operators. The shift between people, taken from a gauge R&R study. If one person does all the measuring, this term can be left out of that site’s budget but not out of a comparison between sites.
- Alignment and fixturing. Centering and squaring the gauge in the bore, or how the part sits on a surface plate, adds error that the instrument data sheet does not show.
- Resolution. The last digit of the display. Treated as a rectangular distribution, it contributes the step size divided by twice the square root of three, which is small once the step is fine.
A useful habit is to write the list before measuring anything. Every term you cannot put a number on is a term the other party may be quietly including, and that is often where the disagreement comes from.
How do you build the uncertainty budget step by step?
Give each source a limit or a standard deviation, divide by the divisor for its distribution to get a standard uncertainty, square and add the terms, take the square root, and multiply by k = 2. The table below does that for a bore measured with a bore gauge set to a master ring.
The divisors come from the Type B rules in NIST Technical Note 1297: a value quoted with a coverage factor on a calibration certificate is divided by that factor; a value known only to lie somewhere between plus and minus a limit, such as resolution or an uncorrected temperature effect, is treated as rectangular and divided by the square root of three; and a standard deviation from repeat readings is used as it is. Independent standard uncertainties are combined as a root sum of squares, the law of propagation in JCGM 100:2008 for inputs already in units of length, and the result is multiplied by a coverage factor; per the NIST page on expanded uncertainty, k = 2 gives a level of confidence of approximately 95 percent when the normal distribution applies.
The example is a 20 mm bore toleranced H7, which per ISO 286-2 gives a total band of 21 µm above basic size. Every input value is illustrative, chosen to be realistic for a dial bore gauge with a fine digital indicator; replace each one with your own certificate, study and room data.
| Source | Basis of the estimate | Distribution and divisor | Standard uncertainty u (µm) | u squared (µm²) |
|---|---|---|---|---|
| Master ring calibration | Certificate states U = 1.0 µm at k = 2 | Normal, divide by 2 | 0.50 | 0.25 |
| Indicator resolution | 1 µm step, so a half-width of 0.5 µm | Rectangular, divide by √3 | 0.29 | 0.08 |
| Repeatability | Standard deviation of 10 repeat readings = 1.2 µm | Type A, used as is | 1.20 | 1.44 |
| Reproducibility (operators) | Operator term from a gauge R&R study = 0.8 µm | Type A, used as is | 0.80 | 0.64 |
| Temperature (uncorrected) | Part and ring up to 2 °C apart; aluminum about 23 × 10⁻⁶ per °C over 20 mm gives a half-width of 0.92 µm | Rectangular, divide by √3 | 0.53 | 0.28 |
| Alignment and centering | Estimated half-width 1.0 µm | Rectangular, divide by √3 | 0.58 | 0.33 |
| Part form (two-point on lobed bore) | Estimated half-width 1.5 µm | Rectangular, divide by √3 | 0.87 | 0.75 |
| Combined standard uncertainty uc | Square root of the sum of the squares | Root sum of squares | 1.94 | 3.78 |
| Expanded uncertainty U | U = k × uc | k = 2, about 95 percent coverage | 3.9 | – |
Read the budget the way you would read a cost breakdown. Repeatability alone is more than a third of the total variance, and part form and operators together are about another third, while resolution barely registers. So a finer indicator would change almost nothing, and the levers worth pulling are averaging several readings, a better centering method, or a form check that tells you whether the two-point reading is meaningful at all. Two cautions apply. Terms are only added as squares when they are independent; if two sources move together, such as part and ring temperature, model the difference once rather than each separately. And if the reported value is the mean of several readings, the repeatability term shrinks with the square root of the number of readings, which is the cheapest improvement available.
How do you use the expanded uncertainty to judge a pass, a fail or a disputed reading?
Compare U with the tolerance, then place each reading against the tolerance limits widened or narrowed by a guard band. A reading inside the guard band proves nothing either way until the decision rule says how to treat it.
First, the size check. The illustrative budget above, built per ASME B89.7.3.2, gives an expanded uncertainty of about 3.9 µm against the 21 µm H7 band from ISO 286-2, a test uncertainty ratio of only about 2.7 to 1 when the band is divided by twice U. That is below the 4:1 convention discussed in the 10:1 rule guide, which already warns that readings near either limit will be contested.
Second, the guard band. Per ISO 14253-1:2017, the default for GPS drawings, the supplier proves conformance only inside the tolerance narrowed at each end, and the customer proves nonconformance only outside the tolerance widened at each end; the 2017 edition ties the default guard band to a 95 percent conformance probability rather than to the full k = 2 uncertainty used in the 1998 edition, which for a normal distribution works out to about 1.65 standard uncertainties. Check the edition and rule your contract cites before using that figure. Third, when the two sides disagree, test whether their readings are even inconsistent: divide the difference between the two readings by the square root of the sum of the two expanded uncertainties squared. A result at or below one, the normalized error check used in laboratory proficiency testing, says the two measurements agree within their uncertainty and the dispute is about the decision rule, not about the part.
| Where the reading falls | Example reading (mm) | What it proves | What to do |
|---|---|---|---|
| Supplier: inside the acceptance zone (20.0032 to 20.0178) | 20.0150 | Conformance proven by the supplier | Accept, provided the supplier’s budget holds up; if receiving still disagrees, check the receiving method first |
| Supplier: inside tolerance but within 3.2 µm of a limit | 20.0190 | Nothing proven; reading is in the guard band | Apply the agreed rule; under the ISO default the supplier cannot claim conformance |
| Receiving: outside tolerance by less than 5.0 µm | 20.0240 | Nonconformance not proven | Do not reject on this reading alone; re-measure with a lower-uncertainty method |
| Receiving: outside tolerance by more than 5.0 µm | 20.0270 | Nonconformance proven | Reject or hold for disposition and corrective action |
| Two readings differ by more than the combined U (7.2 µm) | 20.0150 vs 20.0250 | At least one budget is incomplete | Stop sorting; compare masters, temperature and method on the same marked parts |
In the example the supplier reads 20.019 and receiving reads 20.024, a difference well inside the combined uncertainty of the two methods. Both readings are honest, neither proves the part good or bad, and the fix is a reference measurement on the same parts, not a debate about who is right.
How do you shrink the uncertainty when it is too large for the tolerance?
Attack the largest term in the budget first, because the root sum of squares means small terms hardly matter. If no practical change gets the measurement down far enough, widen the tolerance where function allows or agree a reference method for that one feature.
- Repeatability dominates: report the mean of several readings, add a centering stop or a self-centering gauge head, and train the rocking technique on a bore gauge.
- Part form dominates: measure in several orientations and report the range, or specify the size condition and a roundness or cylindricity tolerance so both sides measure the same thing.
- Temperature dominates: let parts soak to room temperature next to the master before measuring, and record the part temperature. The NIST history of the 20 °C reference explains why this term never goes away on its own.
- Calibration dominates: use a setting ring with a smaller certificate uncertainty and keep its calibration current and traceable.
- Operators dominate: run a gauge R&R, write the method down, and use one trained operator for the disputed feature until the method is fixed.
When the tolerance itself is the problem, look at what the feature does. A clearance bore toleranced like a bearing fit costs more to make and more to prove; the tight tolerance cost guide shows where loosening a band is usually safe.
What should the drawing, RFQ and PO say so the dispute does not recur?
Flag the few characteristics where uncertainty is close to the tolerance, state the decision rule, and ask the supplier to name the instrument and its expanded uncertainty for each one before production. Agree in advance which measurement is the reference if the two sides disagree.
- Drawing. Balloon the critical features and mark them as key characteristics. On an ASME print, add a decision-rule note (ASME B89.7.3.1 gives guidance on how to state one), because the ISO 14253-1 default should not be assumed to carry over; on an ISO GPS print, ISO 14253-1 applies unless the drawing or contract says otherwise.
- RFQ. For each flagged balloon, ask which instrument accepts it, its expanded uncertainty and coverage factor, and whether the reading is taken at room temperature after the part has soaked.
- Purchase order or quality agreement. State the decision rule, the reference temperature, how many readings make up a reported value, and the referee method, for example an accredited laboratory measurement on retained parts, if supplier and receiving results conflict.
- Inspection report. Require the instrument used beside each reading, as the first article inspection guide recommends, and on the flagged balloons add the instrument’s identity and calibration due date.
If a lot is already in dispute, work through it in this order rather than re-measuring the whole lot:
- Quarantine the lot and mark five to ten parts, with the measuring positions and orientations marked on each part.
- Let the marked parts soak beside the master at both sites, record the part temperature, and measure the same marked positions with the reported value defined the same way (single reading or mean of a stated number).
- Check that both sides measure the same thing: two-point size versus a fitted or functional size, before or after coating, and the same setting master value.
- Compute the normalized error for each marked part. If it is at or below one, the measurements agree and only the decision rule needs settling; if it is above one, compare the two budgets line by line and look for the missing term.
- If the difference persists, send the marked parts for a referee measurement with a lower stated uncertainty, and disposition the lot against that result.
Open a formal 8D report on the product once the referee measurement shows the parts really are out; if it shows a measurement problem instead, the corrective action belongs to the method at whichever site the budget was incomplete.
How is measurement uncertainty handled on a Saola order?
Machined dimensions are measured at our lead CNC plant, where the work is done, and the instrument for each flagged characteristic is agreed in the quote stage. Characteristics produced or changed at partner plants we manage, such as cast surfaces or coated diameters, are handled by the plant named on the quote.
Saola Manufacturing is the brand of Aximora Global Trading Co., Ltd., which places and manages metal-parts work. CNC turning and CNC milling run at our lead CNC plant, T.K Viet Nam Precision Company Limited, whose inspection equipment includes a 2D vision measuring machine, a height gauge and a hardness tester alongside calipers and go/no-go gauges. The published equipment list does not include a coordinate measuring machine, air gauging or a temperature-controlled inspection room, so if your budget shows a feature needs one of those, name the balloon and the decision rule in the RFQ and the quote will say how that feature will be measured or flag that it needs a reference measurement elsewhere. Die casting, stamping and surface finishing are done by partner plants we manage, and each quote states the production site and country of origin. For a coated bore, the drawing should say whether the size applies before or after coating, because the two are measured at different points in the route. ISO 9001 certification is in progress, not held. Send the drawing with the flagged balloons through the quote form.
Frequently asked
What is the difference between standard uncertainty and expanded uncertainty?
Standard uncertainty is one standard deviation of the doubt in a result. Expanded uncertainty U is that value multiplied by a coverage factor, usually two, so the stated interval covers roughly 95 percent of the values the measurand could reasonably have. Always report k with U.
Is measurement uncertainty the same as measurement error?
No. Error is the difference between a reading and the true value, which is never known exactly. Uncertainty is the size of the doubt about that reading. A known, correctable error, such as a gauge offset, should be corrected, and only the doubt about the correction goes into the budget.
Do I need an uncertainty budget for every dimension on the drawing?
No. General-tolerance dimensions measured with a gauge far finer than the band do not need one. Build budgets only for the few features where the measurement is close to the tolerance or where supplier and customer results have disagreed.
Can gauge R&R results replace an uncertainty budget?
Not completely. A gauge R&R study supplies the repeatability and operator terms, often the largest ones, but it does not include master calibration, temperature or part form unless the study was designed to capture them. Use the R&R numbers as two rows of the budget; the gage R&R guide covers how to judge the study itself.
Why divide by the square root of three for resolution and temperature?
When all you know is that a value lies somewhere between two limits with no value more likely than another, the rectangular distribution applies, and its standard deviation is the half-width divided by the square root of three. NIST Technical Note 1297 gives this rule for Type B estimates.
Sources
- JCGM 100:2008, Evaluation of measurement data — Guide to the expression of uncertainty in measurement (GUM), Joint Committee for Guides in Metrology, BIPM, 2008
- ASME B89.7.3.2-2007 (R2021), Guidelines for the Evaluation of Dimensional Measurement Uncertainty (Technical Report), ASME, 2007
- Type B evaluation of standard uncertainty, from NIST Technical Note 1297, NIST, 1994
- Expanded uncertainty and coverage factor, from NIST Technical Note 1297, NIST, 1994
- ISO 14253-1:2017, GPS — Inspection by measurement of workpieces and measuring equipment — Part 1: Decision rules for verifying conformity or nonconformity with specifications, ISO, 2017
- ISO 1:2022, GPS — Standard reference temperature for the specification of geometrical and dimensional properties, ISO, 2022
- ISO 286-2:2010, GPS — ISO code system for tolerances on linear sizes — Part 2: Tables of standard tolerance classes and limit deviations for holes and shafts, ISO, 2010
- Doiron, 20 °C — A Short History of the Standard Reference Temperature for Industrial Dimensional Measurements, Journal of Research of NIST, 2007