Gauge Repeatability and Reproducibility (Gauge R&R)

1. Theory

• Definition: A gauge repeatability and reproducibility (R&R) study measures several parts several times with several operators or tools and splits the observed variance into measurement-system and part-to-part components (MathWorks; MetricGate).
• Repeatability: Variation when the same operator or tool measures the same part repeatedly (MathWorks).
• Reproducibility: Variation between operators or tools, including their interaction with parts (MathWorks; MetricGate).
• Capability metrics: Percent study variation, number of distinct categories and precision-to-tolerance ratio judge whether the gauge can see process variation and specifications (MetricGate; MathWorks).

2. Working Principle

• Design: Typically 10 parts spanning the process range, 3 operators or tools, 2 or 3 replicates in random order (MetricGate; Penn State STAT 503).
• ANOVA: Mean squares give variance components through their expected values (Penn State STAT 503).
• Metrics: study variation, ndc and P/T with a stated sigma multiplier (MathWorks; MetricGate).
• Decision: Accept, accept with improvement, or reject the measurement system (MetricGate).

3. Applications

• Tool matching: Reproducibility across a fleet of tools (Tool Matching and TMU page).
• SPC readiness: Ensuring control-chart variation is mostly process, not gauge (SPC page; my reading).
• Uncertainty budgets: Repeatability and reproducibility as Type A inputs (GUM Uncertainty Budgets page).

Variance Components

Name	Formula	Related Formulae	Variables/Constants
Crossed random-effects model	$Y_{ijk}=\mu +P_i+O_j+{\left(PO\right)}_{ij}+{\epsilon }_{ijk}$	${\sigma }_{total}^2={\sigma }_P^2+{\sigma }_O^2+{\sigma }_{PO}^2+{\sigma }_{\epsilon }^2$	$Y_{ijk}:k-threadingofpartibyoperatorj$ $P,O,PO:part,operatorandinteractioneffects$ $\begin{array}{r}\epsilon :replicateerror;MetricGate:crossed\\ modelaswritten,repeatabilityaspure\\ replicate(equipment)variance,\\ reproducibilityasappraiserplus\\ interaction;MathWorks:repeatability,\\ operator,part-by-operator,\\ reproducibility=operator+interaction,\\ gageRandR,part-to-part,total\end{array}$ $\begin{array}{r}(myreading:inafabtheoperatorfactor\\ isoftenatool,chamberorrecipe,so\\ reproducibilitymeasurestoolmatching)\end{array}$
Expected mean squares	$E\left[M{S}_E\right]={\sigma }_{\epsilon }^2,E\left[M{S}_{PO}\right]={\sigma }_{\epsilon }^2+r{\sigma }_{PO}^2$	$E\left[M{S}_O\right]={\sigma }_{\epsilon }^2+r{\sigma }_{PO}^2+pr{\sigma }_O^2,E\left[M{S}_P\right]={\sigma }_{\epsilon }^2+r{\sigma }_{PO}^2+or{\sigma }_P^2$	$\begin{array}{r}p,o,r:numbersofparts,operatorsand\\ replicates\end{array}$ $\begin{array}{r}MS:meansquares;PennStateSTAT503:\\ expectedmeansquaresofthetwo-factor\\ randommodelandvariancecomponentsfrom\\ them;MetricGate:interactiondroppedif\\ itsp-valueexceedsalpha,commonly0.05\end{array}$ $\begin{array}{r}(myreading:estimatescomefromsolving\\ theseequations,withnegativeestimates\\ settozero)\end{array}$
Gauge variance components	${\sigma }_{GRR}^2={\sigma }_{repeat}^2+{\sigma }_{repro}^2,{\sigma }_{repro}^2={\sigma }_O^2+{\sigma }_{PO}^2$	${\sigma }_{repeat}^2={\sigma }_{\epsilon }^2$	$\begin{array}{r}MetricGateexamplewith10parts,3\\ operatorsand3replicates(90rows):\\ repeatability0.2793,reproducibility\\ 0.0770,totalgaugeRandR0.3562(11.54\\ percentofvariance),part-to-part2.7298\\ (88.46percent),partF=132.75and\\ operatorF=13.83,interactionF=0.61\\ withp=0.878\end{array}$ $\begin{array}{r}(computed:total3.086;0.3562/3.086=\\ 11.5percent)\end{array}$

Capability Metrics

Name	Formula	Related Formulae	Variables/Constants
Percent study variation	$SV=100\frac{{\sigma }_{GRR}}{{\sigma }_{total}}$	$<10\text{acceptable},10\text{to}30\text{marginal},>30\text{unacceptable}$	$\begin{array}{r}SV:percentstudyvariation;MetricGate:\\ StudyVardefinitionandAIAGbands,\\ example33.98percenttotal,\\ repeatability30.08andreproducibility\\ 15.79percent,verdictunacceptable;\\ MathWorks:PRRbelow10percentcapable\\ andabove30percentnotcapable\end{array}$ $\begin{array}{r}(computed:33.97,30.08and15.80percent\\ fromthevariancecomponents)\end{array}$
Number of distinct categories	$ndc=\lfloor 1.41\frac{{\sigma }_P}{{\sigma }_{GRR}}\rfloor$	$ndc\ge 5$	$\begin{array}{r}MathWorks:ndc=\sqrt{2}times{\sigma }_{part}/{\sigma }_{gage},5ormore\\ capable,below2notcapable;MetricGate:\\ floorof1.41{\sigma }_{part}/{\sigma }_{GRR},atleast5,example\\ ndc=3\end{array}$ $\begin{array}{r}(computed:3.90fortheexample,floored\\ to3;ndc=5correspondstostudy\\ variation=27.1percent,andstudy\\ variation=30percentgives4.48)\end{array}$
Precision-to-tolerance ratio	$P/T=\frac{k{\sigma }_{GRR}}{USL-LSL}$	$k=6\text{or}5.15$	$USL,LSL:specificationlimits$ $\begin{array}{r}k:sigmamultiplier;MathWorks:PTR=k\\ {\sigma }_{GRR}/|S2-S1|,defaultmultiplier5.15\\ spanningthemiddle99percent,capable\\ below0.1andnotabove0.3;MetricGate:\\ percenttolerance=6{\sigma }_{GRR}/(USL-LSL),\\ default6\end{array}$ $\begin{array}{r}(assumed:{\sigma }_{GRR}=0.1\mathrm{nm}CDprecision,\\ tolerance4\mathrm{nm};computed:P/T=0.15with\\ k=6and0.129with5.15)\end{array}$

Interpretation and Study Design

Name	Formula	Related Formulae	Variables/Constants
Effect on observed capability	${\sigma }_{obs}^2={\sigma }_{process}^2+{\sigma }_{GRR}^2$	$\frac{C_{p,obs}}{C_{p,true}}=\sqrt{1-{\left(\frac{SV}{100}\right)}^2}$	$\begin{array}{r}{C}_p:processcapabilityindex;variance\\ additionforindependentprocessandgauge\\ errors(computed)\end{array}$ $\begin{array}{r}(computed:0.995,0.954and0.866atstudy\\ variationof10,30and50percent)\end{array}$
Sigma multiplier coverage	$5.15=2\times 2.576(99\text{per cent}),6=2\times 3(99.73\text{per cent})$	$\frac{6}{5.15}=1.165$	$\begin{array}{r}MathWorks:5.15spansthemiddle99per\\ centofanormalpopulation,six-sigma\\ exampleuses6;MetricGate:default6;GUM\\ page:normalquantiles\end{array}$ $\begin{array}{r}(computed:P/Tandstudyvariationwithk\\ =6are16.5percentlargerthanwith\\ 5.15)\end{array}$
Study design	$N=por,{\nu }_{repeat}=po\left(r-1\right)$	$p=10,o=3,r=2\text{or}3$	$N:totalreadings$ $\begin{array}{r}{\nu }_{repeat}:degreesoffreedomforrepeatability;\\ MetricGateexample10by3by3=90\\ readings;PennStateSTAT503:3\\ operators,20parts,2replicates;\\ degree-of-freedomcount(computed)\end{array}$ $\begin{array}{r}(computed:60degreesoffreedomfor\\ repeatabilityintheexample,soits\\ standarddeviationisknowntoabout9per\\ cent)\end{array}$

Important Questions

Ques: What does a gauge R&R study separate?

Measurement variation from part-to-part variation. Readings are modelled as $Y_{ijk}=\mu +P_i+O_j+(PO{)}_{ij}+{\epsilon }_{ijk}$ (MetricGate), and ANOVA gives variance components: repeatability is the replicate error, reproducibility the operator plus part-by-operator interaction, and gauge R&R their sum (MathWorks). In MetricGate’s example of 10 parts, 3 operators and 3 replicates, repeatability is 0.2793, reproducibility 0.0770 and part-to-part 2.7298, so the gauge accounts for 11.5 per cent of total variance (computed). The interaction was not significant (p = 0.878) and could be pooled into error, and the large part F of 132.75 shows the parts spanned a wide range, as a study requires. In a fab the operator factor is often a tool or chamber, so reproducibility becomes a matching measure (my reading). Reference: MetricGate, ‘Gauge R&R ANOVA Method: AIAG Measurement System Analysis’ (April 2026) (fetch-tool summary); MathWorks, ‘gagerr: Gage repeatability and reproducibility study’, Statistics and Machine Learning Toolbox documentation (fetch-tool summary).

Ques: Why is 11.5 per cent of variance judged unacceptable?

Because study variation compares standard deviations, not variances. MetricGate’s gauge R&R is 11.54 per cent of the variance but $\sqrt{0.1154}$ = 33.97 per cent of the total standard deviation (computed), above the 30 per cent limit, so the system is rated unacceptable; repeatability alone gives 30.08 per cent (computed). Percent variance looks reassuring while percent study variation, the AIAG metric, does not (my reading). The number of distinct categories, $1.41{\sigma }_P/{\sigma }_{GRR}$, is 3.90, floored to 3, below the required 5 (computed). Improving repeatability, for example by more averaging per reading, would help most, since it dominates reproducibility (my reading). MetricGate’s write-up quotes a part F of 88.98 in one place and 132.75 in another, so its reporting example is internally inconsistent. Reference: MetricGate, ‘Gauge R&R ANOVA Method: AIAG Measurement System Analysis’ (April 2026) (fetch-tool summary); MathWorks, ‘gagerr: Gage repeatability and reproducibility study’, Statistics and Machine Learning Toolbox documentation (fetch-tool summary).

Ques: Should P/T use a 6 sigma or 5.15 sigma multiplier?

The sources default differently. MathWorks uses 5.15, which spans the middle 99 per cent of a normal distribution (2 times 2.576), while MetricGate and the AIAG convention it follows use 6, spanning 99.73 per cent (computed; GUM page). The choice changes P/T and study variation by a factor of 1.165 (computed). For an assumed CD-SEM precision of 0.1 $\mathrm{nm}$ against a 4 $\mathrm{nm}$ tolerance, P/T is 0.15 with 6 and 0.129 with 5.15 (computed), both marginal against MathWorks’ 0.1 and 0.3 limits. Since MetricGate’s AIAG-based method defaults to 6, that multiplier is more likely the expected default, and any report should state which was used (my reading). Reference: MathWorks, ‘gagerr: Gage repeatability and reproducibility study’, Statistics and Machine Learning Toolbox documentation (fetch-tool summary); MetricGate, ‘Gauge R&R ANOVA Method: AIAG Measurement System Analysis’ (April 2026) (fetch-tool summary); GUM Uncertainty Budgets page of this vault.

Ques: Do the study-variation and ndc criteria agree?

Not exactly. With $ndc=1.41{\sigma }_P/{\sigma }_{GRR}$ and ${\sigma }_P^2={\sigma }_{total}^2-{\sigma }_{GRR}^2$, the ndc limit of 5 corresponds to study variation of 27.1 per cent, while the study variation limit of 30 per cent gives ndc of 4.48 (computed). A gauge between 27 and 30 per cent therefore passes the marginal study variation band but fails the ndc rule (my reading). MathWorks treats ndc of 5 or more as capable and below 2 as not, while MetricGate requires at least 5, and both use the 10 and 30 per cent study variation bands. The ndc criterion is the stricter one near the boundary and is more likely the binding requirement for process control (my reading). Reference: MathWorks, ‘gagerr: Gage repeatability and reproducibility study’, Statistics and Machine Learning Toolbox documentation (fetch-tool summary); MetricGate, ‘Gauge R&R ANOVA Method: AIAG Measurement System Analysis’ (April 2026) (fetch-tool summary).

Ques: How does gauge variation affect measured process capability?

It inflates observed variation. Observed variance is the sum of process and measurement variances for independent errors (computed), as in Penn State’s split of total variance into product and gauge parts, so observed capability falls by $\sqrt{1-(SV/100{)}^2}$, where SV is the percent study variation: to 0.995 of the true value at 10 per cent study variation, 0.954 at 30 per cent and 0.866 at 50 per cent (computed). A gauge at the 30 per cent limit therefore hides about 5 per cent of capability, while one at 10 per cent hides only half a per cent. With 10 parts, 3 operators and 3 replicates, repeatability has 60 degrees of freedom and is known to about 9 per cent (computed; GUM page), so small study variation differences between studies are not meaningful (my reading). Reference: Penn State STAT 503, ‘13.2 - Two Factor Factorial with Random Factors’: E(MS_A) = sigma^2 + n sigma_tb^2 + bn sigma_t^2, E(MS_B) = sigma^2 + n sigma_tb^2 + an sigma_b^2, E(MS_AB) = sigma^2 + n sigma_tb^2, E(MS_E) = sigma^2, components by equating mean squares to expectations, gauge example of 3 operators, 20 parts, 2 replicates, sigma^2 as repeatability and the operator component as reproducibility (fetch-tool summary); GUM Uncertainty Budgets page of this vault; Tool Matching and Total Measurement Uncertainty (TMU) page of this vault; Statistical Process Control (SPC) page of this vault.

References

1. MetricGate, “Gauge R&R ANOVA Method: AIAG Measurement System Analysis,” April 29, 2026. Available: https://metricgate.com/docs/gauge-rr-anova-method/
2. MathWorks, “gagerr: Gage repeatability and reproducibility study,” Statistics and Machine Learning Toolbox documentation. Available: https://ch.mathworks.com/help/stats/gagerr.html
3. GUM Uncertainty Budgets page of this vault.
4. Penn State Eberly College of Science, STAT 503 Design of Experiments, “13.2 - Two Factor Factorial with Random Factors.” Available: https://online.stat.psu.edu/stat503/book/export/html/708
5. Tool Matching and Total Measurement Uncertainty (TMU) page of this vault.
6. Statistical Process Control (SPC) page of this vault.

Related pages in this vault: GUM Uncertainty Budgets, CD-SEM, Optical CD Scatterometer (OCD), Overlay Metrology Tool, Process Corners and Statistical Models, SMU and Probe Station, Calibration and Traceability, Statistical Process Control (SPC), Tool Matching and Total Measurement Uncertainty (TMU).