Automatic Defect Classification and Machine Learning

1. Theory

• Definition: Automatic defect classification (ADC) assigns each detected defect, or each wafer map, to a class such as particle, bridge or scratch from its features, so that defect counts become class counts (KLA; Siemens).
• Metrics: Precision $tp/(tp+fp)$, the purity of a class bin, and recall $tp/(tp+fn)$, the share of a class that is found, combined as $F_1$ (scikit-learn; KLA).
• Rejection: Defects whose classifier confidence falls below a per-class threshold go to a person, which guarantees a purity target (KLA).
• Clustering: Density-based clustering groups defect coordinates into clusters and marks isolated points as noise (scikit-learn DBSCAN; Hou et al.).

2. Working Principle

• Features: Image features, coordinates and composition for review ADC; whole-map images or row and column sums for map classification (KLA; Shukla).
• Models: Random forests and SVMs on features; CNN detectors such as YOLOv8 on SEM images with layout (KLA; Siemens).
• Training: Manual labels, augmented with synthetic images; data sufficiency tested by adding 10 to 100 per cent of a class (Siemens; KLA).
• Clustering: DBSCAN with eps and min_samples, or path-filtering methods for mixed patterns (scikit-learn; Hou et al.).

3. Applications

• Review ADC: SEM pattern defects such as bridges and pinches (Siemens).
• Wafer-map classification: Spatial patterns of the Wafer-Map Spatial Signature Analysis page (Kim and Kim; Shukla).
• Nuisance filtering: Separating defects of interest from nuisance, as on the Defect Detection Fundamentals page (my reading).

Classification Metrics

Name	Formula	Related Formulae	Variables/Constants
Precision (purity) and recall	$P=\frac{tp}{tp+fp},R=\frac{tp}{tp+fn}$	$F_1=\frac{2PR}{P+R}$	$\begin{array}{r}scikit-learn:definitions,F-betaas\\ weightedharmonicmeanwithbeta=1;KLA:\\ purityrequirementperclass;Defect\\ DetectionFundamentalspageforcapture\\ andnuisancerates\end{array}$ $\begin{array}{r}(computed:P=0.222andR=0.333give{F}_1\\ =0.267,thedocumentedvalue;myreading:\\ binpurityisprecision,per-classcapture\\ isrecall)\end{array}$
Averaging over classes	$\text{macro}=\frac{1}{K}{\sum }_k{m}_k,\text{weighted}=\frac{{\sum }_k{n}_k{m}_k}{\sum n_k}$	$\text{micro: pooled}tp,fp,fn$	$\begin{array}{r}scikit-learn:macroignoresimbalance,\\ weightedusessupport{n}_k,microcounts\\ globally\end{array}$ $\begin{array}{r}(myreading:macroaveragingexposesrare\\ killerclassesthatmicroaveraginghides)\end{array}$
Confidence threshold and rejection	${\hat{p}}_k<t_k\Rightarrow \text{manual review}$	$P_k\ge P_{req}(90\text{or}85\%)$	$\begin{array}{r}KLA:thresholdsperdefecttypetomeet\\ thepurityrequirement,low-confidence\\ defectssenttoanoperator,training\\ incrementsfrom10to100percent\end{array}$ $\begin{array}{r}(computed:abinof100callswith90\\ correctmeetsa90percentpurity\\ requirement)\end{array}$

Classifiers

Name	Formula	Related Formulae	Variables/Constants
SEM-image classifier	$\text{YOLOv8, SEM + layout input}$	$mA{P}_{0.5}=0.79,AP:0.477\text{to}0.901$	$\begin{array}{r}Siemens:sixpattern-defectclasses,355\\ realand1302syntheticimagesfrom5\\ fabs,addedpatternweakestandbridge\\ strongest\end{array}$ $\begin{array}{r}(myreading:rare,variedclassesstay\\ hardestevenwithsyntheticdata)\end{array}$
Ensemble voting	$V=1:TP=92.9,FP=28.4\%$	$V=5:TP=68.4,FP=5.2\%$	$\begin{array}{r}Siemens:votesfromfivemodelsizes,\\ singlemodels78.8to85.8percentTPat\\ 14.4to15.9percentFP\end{array}$ $\begin{array}{r}(computed:V=3gives84.5and11.6per\\ cent,closetothebestsinglemodelwith\\ fewerfalsepositives)\end{array}$
Ensemble trees	$\text{random forest, SVM}$	$\text{inputs: image features, coordinates, composition}$	$\begin{array}{r}KLA:ensemblelearningclassifier;Shukla:\\ randomforeston104rowandcolumnsums\\ at96.79percent\end{array}$ $\begin{array}{r}(myreading:treeensemblesworkwellon\\ engineeredfeatureswithlittledata)\end{array}$

Clustering and Map Classification

Name	Formula	Related Formulae	Variables/Constants
DBSCAN	$\epsilon ,\text{min\_samples}\Rightarrow \text{core points}$	$\text{label}=-1\Rightarrow \text{noise}$	$\begin{array}{r}scikit-learn:epsthemostimportant\\ parameter,defaults0.5and5;Houetal.:\\ Eps=\sqrt{2}andMinPts=3ondiegrids,not\\ generalising\end{array}$ $\begin{array}{r}(myreading:ondiecoordinateseps=\sqrt{2}\\ joinsdiagonalneighbours)\end{array}$
Clustering quality	$DB{I}_{k-means}=0.4131\text{against}0.1386(NPFC)$	$O(n)\text{against}O(n^2)$	$\begin{array}{r}Houetal.:silhouetteandMahalanobisSD\\ indextochoose2to5clusters,better\\ thanmedianfilteringin99of110maps\end{array}$ $\begin{array}{r}(myreading:lowerDavies-Bouldinindex\\ meanstighter,betterseparatedclusters)\end{array}$
Wafer-map classifiers	$Acc=91.3\text{to}94\%(26\times 26)$	$Acc=81.2\text{to}89.9\%(\text{mixed sizes})$	$\begin{array}{r}KimandKim:polarCNNsonWM-811K;\\ Shukla:96.79percentonMixedWM38;\\ Wafer-MapSpatialSignatureAnalysispage\end{array}$ $\begin{array}{r}(computed:resizingtomixedsizescost\\ 1.4to12.8points)\end{array}$

Important Questions

Ques: What are precision, recall and purity in defect classification?

Precision is the purity of a class bin, recall the share of a class that is found. scikit-learn defines precision as $tp/(tp+fp)$ and recall as $tp/(tp+fn)$, with F-beta as their weighted harmonic mean. Its example gives precision 0.222 and recall 0.333; $F_1=2PR/(P+R)$ reproduces the stated 0.267 (computed). KLA’s patent sets a classification purity requirement, for example 90 or 85 per cent, for each defect type. A bin of 100 defects with 90 correctly classed meets 90 per cent (computed). In my reading, purity is precision and per-class capture is recall, the counterparts of the capture and nuisance rates on the Defect Detection Fundamentals page. Reference: scikit-learn, ‘sklearn.metrics.precision_recall_fscore_support’ documentation: precision tp/(tp + fp), recall tp/(tp + fn), F-beta as a weighted harmonic mean with beta = 1 weighting both equally, support as occurrences of each class, micro, macro and weighted averaging, example giving macro precision 0.222, recall 0.333 and F 0.267 (fetch-tool summary); L. He, C.-H. A. Chen, S. Venkataraman, J. R. Jordan III, H. Ying and H. Sinha (KLA-Tencor), ‘Method and system for defect classification’, US Patent 9,898,811 (priority May 2015): defect review requiring classification, attributes including image features, defect coordinates and composition results, ensemble learning with random forest and SVM classifiers, a user-selected classification purity requirement (examples 90 and 85 per cent) met by confidence thresholds per defect type, defects below threshold sent for manual classification, training data incremented from 10 to 100 per cent of a defect type to test sufficiency, manually built decision trees time-consuming (fetch-tool summary); Defect Detection Fundamentals page of this vault.

Ques: How does ADC guarantee a purity target?

By refusing to classify what it is unsure of. KLA’s patent uses an ensemble learning classifier, such as a random forest, with attributes including image features, defect coordinates and composition analysis results. For each defect type a confidence threshold is set so that the user’s purity requirement is met. Defects falling below the threshold are sent to the user interface for manual classification. The patent also tests whether training data are sufficient by adding 10 to 100 per cent of a type’s examples, and notes that hand-built decision trees took too long to create. In my reading, the threshold trades operator workload against purity: a stricter threshold sends more defects to people. Reference: L. He, C.-H. A. Chen, S. Venkataraman, J. R. Jordan III, H. Ying and H. Sinha (KLA-Tencor), ‘Method and system for defect classification’, US Patent 9,898,811 (priority May 2015): defect review requiring classification, attributes including image features, defect coordinates and composition results, ensemble learning with random forest and SVM classifiers, a user-selected classification purity requirement (examples 90 and 85 per cent) met by confidence thresholds per defect type, defects below threshold sent for manual classification, training data incremented from 10 to 100 per cent of a defect type to test sufficiency, manually built decision trees time-consuming (fetch-tool summary).

Ques: How well do deep-learning models classify SEM review images?

Well for distinct classes, less well for rare, varied ones. Siemens trained YOLOv8 detectors on SEM images with aligned layout clips: 355 real images from five fabs plus 1302 synthetic images from a conditional GAN. Across six classes, the mean average precision at IoU 0.5 on unseen fab images was 0.79. Per-class AP ranged from 0.477 for added pattern to 0.901 for bridge. Single models gave true-positive rates of 78.8 to 85.8 per cent at about 15 per cent false positives. In my reading, classes with the most varied appearance stay hardest even with synthetic data, so per-class metrics matter more than the average. Reference: S. Choi, Q. Xie, N. Greeneltch, H. J. Lee, M. Govindaraj, S. Jayaram, M. Pereira, S. Biswas, S. Bhamidipati and I. Torunoglu (Siemens EDA), ‘Machine learning based SEM image analysis for automatic detection and classification of wafer defects’ (ASMC 2024 manuscript): six classes (missing pattern, added pattern, pinch, line-end extension, line-end pullback, bridge), 355 real SEM images from 5 fabs and 1302 cGAN synthetic images, SEM and layout as two-channel input, YOLOv8 in five sizes, 500 epochs, mAP 0.79 at IoU 0.5 on unseen fabs, Medium model per-class AP from 0.477 (added pattern) to 0.901 (bridge), TP rate 78.8 to 85.8 per cent with FP 14.4 to 15.9 per cent for single models, ensemble votes V = 1 to 5 giving TP 92.9, 88.8, 84.5, 77.5, 68.4 and FP 28.4, 15.1, 11.6, 7.7, 5.2 per cent, TP counted at IoU at least 0.3 regardless of type (fetch-tool summary).

Ques: How does ensemble voting trade capture against false positives?

More required votes mean fewer false positives but more misses. Siemens combined five YOLOv8 sizes and required V of them to agree. With V = 1 the true-positive rate was 92.9 per cent at 28.4 per cent false positives. With V = 5 it was 68.4 and 5.2 per cent. Intermediate values gave 88.8 and 15.1 (V = 2), 84.5 and 11.6 (V = 3), and 77.5 and 7.7 per cent (V = 4). V = 3 matches the best single model’s capture, 85.8 per cent, with fewer false positives (computed comparison). The authors note that single voters degrade the balance between true and false positives. This is the same capture-against-nuisance choice made in inspection (my reading). Reference: S. Choi, Q. Xie, N. Greeneltch, H. J. Lee, M. Govindaraj, S. Jayaram, M. Pereira, S. Biswas, S. Bhamidipati and I. Torunoglu (Siemens EDA), ‘Machine learning based SEM image analysis for automatic detection and classification of wafer defects’ (ASMC 2024 manuscript): six classes (missing pattern, added pattern, pinch, line-end extension, line-end pullback, bridge), 355 real SEM images from 5 fabs and 1302 cGAN synthetic images, SEM and layout as two-channel input, YOLOv8 in five sizes, 500 epochs, mAP 0.79 at IoU 0.5 on unseen fabs, Medium model per-class AP from 0.477 (added pattern) to 0.901 (bridge), TP rate 78.8 to 85.8 per cent with FP 14.4 to 15.9 per cent for single models, ensemble votes V = 1 to 5 giving TP 92.9, 88.8, 84.5, 77.5, 68.4 and FP 28.4, 15.1, 11.6, 7.7, 5.2 per cent, TP counted at IoU at least 0.3 regardless of type (fetch-tool summary); Defect Detection Fundamentals page of this vault.

Ques: How is DBSCAN used to find defect clusters?

By growing clusters from dense core points and labelling the rest as noise. scikit-learn’s DBSCAN takes eps, the neighbourhood distance and its most important parameter, and min_samples, the points needed for a core point including itself. Defaults are 0.5 and 5, and noise gets label -1. In its example, eps = 3 and min_samples = 2 split six points into two clusters and one noise point. Hou et al. note that a published wafer-map choice of eps $\sqrt{2}$ and MinPts 3 does not generalise across maps. On a die grid, $\sqrt{2}$ joins diagonal neighbours (my reading). Their path-filtering method beat k-means, with a Davies-Bouldin index of 0.1386 against 0.4131 on one map. Reference: scikit-learn 1.0, ‘sklearn.cluster.DBSCAN’ documentation: core samples of high density with clusters expanded from them, eps (default 0.5) as the neighbourhood distance and the most important parameter, min_samples (default 5) including the point itself, Euclidean metric by default, noise labelled -1, example eps = 3 and min_samples = 2 on six points giving labels 0, 0, 0, 1, 1, -1 (fetch-tool summary); X. Hou, G. Qin, Y. Lu, M. Yi and S. Chen, ‘A defect detection method of mixed wafer map using neighborhood path filtering clustering algorithm’, Journal of Electronic Testing 40, 419-433 (2024): DBSCAN depending on Eps and MinPts, a cited choice of Eps = sqrt(2) and MinPts = 3 for wafer-map filtering not generalising, NPFC with 3 x 3 neighbourhoods, noise removal and 2 to 5 core clusters chosen by an SD index of silhouette and Mahalanobis terms, 110 WM-811K mixed maps, better than median filtering in 99 of 110 (MAE 0.0238 against 0.0327), DBI 0.4131 (k-means) against 0.1386 (NPFC) on one map, O(n) against O(n^2) for DBSCAN (fetch-tool summary).

Ques: How accurate are wafer-map pattern classifiers, and which figure is more likely in practice?

Roughly 80 to 90 per cent on real mixed data, despite higher headline figures. Shukla reports 96.79 per cent accuracy with a random forest on 104 row and column sums of MixedWM38, a set of 7015 uniform 52 by 52 maps. Kim and Kim reach 91.3 to 94 per cent on WM-811K maps resized to 26 by 26, but 81.2 to 89.9 per cent on all 25 519 patterned maps of mixed sizes, against 85.58 per cent for an image baseline. Mixed sizes cost 1.4 to 12.8 points (computed). The mixed-size results are more likely to reflect a fab, where die sizes vary and labels for Edge-Loc, Loc and Scratch are subjective, as Kim and Kim note (my reading). Reference: N. Shukla, ‘An embarrassingly simple approach for wafer feature extraction and defect pattern recognition’, arXiv:2303.11632: MixedWM38 of 7015 maps at 52 x 52, row and column sums as a 104-value feature vector after a 3 x 3 median filter, random forest at 96.79 per cent accuracy, 96.80 per cent precision and 96.79 per cent recall against 96.44 to 96.62 per cent for deep models, no variance reported (fetch-tool summary); M. H. Kim and T. S. Kim, Electronics 13, 1360 (2024): polar-coordinate CNN on WM-811K at 91.3 per cent and a tree of binary CNNs at 94 per cent on 26 x 26 maps, falling to 89.89 and 81.2 per cent on 25 519 maps of mixed sizes against 85.58 per cent for a Cartesian image baseline, subjective labels for Edge-Loc, Loc and Scratch (fetch-tool summary); Wafer-Map Spatial Signature Analysis page of this vault.

References

1. L. He, C.-H. A. Chen, S. Venkataraman, J. R. Jordan III, H. Ying and H. Sinha, “Method and system for defect classification,” US Patent 9,898,811, KLA-Tencor. Available: https://patents.google.com/patent/US9898811
2. S. Choi et al., “Machine learning based SEM image analysis for automatic detection and classification of wafer defects,” Siemens EDA, ASMC 2024 manuscript. Available: https://support.sw.siemens.com/en-US/okba/KB000133350_EN_US/Machine-Learning-Based-SEM-Image-Analysis-for-Automatic-Detection-and-Classification-of-Wafer-Detects/media/ASMC-2024_ML-SEM-ADC_manuscript_final.pdf
3. scikit-learn, “sklearn.metrics.precision_recall_fscore_support,” documentation. Available: https://scikit-learn.org/stable/modules/generated/sklearn.metrics.precision_recall_fscore_support.html
4. scikit-learn 1.0, “sklearn.cluster.DBSCAN,” documentation. Available: https://scikit-learn.org/1.0/modules/generated/sklearn.cluster.DBSCAN.html
5. X. Hou, G. Qin, Y. Lu, M. Yi and S. Chen, “A defect detection method of mixed wafer map using neighborhood path filtering clustering algorithm,” Journal of Electronic Testing, vol. 40, pp. 419-433, 2024. Available: https://www.eng.auburn.edu/~agrawvd/JETTA/FULL_ISSUE_40-4/P01_40-4_Hou_p419-433.pdf
6. M. H. Kim and T. S. Kim, “Development of a wafer defect pattern classifier using polar coordinate system transformed inputs and convolutional neural networks,” Electronics, vol. 13, p. 1360, 2024. Available: https://mdpi.com/2079-9292/13/7/1360
7. N. Shukla, “An embarrassingly simple approach for wafer feature extraction and defect pattern recognition,” arXiv:2303.11632. Available: https://arxiv.org/pdf/2303.11632
8. Defect Detection Fundamentals page of this vault.
9. Wafer-Map Spatial Signature Analysis page of this vault.

Related pages in this vault: Defect Detection Fundamentals, Wafer-Map Spatial Signature Analysis, Defect Sources and Classification, E-Beam Inspection and Defect Review, Defect Inspection Workflow and Sampling Plans, Defect Data Handling with Python and Perl, Defect SPC and Excursion Monitoring.