Data-Efficient Graph Embedding Learning for PCB Component Detection
Abstract
Abstract
This paper presents a challenging computer vision task, namely the detection of generic components on a PCB, and a novel set of deep-learning methods that are able to jointly leverage the appearance of individual components and the propagation of information across the structure of the board to accurately detect and identify various types of components on a PCB. Due to the expense of manual data labeling, a highly unbalanced distribution of component types, and significant domain shift across boards, most earlier attempts based on traditional image processing techniques fail to generalize well to PCB images with various quality, lighting conditions, etc. Newer object detection pipelines such as Faster R-CNN, on the other hand, require a large amount of labeled data, do not deal with domain shift, and do not leverage structure. To address these issues, we propose a three stage pipeline in which a class-agnostic region proposal network is followed by a low-shot similarity prediction classifier. In order to exploit the data dependency within a PCB, we design a novel Graph Network block to refine the component features conditioned on each PCB. To the best of our knowledge, this is one of the earliest attempts to train a deep learning based model for such tasks, and we demonstrate improvements over recent graph networks for this task. We also provide in-depth analysis and discussion for this challenging task, pointing to future research.
This paper presents a challenging computer vision task, namely the detection of generic components on a PCB, and a novel set of deep-learning methods that are able to jointly leverage the appearance of individual components and the propagation of information across the structure of the board to accurately detect and identify various types of components on a PCB. Due to the expense of manual data labeling, a highly unbalanced distribution of component types, and significant domain shift across boards, most earlier attempts based on traditional image processing techniques fail to generalize well to PCB images with various quality, lighting conditions, etc. Newer object detection pipelines such as Faster R-CNN, on the other hand, require a large amount of labeled data, do not deal with domain shift, and do not leverage structure. To address these issues, we propose a three stage pipeline in which a class-agnostic region proposal network is followed by a low-shot similarity prediction classifier. In order to exploit the data dependency within a PCB, we design a novel Graph Network block to refine the component features conditioned on each PCB. To the best of our knowledge, this is one of the earliest attempts to train a deep learning based model for such tasks, and we demonstrate improvements over recent graph networks for this task. We also provide in-depth analysis and discussion for this challenging task, pointing to future research.
Problem Description
Problem Description
Goal: Generate a bill of components for a printed circuit board (PCB) and match to design documents, with applications in reverse engineering and validation.
Goal: Generate a bill of components for a printed circuit board (PCB) and match to design documents, with applications in reverse engineering and validation.
Problem: Detection of individual components on a PCB.
Problem: Detection of individual components on a PCB.
- Input: High-resolution images, which can be 15 megapixels or more in resolution.
- Output: A list of component types and their bounding boxes.
- Requires high resolution images, dense annotation (~500 components per PCB), and professional labeling. There are very few high-quality datasets.
- High intra-class variance and low inter-class variance for certain types of components.
- Skewed distribution
Approach: Leverage similarity learning and spatial structure through graphs.
Approach: Leverage similarity learning and spatial structure through graphs.
PCB Dataset
PCB Dataset
We develop, annotate, and release a dataset with
We develop, annotate, and release a dataset with
a) 47 high-resolution images
a) 47 high-resolution images
b) 31 component types
b) 31 component types
c) ~62k component instances.
c) ~62k component instances.
The distribution is extremely skewed with most of the components being resistors, capacitors, and connectors.
The distribution is extremely skewed with most of the components being resistors, capacitors, and connectors.
Example of high intra-class variance.
Examples of low inter-class variance.
There is a high intra-class variance, where components are the same but look very different due to the large variety of types available.
There is a high intra-class variance, where components are the same but look very different due to the large variety of types available.
There is also low inter-class variance, where components that are different look very similar due to similar packaging.
There is also low inter-class variance, where components that are different look very similar due to similar packaging.
Proposed Method
Proposed Method
We seek to leverage additional information through graphs:
We seek to leverage additional information through graphs:
1. Capture the spatial regularities in component layouts.
1. Capture the spatial regularities in component layouts.
2. Capture the global statistics of the whole board.
2. Capture the global statistics of the whole board.
3. Incorporate one template per type in query board within graph
3. Incorporate one template per type in query board within graph
> Refine the feature of object proposals based on these additional sources of information.
> Refine the feature of object proposals based on these additional sources of information.
> Everything is learned and jointly optimized including graph edges and node features.
> Everything is learned and jointly optimized including graph edges and node features.
> Typical graph convolution networks require fixed, known connectivity. Instead we use similarity prediction to initialize the connectivity and then further optimize it.
> Typical graph convolution networks require fixed, known connectivity. Instead we use similarity prediction to initialize the connectivity and then further optimize it.
Please see our paper for more details.
Please see our paper for more details.
Results
Results
Blue: correct type and precise bounding box location
Blue: correct type and precise bounding box location
Cyan: imprecise bounding box location
Cyan: imprecise bounding box location
Magenta: miss-detected component
Magenta: miss-detected component
Yellow: precise bounding box location but wrong type
Yellow: precise bounding box location but wrong type
Citation
Citation
@inproceedings{kuo2019data, title = {Data-Efficient Graph Embedding Learning for PCB Component Detection}, author = {Kuo, Chia-Wen and Ashmore, Jacob and Huggins, David and Kira, Zsolt}, booktitle={2019 IEEE Winter Conference on Applications of Computer Vision (WACV)}, year = {2019}, organization={IEEE}}