Ultra-Fast Detection of Loosened Bolts Based on Binary Image Processing

Bolts are one of the main components in constructional structures such as bridges, screens, and pillars. Also, bolts are used in many mechanical systems such as bicycles, vibrators, and industrial centrifuges. Automatic and low-error diagnosis of loosened bolts in worn systems such as constructional structures and bridges is a major problem. Human inspection is one of the simple methods for inspecting loosened bolts in constructional and mechanical structures. However, human inspections are slow, time-consuming, and periodic. So, it is necessary to design automatic electronic and/or computer systems. In electronic methods, various kinds of sensors are used to investigate the loosened bolts. In different kinds of constructional and mechanical structures, piezoelectric sensors are used under bolts in order to diagnose the damages. However, due to the wide-spreading usage of these structures, this method is practically impossible and costly. Also, automatic techniques based on machine vision which use rod detectors to diagnose loosened bolts, are not practical for all structures in the world [1-3]. Recently, some studies have been conducted based on contact sensors in order to the detection of damages. However, these methods suffer from temperature- and humidity-dependency [4].
Nonetheless, some methods have been utilized to solve these problems [5,6]. Ghen et al. have reported a semi-automatic system to measure the thickness of cracks by using the first derivative of the Gaussian function [7]. Also, Lee et al. have designed a semi-automatic computer system that controls a moving arm [8]. Different systems for the detection of cracks by using Sobel and Laplacian operators [9] and percolation technique [10,11] are reported. Jahanshahi et al. [12] have used a camera on a structure to control and monitor a vast area for diagnosing damages and cracks. They have utilized Sinusoidal Frequency Modulated (SFM) and Laser Doppler anemometry (LDA) image processing techniques [12]. Nan et al. [13] have used image processing techniques such as a Gradient curve for monitoring and diagnosing the bolt defects of a porterage car [13]. Park et al. [14] have reported a new method for diagnosing the looseness of bolts by processing the images. In this method, the edge of each nut is extracted, their rotation angles are identified, and detection is done by comparing with the pre-identified nut angles [14].
Also, Cha et al. [15] offered a computer-based method using image processing techniques and rod detectors to diagnose the looseness of bolts [15]. However, using this method in structures whose bolts do not have rod detectors is impossible. Also, this method cannot be applied to different states of bolts and nuts. To deal with these, in this paper, problems an automatic, computerbased method by using such image processing techniques like Otsu thresholding, the binary complement of Otsu thresholding, dilation, and counting the number of zero pixels of bolts. The obtained area from the zero pixels determines the status of the bolts which they are tight or loose. The rest of the paper is organized as follows. In Section 2, the proposed method, including two training and testing steps of the system, is introduced. Also, in this Section, the bolts in previously reported work are tested by the presented method. Finally, the conclusion is expressed in Section 3.

Training step

In the first stage, which is called “training the system”, several images of studied bolts in different states are given to the system, and for each image, image processing techniques are applied. These techniques are as follows.
A. The noise of the image is eliminated (Removing noise).
B. The colored image is transformed into grayscale (rgb2gray).
C. The CHT is used to diagnose the center of the bolt.
D. After diagnosing the place of the bolts, the cropping operation is carried out.
E. Otsu thresholding is performed.
F. The binary complement of Otsu thresholding is applied.
G. The enhancement of images with a dilation structure element is done.
H. The area of the bolt is calculated.
I. Based on the number of zero pixels, the decision about the loosening or tightness of the bolt is made.
The height of the bolt shows the placement of the bolt in the structure. Also, there is a relation between the height and the area of the zero pixels. According to Figure 1, the area of the zero pixels can be calculated by

Figure 1: Calculation of zero-pixel area.

Where hi, m, and S the height of each column, number of the pixels, and area of the zero pixels of the image, respectively. So, by defining a threshold for the minimum area of the zero pixels, the loosened and tight bolts can be diagnosed with high accuracy. It should be noted that the detection of loosening or tightness of the bolts is the aim of this method, and the looseness percentage of the bolts is not included. Moreover, for each 360-degree rotation of the bolt in the opening direction, about 180 zero pixels are added to the number of zero pixels. Thus, there is no need to make a Support Vector Machine (SVM) to compare the answer to the training state with the testing state. Figure 2 shows the block diagram of the training step.

Figure 2: Block diagram of calculating the zero-pixel area in the training step.

Testing step

The block diagram of the second stage (testing) is shown in Figure 3 In this stage, an image with 4 bolts in different states is given to the system. The same operations mentioned in the training step are carried out on this image. After denoising operation, the image is transformed into a grayscale image. Then, CHT is used to detect the center of the bolts. Besides, the four bolts are separated by cropping operation. The Otsu thresholding and binary complement of Otsu thresholding are separately applied to the cropping images. After counting the zero pixels of each bolt and comparing it with a threshold of the minimum area obtained from the training step, the status of each bolt is determined. It is clear that this method can diagnose the looseness and tightness of the bolts with high accuracy.

Figure 3: Block diagram of calculating the zero pixel area for each bolt in the testing step.

Validating the proposed method

For validating the proposed method, we apply this method to recently reported work [15]. In this work, a structure with four bolts is used to detect the loosened bolts Figure 4.

Figure 4: Validating the proposed method by results of [15].

In this paper, a new method based on image processing techniques for the detection of looseness or tightness of bolts was presented. The proposed method is carried out in two stages of training and testing. In the training stage, several images from the bolt in different states are taken, and by using image processing techniques such as denoising, rgb2gray scale, CHT, Otsu thresholding, the binary complement of Otsu thresholding, and dilation, the area of the zero pixels are calculated. For a threshold value of the area, the decision about the looseness or tightness of the bolt is made. In the testing step, four bolts in different conditions are used, and the abovementioned techniques are separately applied to all bolts. The obtained results show that this method is suitable for all kinds of bolts regardless of their shapes, angles, and sizes. In Comparison with recently reported works, this method benefits from high speed and precision. Also, it has a low computing load that saves time and memory.

1. Phares BM, Rolander DD, Graybeal BA, Washer GA (2001) Reliability of visual bridge inspection. Public Roads 64(5): 22-29.
2. Farrar CR, Lieven NA (2006) Damage Prognosis: The future of structural health monitoring', Philosophical transactions of the royal society a: Mathematical, physical and engineering sciences 365(1851): 623-632.
3. Farrar CR, Worden K (2012) Structural health monitoring: A Machine Learning Perspective, John Wiley & Sons, USA.
4. Xia Y, Chen B, Weng S, Ni YQ, Xu YL (2012) Temperature effect on vibration properties of civil structures: A literature review and case studies. Journal of Civil Structural Health Monitoring 2(1): 29-46.
5. Ni Y, Hua X, Fan K, Ko J (2005) Correlating modal properties with temperature using long-term monitoring data and support vector machine technique. Engineering Structures 27(12): 1762-1773.
6. Sohn H (2006) Effects of environmental and operational variability on structural health monitoring. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 365(1851): 539-560.
7. Chen LC, Shao YC, Jan HH, Huang CW, Tien YM (2006) Measuring system for cracks in concrete using multitemporal images. Journal of Surveying Engineering 132(2): 77-82.
8. Lee BJ, Shin DH, Seo JW, Jung JD, Lee JY (2011) Intelligent bridge inspection using remote controlled robot and image processing technique in, International Symposium on Automation and Robotics in Construction (ISARC), Seoul, Korea.
9. Yu SN, Jang JH, Han CS (2007) Auto inspection system using a mobile robot for detecting concrete cracks in a tunnel. Automation in Construction 16(3): 255-261.
10. Yamaguchi T, Hashimoto S (2006) Image processing based on percolation model'. IEICE transactions on information and systems 89(7): 2044-2052.
11. Yamaguchi T, Hashimoto S (2010) Fast crack detection method for large-size concrete surface images using percolation-based image processing. Machine Vision and Applications 21(5): 797-809.
12. Jahanshahi MR, Masri SF, Padgett CW, Sukhatme GS (2013) An innovative methodology for detection and quantification of cracks through incorporation of depth perception. Machine Vision and Applications 24(2): 227-241.
13. Nan G, Yao J (2015) A real-time visual inspection method of fastening bolts in freight car operation in AOPC 2015: Image Processing and Analysis, International Society for Optics and Photonics, USA.
14. Park JH, Kim TH, Lee KS, Nguyen TC, Kim JT (2015) Novel bolt-loosening detection technique using image processing for bolt joints in steel bridges. Incheon, Korea.
15. Cha YJ, You K, Choi W (2016) Vision-based detection of loosened bolts using the hough transform and support vector machines. Automation in Construction 71(2): 181-188.