Mir Hamid Rezaei1* and Javad Ostadieh2
1Department of Electrical Engineering, Technical and Vocational University, Iran
2Department of Communications and Electronics, Urmia University, Iran
*Corresponding author: Mir Hamid Rezaei, Department of Electrical Engineering, Faculty of Ghazi Tabatabai, West Azarbaijan Branch, Technical and Vocational University, Urmia, Iran
Submission: May 05, 2021Published: July 29, 2021
ISSN: 2576-8816Volume9 Issue3
In this paper, we offer a new method for detecting tight or loosened bolts based on image processing. The proposed method is independent of bolt appearances, and no height detector is needed. Firstly, several pictures are taken from the bolts in tight and different loosened states for training the system. After the training step, the status of the study bolts is processed by MATLAB software. By using image processing techniques including noise removal, Circular Hough Transform (CHT), transformation of colored images to grayscale, Otsu thresholding, binary complement of Otsu thresholding, enhancement of images with dilation structure element, and after calculating the area of the bolts, the decision about the states of the bolts can be accurately determined based on the number of zero pixels. The proposed method has the advantages of the low calculation process, very fast, and high precision response.
Keywords: Circle hough transform; Image processing; Inspecting; Loosened bolt; Otsu thresholding
Bolts are one of the main components in constructional structures such as bridges,
screens, and pillars. Also, bolts are used in many mechanical systems such as bicycles,
vibrators, and industrial centrifuges. Automatic and low-error diagnosis of loosened bolts
in worn systems such as constructional structures and bridges is a major problem. Human
inspection is one of the simple methods for inspecting loosened bolts in constructional and
mechanical structures. However, human inspections are slow, time-consuming, and periodic.
So, it is necessary to design automatic electronic and/or computer systems. In electronic
methods, various kinds of sensors are used to investigate the loosened bolts. In different kinds
of constructional and mechanical structures, piezoelectric sensors are used under bolts in
order to diagnose the damages. However, due to the wide-spreading usage of these structures,
this method is practically impossible and costly. Also, automatic techniques based on machine
vision which use rod detectors to diagnose loosened bolts, are not practical for all structures
in the world [1-3]. Recently, some studies have been conducted based on contact sensors in
order to the detection of damages. However, these methods suffer from temperature- and
Nonetheless, some methods have been utilized to solve these problems [5,6]. Ghen et al. have reported a semi-automatic system to measure the thickness of cracks by using the first derivative of the Gaussian function . Also, Lee et al. have designed a semi-automatic computer system that controls a moving arm . Different systems for the detection of cracks by using Sobel and Laplacian operators  and percolation technique [10,11] are reported. Jahanshahi et al.  have used a camera on a structure to control and monitor a vast area for diagnosing damages and cracks. They have utilized Sinusoidal Frequency Modulated (SFM) and Laser Doppler anemometry (LDA) image processing techniques . Nan et al.  have used image processing techniques such as a Gradient curve for monitoring and diagnosing the bolt defects of a porterage car . Park et al.  have reported a new method for diagnosing the looseness of bolts by processing the images. In this method, the edge of each nut is extracted, their rotation angles are identified, and detection is done by comparing with the pre-identified nut angles .
Also, Cha et al.  offered a computer-based method using image processing techniques and rod detectors to diagnose the looseness of bolts . However, using this method in structures whose bolts do not have rod detectors is impossible. Also, this method cannot be applied to different states of bolts and nuts. To deal with these, in this paper, problems an automatic, computerbased method by using such image processing techniques like Otsu thresholding, the binary complement of Otsu thresholding, dilation, and counting the number of zero pixels of bolts. The obtained area from the zero pixels determines the status of the bolts which they are tight or loose. The rest of the paper is organized as follows. In Section 2, the proposed method, including two training and testing steps of the system, is introduced. Also, in this Section, the bolts in previously reported work are tested by the presented method. Finally, the conclusion is expressed in Section 3.
In the first stage, which is called “training the system”, several
images of studied bolts in different states are given to the system,
and for each image, image processing techniques are applied. These
techniques are as follows.
A. The noise of the image is eliminated (Removing noise).
B. The colored image is transformed into grayscale (rgb2gray).
C. The CHT is used to diagnose the center of the bolt.
D. After diagnosing the place of the bolts, the cropping operation is carried out.
E. Otsu thresholding is performed.
F. The binary complement of Otsu thresholding is applied.
G. The enhancement of images with a dilation structure element is done.
H. The area of the bolt is calculated.
I. Based on the number of zero pixels, the decision about the loosening or tightness of the bolt is made.
The height of the bolt shows the placement of the bolt in the structure. Also, there is a relation between the height and the area of the zero pixels. According to Figure 1, the area of the zero pixels can be calculated by
Figure 1: Calculation of zero-pixel area.
Where hi, m, and S the height of each column, number of the pixels, and area of the zero pixels of the image, respectively. So, by defining a threshold for the minimum area of the zero pixels, the loosened and tight bolts can be diagnosed with high accuracy. It should be noted that the detection of loosening or tightness of the bolts is the aim of this method, and the looseness percentage of the bolts is not included. Moreover, for each 360-degree rotation of the bolt in the opening direction, about 180 zero pixels are added to the number of zero pixels. Thus, there is no need to make a Support Vector Machine (SVM) to compare the answer to the training state with the testing state. Figure 2 shows the block diagram of the training step.
Figure 2: Block diagram of calculating the zero-pixel area in the training step.
The block diagram of the second stage (testing) is shown in Figure 3 In this stage, an image with 4 bolts in different states is given to the system. The same operations mentioned in the training step are carried out on this image. After denoising operation, the image is transformed into a grayscale image. Then, CHT is used to detect the center of the bolts. Besides, the four bolts are separated by cropping operation. The Otsu thresholding and binary complement of Otsu thresholding are separately applied to the cropping images. After counting the zero pixels of each bolt and comparing it with a threshold of the minimum area obtained from the training step, the status of each bolt is determined. It is clear that this method can diagnose the looseness and tightness of the bolts with high accuracy.
Figure 3: Block diagram of calculating the zero pixel area for each bolt in the testing step.
Validating the proposed method
For validating the proposed method, we apply this method to recently reported work . In this work, a structure with four bolts is used to detect the loosened bolts Figure 4.
Figure 4: Validating the proposed method by results of .
In this paper, a new method based on image processing techniques for the detection of looseness or tightness of bolts was presented. The proposed method is carried out in two stages of training and testing. In the training stage, several images from the bolt in different states are taken, and by using image processing techniques such as denoising, rgb2gray scale, CHT, Otsu thresholding, the binary complement of Otsu thresholding, and dilation, the area of the zero pixels are calculated. For a threshold value of the area, the decision about the looseness or tightness of the bolt is made. In the testing step, four bolts in different conditions are used, and the abovementioned techniques are separately applied to all bolts. The obtained results show that this method is suitable for all kinds of bolts regardless of their shapes, angles, and sizes. In Comparison with recently reported works, this method benefits from high speed and precision. Also, it has a low computing load that saves time and memory.
© 2021 Mir Hamid Rezaei. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and build upon your work non-commercially.