Study on image processingmethod to classify objects on dynamic conveyor

Use your smartphone to scan this QR code and download this article ABSTRACT Controlling a robotic arms for applications such as detection and classification moving object using the vision sensor is a trend in the field of industrial robots. In particular, the vision sensor is the "eye" of the robot. To solve this problem, we need an efficient image processing algorithm for object identification to optimize the speed. Our classification principle based on the color of the object to be classified first, then separating contour to classify according to the shape of the object. In addition, our paper also propose a classification method that rarely mentioned in the relevant documents that classify based on object's characteristic. In fact, the product packaging not only has one color, but also includes complex color and patterns. Being able to classify these products shows the practicality of the proposed method. For complex colors and patterns object, the PCASIFT algorithm is useful, where SIFT extracts the local characteristics of the object and PCA reduces the number of dimensionality and retain only the best characteristics for identification. To picking object, a proposed design with the optimal requirements of picking order so that picking time is the shortest to minimize the delay for the next picking. The other outstanding advantage is a system of robotic arm to perform pick-up and sorting. This helps to verify good running algorithms in real time. The items are randomly released and the rotation of items is random. The speed of the conveyor is 5cm/s, an average of more than 2 seconds to pick up an object and robot arm processing precisely at high speed. The experimental results using camera Logitech C270, Yamaha Scara YK-400X robotic arm, LabVolt conveyor andOpenCV library are satisfactory, reliable and applicable.


INTRODUCTION
Due to the need to use more and more robots in complex manufacturing processes to improve productivity, reduce cost, enhance quality, accuracy and minimize risk when people working in toxic environments. The design of the robot operation system does not need human intervention is truly needed and fully capable, especially in an era of rapid development of current computer vision 1,2 . To solve the specific problem posed "Sorting and picking objects on conveyor belt" 3 . This article propose a simple and highly accurate method for classifying real-time objects 1,3 . Our algorithm will select the correct objects on the conveyor belt at any speed and choose the optimal order to avoid missing the process 1 . Image processing program to determine the direction of the object, to solve the problem of placing the object in a neat box and with less space. Many articles mentioned the problem of sorting objects. However, there are limitations. The article "Practical Applications for Robotic Arms Using Image Processing" (Mihai Dragusu, Anca Nicoleta Mihalache and Razvan Solea, 2012) combined robotic arm to classify objects based on image processing but objects in a state of standing still and based on the contours of the object without considering the colour. The article "Visual processing and classification of items on a moving conveyor: a selective perception approach" (H. I-sil Bozma and H.ulya Yal-cin, 2002) focusing only on the shape of the object without regard to color, on the other hand the results of the article only stop at the level of identification results, no system for picking and sorting. The article "Moving object detecting and tracking method based on color image" (Hong-Kui Liu and Jun Zhou, 2008) relied on the color element of the object to classify but did not rely on other factors such as the shape of objects and also no system for picking and sorting. Based on the research process of previous relevant articles, this article developed an object classification method that combines the advantages of these articles. Our classification principle based on the color of the object to be classified first, then separating contour to classify according to the shape of the object. In addition, our paper also propose a classification method that rarely mentioned in the relevant documents that classify based on object's characteristic. In fact, the product packaging not only has one color, but also includes complex color and patterns. Being able to classify these products shows the practicality of the proposed method. The other outstanding advantage is a system of robotic arm to perform pick-up and sorting. This helps to verify good running algorithms in real time.

Approach method
The approach to the problem of picking and sorting objects on a conveyor ( Figure 1) consists of two main steps: • First, the process of identifying the object from the input image, then giving the position, direction of the object. For identification based on color and shape, an image needs to determine the threshold 4,5 , then based on the contour to sort by shape 2,6 . For complex colors and patterns object, the PCA-SIFT algorithm is useful, where SIFT extracts the local characteristics of the object and PCA reduces the number of dimensionality and retain only the best characteristics for identification 3 . • Second, picking object, this step designs with the optimal requirements of picking order so that picking time is the shortest to minimize the delay for the next picking.

Image processing
A typical image processing system includes the following steps: • Collect data from camera, preprocessing.
• Advanced image processing to perform a specific request.

Collect data from camera and preprocessing
Image data is collected from the camera Logitech C270. In order to increase the efficiency of the image identification process, preprocessing is used to helps clear redundancy images, noise filters, and speed up processing. Images collected from camera are 24-bit RGB images. There are plenty of surplus parts not used. If images do not have these parts, the speed of processing will increase significantly. Therefore, the next step is crop the image again; only retain the image portion of the conveyor 3    To eliminate this part, image in Figure 2 a is putted through a quadrilateral mask (with two sides coinciding with the upper and lower edges of the conveyor belt), keeping only the parts in the mask. The result of this process is as follows Figure 2 b. Finally, the rest of the image is called Region of Interest (ROI), the next processing will be applied to this ROI.

Identify colors and shapes
An image obtained from the camera consists of threecolor channels R, G, and B. Each pixel consists of three different values R, G, B, which define three values of three basic colors red, green, and blue 7 .
For example, to identify red color and shape (Figure 3), photos from the camera are taken to preprocessing the image. Second, select the red threshold and convert to binary image only with the red object. Third, find the outline and draw it. Finally, the number of edges is counted to determine the polygon type. Based on that feature, the image can split into three different images with R, G, or B channel and use low and high thresholds to obtain the required values.  The boundaries of the objects are the places where gray intensity changes most strongly. These areas can be found by defining the gradient of the image. In OpenCV, there is a tool available for separating the boundary that is the "findContours" function 7 . This function uses the canny algorithm to find the edge and shape of an object. There are only three specific shapes (circles, triangles, squares). If a polygon has an edge greater than 8, it will be classified as a circle. Thus, the classification of the shape uses the following criteria: • Square: The number of edges is 4, the error cosine value of the angles is less than 0.2. • Circle: The number of edges more than 8.
• Triangle: The number of edges is equal to 3. • The polygons found must be polygonal convex.
• The area of the polygon must be large enough.
From the object's edge data, use the Ramer-Douglas-Peucker algorithm to approximate that data to a simpler polygon, i.e., reduce the number of points that make up the polygon. The result for the white square as followsFigure 5, for triangles and circles have similar results. As shown in Figure 6, the object identification method used is the matching object, which uses the characteristics of the feature or feature point image 8 . This approach overcomes some of the disadvantages of conventional image processing such as noise sensitivity, rotating objects, and brightness changes. The SURF/SIFT algorithm is used to extract key point characteristics of an object, which is the two most popular methods today due to the high precision of SIFT and the fast processing speed of SURF 3,9 . Then the FLANN algorithm is used to matching. An input image will be cropped to obtain ROI and have a database of reference/sample object. SIFT is used to extract key point both input image and reference image and apply PCA to the SIFT algorithm. Next step is compared to find the same key point. If the key point matching number is greater than a threshold level, the input object coincides with the reference object. Finally, the Homography algorithm is used to map the input image with the reference image plane. PCA is a transformative method that reduces the large number of correlated variables to a small set of variables such that the new variables are linear combinations of old variables that are not interrelated. It makes the processing is faster.

Calculate the center of the object
The center of the object is the position at which the robot will pick up the object. When the object is picked at that point, the weight will be distributed evenly, the object will not fall. In general, for any polygon with n edges have the following formula: Where the coordinates of the vertices of the polygon are (x i , y i ).

Calculate the rotation of the object
During picking, the object must be swiveled to fit the tray and not fall out. The swing angle must be optimized to save time and energy. For triangles, if zero is the point with the smallest y coordinate, the other two points are numbered 1 and 2. The swing angle is between line 12 and the horizontal axis. On Figure 7, the angle of object is the angle of line 12 relative to the horizontal axis:

Science & Technology Development Journal -Engineering and Technology, 2(SI2):SI127-SI136
On Figure 8, the formula (3) calculates the rotation angle as follows: For optimal speed, the angle which makes the machine's travel time is smallest is computed. In the other words, the tangent of the rotating angle is less than one. The formula for the rotation angle will be:

CALIBRATION
The object is picked up rely on the position of the object against the coordinates of the robot. Therefore necessary to change the coordinates of the camera to the coordinates of the robot so that the coordinates of the object can be determined and picked exactly. The calibration method in this project uses three circular objects located at three vertices of the rectangle on the conveyor. The coordinates of three vertices relative to the camera coordinate system and the coordinate system of the robot are calculated Three circles objects locate at three vertex of the rectangle on the conveyor and determine the position of three vertex in the camera coordinate system and in the robot coordinate system. On Figure 9, the relationship of two coordinates system as follows: By using the basic geometric theorem, the coordinates of any point in the coordinates of these three points is calculated as follow the formulas (6).

SOFTWARE CONTROL INTERFACE
The console is written on the Visual Studio software and the QT library. Visual Studio provides an environment for QT programming on it. QT library used is 64bit version for faster processing speed. QT supports powerful interface programming tools. The interface, as shown in Figure 10 and Figure 11, consists of the following main functions:

THE ALGORITHM FOR PICKING AND SORTING OBJECT ON THE CONVEYOR BELT
This section will include algorithm diagrams for picking and sorting objects on the conveyor belt. Control algorithms are written and coordinated between three major devices: computer, Atmega328 MCU, and SCARA YK400X robot controller ( Figure 12).

Control algorithm on SCARA robot
The first time, robot will reset all variables and outputs, move to home point and wait for the control signal from the computer (Figure 13). When a control signal from the computer, robot moves to the required point to pick up the object, then moves to the destination point to drop it. At the end of the process, the robot sends a signal to the PC that means ready for new control signals.

Control algorithm on computer
When started, the program will check to see if the camera has calibrated or not (Figure 14). When the objects is running on the conveyor, two images will be taken at each processing time. From two pictures will predict the coordinates of the object. The program will arrange the objects in order from right to left, calculate the direction of the object, and send coordinates to the robot. Then stop taking photos from the camera, send a grip signal to the robot. Program will wait for the robot to finish, send the signal confirm back to repeat the process again.

The results of sort by colors and shapes
Randomly dropping 50 objects with the specific number of each item as the following Table 1. As a result, an average of more than 2 seconds to pick up an object. Robots processing precisely at high speed. The number of times cannot pick up (2 times), the robot has identified and moved to the pickup position, but due to the pneumatic valve is not working well due to longevity.

The results of sort by characteristic
Classification results are based on the characteristics of five types of patterns as show in Figure 15. The percentage achieved is the number of identifiable characteristics between the reality image and the sample image. A threshold will be chose to conclude when identifying.
Percentage identity from different types of patterns is quite high. In the process of identifying the object on the conveyor belt, the object also moves along. The color of the object obtained on the camera will be partially blurred. The reduced characteristics leads to reduced recognition percentages. In addition, the ability to identify also depends on the complexity of the pattern.

CONCLUSION
In this article, the task is pick and classify objects on the conveyor belt with different shapes, colors as well as patterns. The objects are (circle, triangle, and square) randomly arranged on conveyors in different directions. Our methods is identified, picked and classified objects in an effective way. This method can be proposed to solve picking and sorting objects with different shapes, colors, and even complex patterns products running on conveyor belts. Identification of objects is sometimes affected by unstable lighting conditions, so some light bulbs is used to create a stable light environment and not be affected by external light.