The resolution of this system.
Artificial vision for manufacturing is a very important step when the process is automatic, so to improve productivity with high quality, this investigation presents a new method for manufacturing by applying artificial intelligence for tool selection; in the proposed method, a camera takes a picture and artificial program is applied to process the image and thus generate it to the size of the cutting tool. This chapter describes the development of a new method by programming using artificial vision to select the cutting tool. The experimental results show that the combination of artificial vision and programming is capable of selecting the correct tool.
- artificial vision
- cutting tool
- artificial intelligence
In recent years, artificial vision is impacting the manufacturing process due to the mass adoption in different lines of research such as robotics [1, 2], machining [3, 4], and automotive , among others. However, numerous variables affect the machining process, for example, tool materials, total error compensation, concepts of autonomous manufacturing, and process condition monitoring, among others. On the other hand, the computer vision for cutting condition effects as roughness (presented by Sarma et al. ), the prediction of cutting conditions (by Gadelmawla et al. , and path generation are included for milling Computerized Numerical Control (CNC) machines (by Eladawi et al. ).
At present, the development of a system capable of recognizing complex features for tool selection in commercial computer‐aided manufacturing (CAM) transfers this task to the workers who do it based on their own experience, ability, and knowledge. The literature on automatic tool selection using image processing is minimal; on the other hand, simple and easy to implement algorithms are the core of this method .
2. Artificial vision in manufacture
This research presents a new technique based on artificial vision using image processing for automatic tool selection in operations of milling‐turning manufacture.
The proposed method starts recognizing an image taken by a photographic camera. Then, different techniques of image processing are applied, such as binarization to convert the image of gray scale to black and white; after segmentation to reduce the contour or increase it if necessary; directional morphology to determine the magnitude, position, and direction of edge of the part; and finally, a structural element with the shape of the tool is moved to generate the trajectory and dimensions of the cutting tool .
3. Artificial vision in lathe
This research consists of eight steps for machining 2D objects mainly for two‐axis lathe machines: in step 1, an image designed in CAD or taken from a camera is extracted and saved to a DXF file; the next step (step 2) consists of transforming the DXF file to the BMP file (image); in step 3, a binarization is applied to convert the image to grayscale; step 4 consists of labeling to separate the object of the picture; in step 5, the perimeter of the piece is obtained; in step 6, a partial derivative to obtain the gradient in the edge of the piece is applied; in step 7, a structure element with the shape of the tool is moved in the image; and finally, in steps 8 and 9, a condition is applied to determine if there is no intersection between the selected tool and the object to generate the selection of the tool and the trajectory .
The method for automatic selection of the tool consists in identifying when an element structure with the shape of the tool travels in each pixel on the edge of the part in the image. If there is an intersection between the cutting tool and the workpiece, then the diameter of the cutting tool is automatically changed until there is no intersection. These steps and the previous steps are shown in Figure 1, as well as in the following steps :
When different transformations are applied, it is necessary to know the dimensions of the piece and keep the precision nearest between the part in physical (length in mm) and the image (length in pixels), an analysis applied is shown in Table 1 .
|Resolution||Scale||mm||Length (pixels)||Height (pixels)|
|800 × 600||4:3||1||15||11|
|3200 × 2048||25:16||1||62||40|
|5120 × 4096||5:4||1||100||80|
|7680 × 4800||16:10||1||151||94|
|10,000 × 10,000||1:1||1||196||196|
|50,800 × 50,800||1:1||1||1000||1000|
The calibration value is also the resolution of this system and is approximately 1 μm. A method of data extraction  is applied to convert the DXF file to BMP file, as shown in Figure 3(a) and (b). The same distance is then automatically determined in pixels in the image, and a calibration value is obtained by dividing the distance in millimeters by the distance in pixels to obtain a relation of mm per pixel:
All the dimensions are known in the DXF file. In order to find the directional gradient of the edge, image processing is applied.
where is the total number of pixels in the image and is the region of the image. After starting with the image that is shown in Figure 3(b), a transformation (thresholding) is applied to convert the image of gray scale to binary using Eq. (3):
Each pixel has a value of 255 representing the largest tonality (white) and 0 when the minimum (black) cannot work with this whole range of colors.
Or it can also be written as Eq. (5):
Another way to calculate an integral is the arithmetic sum of each pixel of the image represented with the symbol
Figure 4(b) shows the labeling of the perimeter
There are several methods to obtain the gradient (variations in intensity of pixels) (Eq. (9)), but among them the Sobel method is selected for its computational easeas is shown in Eqs. (10) and (11) , the gradient generates directional vectors, orientation of the piece, and direction of the contour of the workpiece:
Figure 5(a) illustrates the vector direction of the edge of the part sample.
Figure 5(b) shows the directional angles according to the trajectory of the edge, when the gradient is applied to binary image in the part.
The magnitude and direction of the edge are saved in the variable using Eq. (14) to generate the dimension of structural element:
The workspace is considered according to the size of the piece but it is necessary to consider other collisions as the turret with the part, these will be considered in future issues . Table 2 shows the common cutting tool, the dimensions of the cutting tool are obtained from manuals and introduced into the software to be converted into structural elements with the length, orientation, and radius of the nose, this part is shown in Figure 6(a). Among the common dimensions are those of 15°, 35°, and 45°, with 9525 mm requiring 36 pixels, 2540 requiring a mesh of 25.5 square units of pixels, and the structural element requiring 414 pixels, see Figure 6(a) .
There are several conditions for machining between them and it is necessary to determine if the cut is internal or external, and whether it is an insert right () or left (). The common are those that are situated in the last two quadrants, the first two quadrants are generally used for internal machining. Eq. (15) generates the movements of the structural element through the image; the angle marks the start of the selection of the cutting tool in machining
where is the dimension for cutting tool presented in Eq. (16):
Table 2 shows the common inserts that are generated in structural elements. Figure 6(a) illustrates a structural element generated with pixels to be moved through the contour so that the contour matches the nose radius of the tool. The number of pixels in a line according to the used resolution of 800 × 600 is five pixels; the evaluation is taken from reference . Second, the pixels that are represented the line in Figure 6(a) are used only to demonstrate the method; the pixels processed are the inner using centroid method.
When a structural element passes through the edge of the workpiece, it is necessary to know if this insert is correct, free of intersection, if there is intersection, and if another insert is treated. The structural element (Figure 6b) is displaced through all the images, when a structural element (Figure 6b) collides (Figure 6c), other structural element is chosen automatically, this process is determined in the software itself. The technique for automatic tool selection may be to change the radius, length, or angle of the insert according to the inserts, obtained from handbooks .
After the correct holders and inserts have been selected to work on each area, many changes in cutting tools are probably required. The number of changes can be reduced by ordering the changes with the next procedure (Eq. (17)):
To determine the tool path, each pixel should share information with eight neighbors and follow the path of the perimeter (edge
Figure 7(a) illustrates the method of zig using one way, but there are other methods of creating paths such as zigzag of two paths (back and forth), zig with contour, which follows only the path of the edge of the piece widely used for finishing and roughing at the same time; the follow‐periphery that has the function to be used only for finishing; the trochoidal profile is used in special cases when the piece has peaks, elevations, or very long inclinations; and on the other hand, the generation of trajectory is used only for rough cutting. To generate the trajectories, a structural element with the shape of the cutting tool with angle , length, and noise angle is displaced through all the images from right to left.
Machining time is given by Eq. (19):
4. Results in lathe
With the proposed method, an image can be directly taken by a camera or other devices for automatic tool selection. The use of an image generated from DXF file is to compare the structural elements (part and tool) in pixels using PI; the original dimensions from DXF file are used to compare them if the design comes from CAD file. This new method, which reduces the complex, complicated, and difficult to understand mathematical algorithms, shows an easy simulated viewing for selecting cutting tool. To show the method for automatic selection of the tool, two pieces were used, the results are presented in Table 2. The generation of trajectories is shown in Figure 7(a) and (b), and the selection of the tool in Table 3 for the first piece.
|Rough turn||Finish turn|
Table 3 shows the results of the selected inserts by applying the proposed method. Where the numbers represent the area for each insert, and L, R, or N represents the types of inserts more common than before which was proposed relative to its size.
The second example is presented in Figure 8. The selected cutting tools are presented in Table 4, where an existing thread machining is represented by the number four and an internal machining is labeled number five.
|Rough turn||Finish turn|
The developed software is shown in Figure 9. It was developed in Microsoft Visual C++ 2010 and tested in 2D images generated in any CAD software and exported to DXF file.
In this work, a new method for automatic tool selection using techniques of image processing for computer numerical control lathe machines has been presented. The proposed methodology has been implemented in images generated by CAD software and exported to DXF file. The resolution of this system is determined as approximately 1
From this, it is clear that this methodology can be implemented in commercial CAM software in order to systematize the lathe CNC process. The novelty of this article is the use of image processing to automatically generate the selection of the cutting tool, the design of the part can be from a photograph taken from a camera or other device or directly from a CAD.
6. Artificial vision in mill
A 2.5D solid model can be defined as a cut with a series of 2D tool paths at different Z slices of a 3D solid model. Nowadays, a large percentage of pieces used in the industry have a shape of contours, where the base face is a plane denominated by 2.5 axis. Automatic tool selection in milling operation is one of the important steps of process planning. Moreover, at present, the commercial computer‐aided manufacturing software transfers this task to the worker who does it based on its own experience, ability, and knowledge. Considerable articles have reported about the variables that affect the milling process such as material piece selection [7, 12], tool selection , cutting conditions , tool materials , tool sequence , cutting fluids tool selection , tool path , and control and identification, among others.
The tool selection, task commonly made by a human operator, is an important aspect in machining processes, since if the tool selected is incorrect, it can produce dimensional errors in workpiece, possible crashes, and consequently, reject the piece. In this way, some researches based their principles on the selection of the tool for machining 2.5D parts as in the case of Ahmad et al. , who present an optimization algorithm for the problem of tool sequence . Or the method proposed by Lim et al.  who used experimental algorithms using mathematical Boolean to determine the optimal setoftools in pockets with the integration of CAD/CAM.
In the same way, Hemant et al.  developed an algorithm for the tool selection used by Veeramani tool, although using dynamic programming, human intervention is necessary because of its mathematical complexity . The tool selection using image processing based on the shape is a new method in the literature, the core of this research issimple and easy to implement algorithms .
Although the image processing plays a very important role when the images are manipulated inside the morphology (opening operation), the main research is the defects by Tunák et al. , the cutting conditions by Sarma et al.  and Gadelmawla et al. , and the path generation by Eladawi et al. [6, 8].
The contribution of this research is the easy way to simulate the cutting path, automatic tool selection, easy method for 2.5 axis machining in operation of milling, reduction of errors, and not requiring prior knowledge. The novelty of this work is the use of morphology based on the image processing applied for the automatic tool selection. The advantages of this method are: easy to implement, eliminates the decisions and errors when the tool is selected .
Application of directional morphology in the tool and the workpiece with the sole aim of finding the selection of the cutters for 2.5‐axis machining in the bibliography is a new proposal. The method begins by taking the mesh of a 3D object exported to a DXF file (Mejia et al. ) or an image taken by a camera. To manipulate this file, it is necessary to extract the coordinates, lengths, and positions of the pixels generated on the frontier of the piece on the surface. The pixels are discretized on the edge of the piece to obtain normal vectors and stored in a text file. The file data gives the necessary information about the orientation of tool when it passes through that place. The dimensions of the tools are stored and discretized; these were extracted from the handbook more common in the industry. The resolution of the system depends on the size of the tool and the complexity of the piece. To improve the algorithm, it is necessary to reduce the size of the pixels, after the pixels of the tool are displaced on the frontier of the piece. Each movement of the tool is inspected if there is no collision.
The image processing techniques used are: (a) morphological operations: using erosion and dilation [17, 18], these methods reduce or increase the contour of the workpiece; (b) binary image: the pixels that contain a gray scale
The work published by Bithika and Asit  applies mathematical morphology to detect manufacturing defects. It is an example of the application of image processing, using such techniques for measuring the effect of cutting speed on the surface roughness, which is another example that provides a new strategy in manufacturing, as presented by Sarma et al. .
This research consists of six steps for machining 2D and 2.5D objects mainly for 2.5‐axis lathe machines, an image designed in CAD or taken from a camera is extracted and saved to a DXF file, after transforming the DXF file to the BMP file (image), a binarization is applied to convert the image to grayscale, the labeling to separate the object of the picture, the perimeter of the piece is obtained, a partial derivative to obtain the gradient in the edge of the piece is applied, in step seven, a structure element with the shape of the tool is moved in the image, finally, a condition is applied to determine if there is no intersection, using mathematical morphology, especially erosion and dilation to generate automatic tool selection and tool path as it is shown in Figure 10 .
When different transformations are applied (DXF to BMP), it is necessary to know the dimensions of the piece and keep the precision nearest between the part in physical (length in mm) and the image (length in pixels), an analysis is applied, the calibration value of this system is in micrometer, the adjustment parameters for 1 mm of 800×600 (scale 4:3, 15×11 pixels) and 50,800×50,800 (scale 1:1, 1000×1000 pixels) .
The archive DXF file contains the necessary information about the part that is necessary when the DXF file is converted to a BMP file, as shown in Figure 12(a) and (b). The calibration is the relation between the distance in millimeters () and the relation of millimeters per pixel ().
Binary image: a binary image (
Detection and labeling of gradient and perimeter in the edge of the piece: The following step is the binary label, or limitation, of the piece
Perimeter of the piece: The perimeter () can be found by applying an edge () into the image () to provide information about the shape of the object, and the labeled perimeter can be defined as the position vector for the edge. Figure 14(b) presents the result of the trajectory of the labeled perimeter represented with label in the point of the image.
The magnitude () and direction () of the labeling gradient and perimeter () of the edge are saved in the variable
Directional morphologies to generate size of tool, erosion, and dilation are fundamental operations in morphological image processing, defined for binary images. To grayscale images and to complete lattices, dilation () and erosion () are formed by a structural element.
With functions with respect to and .
Dilation is a joining of the translations of one picture for each pixel of an image B, called structural element as
With functions with respect to and .
The set of edges reduction, elimination of white dots, and the expansion of the small black dots of an image B are called erosion:
Modifying Eqs. (29) and (35) of directional morphology to generate the structural element of insert and tool path using the workspace boundaries, edge, and the perimeter function of the piece generating Eq. (39) and as results provided by Figures 16 and 8:
where () is the first piece of structural element and () is given by
Table 5 shows the most common cutting tool. The dimensions of the cutting tool are obtained from manuals and introduced into the software to be converted into structural elements with the length, orientation, and radius of the nose, this part is shown in Figure 16(a). Among the common dimensions are those of 15°, 35°, and 45°, with 9525 mm needing 36 pixels, 2540 requiring a mesh of 25.5 square units of pixels, and the structural element having 414 pixels, see Figure 17(a) .
The cutting tools are shown in Table 5, where is the diameter, is the tip diameter, is the major length, is the minor length, is the usable length, is presetting, and is the corner radius of the cutting tool. The tool is generated from handbooks; although, a real insert as the drilling cutting tool with a diameter of 5.56 mm (, 21 pixels) requires a structural element as a tool of pixels. For more details see Figure 17(a).
The conditions for machining to determine the cutting tool are given by Eq. (42) that generates the movements of the structural element through the image, and the angle labels the start of the selection of the cutting tool in machining
The structural element (Figure 17a) is designed to create a trajectory using Eq. (42). To check the intersection (Figure 17b), it is necessary that the pixels have the same coordinates. The design of the software to create the automatic tool selection is developed in Microsoft Visual C ++ 2010, generating a matrix with coordinates (
After the correct holders and inserts have been selected to work in each area, many changes in cutting tools are probably required. The number of changes can be reduced by ordering the changes with the next procedure:
To determine the tool path, each pixel should share information with eight neighbors and follow the path of the perimeter (edge ) in the mesh .
In the present article, zig and zig with contour were developments to generate the trajectories. A structural element with the shape of the cutting tool, with diameter
9. Results in mill
A new method is presented for tool selection using directional morphology. To validate the method, three examples were proposed.
Figure 19(a) and (b) depict machining with tools CB and C with labels “4” and “3,” respectively. Figure 20 shows four selected tools, using pocket in zigzag with contour, applying dilation because the machining is external. After this, in Figure 19(c), the machined area (labeled with “2”) can be seen. In the same way, the piece finally machined with a B tool is shown in Figure 19(d). Figure 19(e) depicts the final workpiece obtained. In Table 6, parameters of cutting tool selected by the proposed method can be seen.
|Rough milling||Finish milling|
|Area||Type||D (mm)||D (mm)|
The second piece is shown in Figure 20. In this case, the piece is machined using zigzag with contour and boundaries. An erosion operation, to automatically select the tool, applied to the surface to remove pixels is internal by the geometry desired. The three contours and approximations of machining with different tools marked with labels “3,” “2,” and “1” are depicted in Figure 20(a), (b), and (c), respectively. Finally, Figure 20(d) depicts the machined piece. Table 7 shows the parameters of the three different tools selected by the method (see Table 8).
Figure 21 shows the rough mill with the last piece using zig with contour. Figure 21(a)–(c) with their respective tool selections. Figure 21(d) shows the finish milling and Figure 21(e) the final piece.
|Rough milling||Finish milling|
|Area||Type||D (mm)||D (mm)|
|Rough milling||Finish milling|
|Area||Type||D (mm)||D (mm)|
Figure 22 shows the software developed in Microsoft Visual C ++ 2010 for the automatic tool selection.
10. Conclusions of mill
Image processing for manufacturing three‐dimensional models that require movements in three axes based on the directional morphology to detect collisions when the workpiece and the tool are moving is a new method in the literature. The automatic selection of cutting tool and the generation of tool paths to manufacture pieces in three‐axis machining have been presented. The advantages of this method arefast and easy‐to‐implement programming, the proposed method can correctly select cutting tool; traditional methods of morphology as dilation and erosion in conjunction with edge piece to create tool dimensions and automatic tool selection are used. In order to diminish the possible error in boundary obtaining, a DXF file was used to compare and correct it when an image is utilized with a resolution of 1 μm.
Perez Paina G., Araguas G., Gaydou D., Steiner G., and Rafael Canali L., RoMAA‐II, an open architecture mobile robot. Lat Am Trans IEEE (Revista IEEE Am Lat). 2014; 12(5): 915–921.
Jimenez Moreno R. and Brito M., Path planning for a mobile robot in a 3D environment. In 2014 IEEE Biennial Congress of Argentina (ARGENCON), 2014; 125–129.
Gadelmawla E.S., Eladawi A.E., Abouelatta O.B., et al., Application of computer vision for the prediction of cutting conditions in milling operations. , Part B: J Eng Manuf. 2009; 223: 791–800.
Al‐Kindi G. and Zughaer H., An approach to improved CNC machining using vision‐based system. Mater Manuf Process. 2012; 27(7): 765–774.
Boix E., Ferrer A., Fernandez S., and Sellart X., Vehicle guiding system through image processing in crash and misuse tests. In Proc SAE‐China Congress 2014: Selected Papers. 2015; 411–424.
Eladawi A.E., Gadelmawla E.S., Elewa I.M., et al., An application of computer vision for programming computer numerical control machines. Proc IMechE, Part B: J Eng Manuf. 2003; 217: 1315–1324.
Sarma P.M.M.S., Karunamoorthy L., and Palanikumar K., Surface roughness parameters evaluation in machining GFRP composites by PCD tool using digital image processing. J Reinf Plast Compos. 2009; 28: 1567–1585.
Mejia‐Ugalde M., Trejo‐Hernandez M., Dominguez‐Gonzalez A., Osornio‐Rios R.A., and Benitez‐Rangel J.P., Directional morphological approaches from image processing applied to automatic tool selection in computer numerical control milling machine. Proc IMechE, Part B: J Eng Manuf. 2013; 227: 1607–1619
Mejia‐Ugalde M., Dominguez‐Gonzalez A., Trejo‐Hernandez M., Morales‐Hernandez L.A., and Benitez‐Rangel J.P., New approach for automatic tool selection in computer numerically controlled lathe by applying image processing. Proc IMechE, Part B: J Eng Manuf. 2012; 226: 1298–1308.
Mejia‐Ugalde M., Dominguez‐Gonzalez A., Trejo‐Hernandez M., Morales‐Hernandez L., Osornio‐Rios R., and Benitez‐Rangel J., Triangulation intersection approach from Poissons equation applied to automatic tool selection in computer numerical control mill‐lathe. Proc IMechE, Part B: J Eng Manuf. 2014; 230: 722–731.
Chen T.H., Chang W.T., Shen P.H., and Tarng Y.S., Examining the profile accuracy of grinding wheels used for microdrill fluting by an image‐based contour matching method. Proc IMechE, Part B: J Eng Manuf. 2010; 224: 899.
Siller H.R., Vila C., Rodriguez C.A., and Abellan J.V., Study of face milling of hardened AISI D3 steel with a special design of carbide tools. Int J Adv Manuf Technol. 2009; 40: 12–25.
Ahmad Z., Rahmani K., and D'Souza R.M., Applications of genetic algorithms in process planning: tool sequence selection for 2.5‐axis pocket machining. J Intell Manuf. 2010; 21: 461–470.
Lim T., Corney J., Ritchie J.M., and Clark D.E.R., Optimizing tool selection. Int JProd Res. 2001; 39: 1239–1256.
Ramaswami H., Shaw R.S., and Anand S., Selection of optimal set of cutting tools for machining of polygonal pockets with islands. Int J Adv Manuf Technol. 2011; 53: 963–977.
Maros T., Vladimír B., and Testik M.C., Monitoring chenille yarn defects using image processing with control charts. Tex Res J. 2011; 81: 1344–1353.
Mallik‐Goswami B. and Datta A.K., Detecting defects in fabric with laser‐based morphological image processing. TexRes J. 2000; 70: 758–762.
Li J., Huang P., Wang X., et al. Image edge detection based on beamlet transform. J Syst Eng Electron. 2009; 20: 1–5.
Tufoi M., Vela I., Marta C., et al. Optimization of withdrawing cylinder at vertical continuous casting of steel using CAD and CAE. Int J Mech. 2011; 5: 10–18.