Automatic Optical Inspection of Soldering

Automatic Optical Inspection (AOI) or Automated Visual Inspection (AVI) is a control process. It evaluates the quality of manufactured products with the help of visual information. Amongst its several uses, one is the inspection of PWB (Printed Wiring Boards) after their assembling sequences i.e. paste printing, component placement and soldering. Nowadays, surface mount technology is the main method of assembly. It can be automated with ease. The increasing widespread use of SMT (Surface Mount Technology) in PWB assembly results in down-scaling of component size, increasing of lead count and component density. Parallel to this the latest manufacturing assembly lines have a very high rate of productivity. Not only is productivity required but a high quality also is expected. The quality requirements for electronic devices have already been standardized, e.g. IPC, ANSI-JSTD standards. Modern machines used in manufacturing lines such as paste printers, component placement machines etc. are capable of producing significantly better results than those required in normal standards specifica‐ tions. Nowadays, the capability of modern manufacturing machines now reaches 6 σ as usually applied specification norm and 5 σ for more stringent ones. Even so, manufacturing processes are still kept under constant supervision. There still occasions when even a modern assembly line fails to create fully operational devices.


Introduction
Automatic Optical Inspection (AOI) or Automated Visual Inspection (AVI) is a control process. It evaluates the quality of manufactured products with the help of visual information. Amongst its several uses, one is the inspection of PWB (Printed Wiring Boards) after their assembling sequences i.e. paste printing, component placement and soldering. Nowadays, surface mount technology is the main method of assembly. It can be automated with ease. The increasing widespread use of SMT (Surface Mount Technology) in PWB assembly results in down-scaling of component size, increasing of lead count and component density. Parallel to this the latest manufacturing assembly lines have a very high rate of productivity. Not only is productivity required but a high quality also is expected. The quality requirements for electronic devices have already been standardized, e.g. IPC, ANSI-JSTD standards. Modern machines used in manufacturing lines such as paste printers, component placement machines etc. are capable of producing significantly better results than those required in normal standards specifications. Nowadays, the capability of modern manufacturing machines now reaches 6 σ as usually applied specification norm and 5 σ for more stringent ones. Even so, manufacturing processes are still kept under constant supervision. There still occasions when even a modern assembly line fails to create fully operational devices.
Besides the "classic" electric tests, such as in-circuit-test (ICT) and/or functional tests, there are in-line inspection possibilities: automatic optical inspection and automatic X-ray inspection systems. Because of their capabilities and properties, mostly the AOI systems are used as inline quality inspection appliances. The main advantage of these systems is their ability to detect failures earlier and not only when the product has been assembled. AOI systems can be used to inspect the quality at each stage of the manufacturing process of the electronic device. Accordingly, there are real financial advantages by using such systems because the sooner a

Identifying PWB
AOI appliances identify the PWBs with the help of a separate built-in unit i.e. laser-scanner or by using its inbuilt functionality. On this basis, machines can decide what inspection is necessary. According to data contained in a barcode, the AOI system loads the appropriate inspection program. As barcodes ( Fig. 2.a) became more widely used, in some cases the amount of data that could be stored in them was too limited and this became a barrier to its applicability. To solve this problem, the so-called 'matrix codes' (Fig. 2.b) were developed. In [30] 22 types of linear barcodes and 48 types of matrix codes are described.

Inspection of bare PWBs
There are several possibilities, appliances and algorithms when inspecting bare PWBs optically. These are able to inspect the copper wire-patterns on a PWB surface with high precision. Optical inspection gives rapid and reliable results regarding the quality of the PWB. Electrical detection methods, (e.g. ICT, Flying Probes) are slower and more expensive. Bare PWB inspecting AOIs have a special name: Automatic Optical Test (AOT) systems. There are several research and survey papers about this topic [31]- [43] and two manufacturers now have AOT machines [44]- [49]. In Table I a comparison between these appliances is shown.

Inspections following SMT sequences
In the SMT assembling process there are three main phases where AOI plays an important role; after-paste printing, component placement and soldering. In the case of wave, r selective or partial soldering there are other sequences e.g. glue dispensing, through-hole component insertion etc. But SMT processes are used mainly for inspection in discussions about this method of assembly. Possible locations where AOI can be placed in an SMT line are: the postpaste, post-placement or pre-reflow and post reflow (Fig. 3.).
At each location AOI appliances have a special name. These are: Solder Paste Inspection (SPI, also known as Post-Printing Inspection), Automatic Placement Inspection (API. also known as Post-Placement Inspection) and Post-Soldering Inspection (PSI). The AOI systems able to inspect each manufacturing sequence are called: Universal AOI (UAOI). If there is a possibility that the equipment can inspect the finished product optically, it is then called the Automatic Final Inspection (AFI).

Solder paste inspection
According to PWB assemblers, it is very important that the quality of the print solder paste is inspected because it heavily influences the quality of solder joints. In some papers it has been reported that 52%-71% of SMT defects are related to the printing process [50]- [53]. As failures can be detected much earlier, this obviously results in cost savings. According to some other opinions, inspection of the solder paste is not so relevant: "Contrary to the common, frequently quoted opinion that paste faults represent the primary percentage or 70% of all faults in the printed circuit assembly process, this detailed analysis shows that those faults amounted to only 8.3%." [54].
Special AOI machines are able to inspect the quality of print of the solder paste. It is an important option because in case of failure, the product can be repaired with minimum cost and without scrap loss. The size of the print in the 3 dimensions examined (latitude, longitude, altitude) must fall within the limits specified. To measure these parameters, so-called SPI (Solder Paste Inspection) machines have been developed. These machines are able to inspect only one step i.e. paste printing, but they are cheaper than universal AOI machines. As the control of solder paste presence is one of the easier tasks, then only the width, length and position needs to be inspected and so several failures can be detected such as bridges [55]. This can be solved using image capturing (usually with the help of line scan cameras) and subsequent evaluation.
But to measure volume as well the paste thickness is equally as important. Comparison between 2D and 3D solder paste inspections are reported in [56] and [57]. There are several possibilities to enable the measurement of paste volume optically: laser scanner [58]- [63]; projected sinusoidal fringe pattern as in [64]; the development of this technique for solder paste geometry measurement in [65], [66] and some special methods shown in [67]- [69]. Nine manufacturers offer SPI systems [70]- [82]. Several different solutions have been developed in these appliances as can be found in the scientific literature, described above. Comparison between the different methods is shown in Table 2.

Automatic placement inspection
Inspection of the PWB after component placement is the next possibility. With this method possible placement failures can be detected and some defective paste printing phenomena as well. If there is a sign or mark on the component, it can be read and identified with the help of modern image processing algorithms even if it has more than one different-looking label type. APIs (Table III) are able to measure most parameters of components objectively e.g. X-Y shift, rotation, polarity, labels, size etc. [83]- [90]. Four manufacturers have this special appliance in stock. [91]- [94].

Post soldering inspection
Most manufacturers agree from a strategic point of view, that optical inspection after soldering has been completed should not be missed out. At the very least, the defective products must be eliminated because many failures are generated during soldering: "Forty-nine percent of the true faults were detectable only after soldering. These consisted of component and soldering faults. Forty-eight percent of the optically recognizable faults could not be recognized electrically." [54].
In consequence, this is the most important part of the AOI inspection process. Most scientific papers are preoccupied with this subject [95]- [123]. The quality of the solder joint (and the soldering process) can be inspected with the methods described in this section. The quality of the solder joints is determined from geometric and optical properties of the solder meniscus. These parameters determine the reflection properties of the meniscus which is formed from the liquid alloy during the soldering process. After cooling, the meniscus becomes solid and reflects illumination which means that we can evaluate it (Fig.5, Fig. 6). From these reflection patterns and with the help of image processing algorithms we are able to determine the quality of the solder joints.   Using reflection patterns is the basis of all papers that have been published in this field of study. There are two solutions: gray-scale or colour inspection. Supplier and appliances are shown in Table IV [124]- [126].
One interesting area is wave soldering. It needs different types of algorithms because of the circular solder shape and the pin. Some solutions for this kind of inspection are reported in [127]- [129]. A summary of possible failures and appliances that can detect them are shown in Table V.

Combined appliances
There are systems that are able to inspect more sequences. These are combined systems, namely API&PSI [140]- [146] (Table VI).
And the all-in-one machines are the UAOI systems, detailed: SPI&API&PSI [147]- [163] (Table  VII).     Assuming that the component is fully operational, these systems practically are able to prove that the whole circuit board is working correctly thus replacing the ICT. However, because they are usually connected to SPC (Statistical Process Control) servers, they can also provide much information about the SMT process itself and provides help as to how to improve it.
But of course there are disadvantages to using AOI systems. They are not able to inspect hidden failures such as soldered BGA (Ball Grid Array) bumps and usually the parameters of inspection algorithms cannot be adjusted perfectly. So from time to time they do not detect real failures which are called 'slip-through failures'. These are the most significant malfunctions during the operation of AOI systems because in these cases, they fail to do what they were programmed for. So the number of slip-throughs must be zero and -if they arise -close investigation is necessary to prevent and eliminate them. However if this occurs repeatedly, then the appropriate parts would seem to be defective. These are the pseudo-failures which can reduce productivity so their number should as close to zero as possible. [164] ALSO indicates some other image processing problems. The problems of AOI systems will be described in more detail later in the chapter.
Another disadvantage is that they are usually in the 'bottle-neck' of the manufacturing production line because they are not able to inspect the whole circuit board as fast as the line can produce them. Therefore, the practice is usually to place more machines behind each other to enable inspections to take place in parallel. Of course, this also has financial implications which should be taken into consideration.

Special AOI solutions -Inspection of lead-free solder joints, flexible substrates, wire bonding and semiconductors
According to RoHS and WEEE directives, lead-free solder alloys have to be used in commercial electronics. This has presented a new challenge for AOI systems because of the differing optical properties of lead-free alloy. Some solutions are shown in the following studies [166]- [173]. AOI has several further application possibilities in electronic device manufacturing e.g. semiconductor and wire-bonding inspection. These appliances need extremely high-resolution cameras to detect defects in the μm scale. Another interesting area is flexible substrate inspection. Some of these special inspections are described in [174]- [179].

Differences between lead-based and lead-free solder alloys
Solders that contain lead are available with a tin content of between 5% and 70%. The composition of the most commonly used lead solder is 63/37 Sn/Pb; this was the main type used in electronics manufacturing until strict controls were imposed on its use for environmental reasons. The homogeneity of the solder meniscus that formed was beneficial in that the melting point of eutectic solder really is manifested as a single point on the phase diagram; in other words the molten alloy solidifies at a specific temperature, rather than within a broader temperature range. The solidified alloy can be broken down into tiny lead and tin phases of almost 100% purity, without intermetallic layers.
In the case of non-eutectic solders, the crystallisation begins around cores of differing composition and crystal structure, and at differing temperatures, so that during the accretion of the individual cores the composition of the residual melt also changes. Due to this, in the case of lead-free, non-eutectic solder alloys, certain phases solidify earlier, and these solid cores do not form a completely mirror-like, smooth surface on the face of the solder meniscus (and naturally, they also cause differences in the volume of the material).
Lead-free solders usually contain tin, silver and copper. Compared to lead-based solders they have several negative properties: they are more expensive, their melting point is higher, and they give rise to problems that do not occur when soldering with lead (the phenomenon of whisker formation has still not been fully explored). Because their surface differs from that of lead-based solder alloys -which are much more even and mirror-like -they reflect light differently, so different procedures may be used to verify the presence and quality of the solder menisci.
In the case of tin-lead solders, the solidification of the melt begins around the cores that are solid at melting point, and the individual solid phases grow at a virtually consistent rate as the two elements separate from the melt. This is how the volumes that are rich in lead and tin become a smooth-surfaced alloy consisting of lead and tin patches, typical of eutectic solder, that can easily differentiated on the cross-section.
Lead-free solders do not usually form eutectic alloys, and exist in many variants with different compositions. Tin is usually alloyed with copper and silver, but there are also alloys containing, for example, bismuth and indium.
In the case of the non-eutectic alloys (the vast majority of lead-free alloys), however, one of the phases begins to solidify earlier, and the alloying metal concentration of this phase will be smaller than that of the melt. This means that the composition of the remaining part of the alloy, which is still in a liquid state, continues to change until the eutectic composition is achieved, when it cools and solidifies. As a result of this, the microstructure that is created has a greater surface roughness than in the previous case: as the eutectic melt ebbs away, the intermetallic crystals that were the first to solidify create a more irregular surface. This surface scatters light much more than the smoother, more even surface of the tin-lead solder; in other words the proportion of diffuse reflection will be greater than that of specular reflection.
An attempt to measure the two solders with AOI equipment using the same settings will probably result in several errors, because after the necessary image conversion procedures the images made by the equipment will differ. For this reason, it would clearly be useful to calibrate the AOI equipment specifically for the different solders.
In what follows we present a series of images of tin-lead eutectic and lead-free Sn-Ag-Cu solder alloys made using a scanning electron microscope (SEM). This instrument is not suitable for measuring the surface roughness, but it does provide an accurate, high-resolution image of the examined surfaces and of the two solder alloys with differing composition and surface roughness, showing the differences in height and material with spectacular contrast.  An important question is precisely what the roughness of the pattern formed on the surface of lead-free solder alloys depends on, and how "reliably" predictable the process of its formation is. Figure 11. SEM image of the surface of a lead-free solder meniscus In the above image the two types of solder can be clearly differentiated due to the rougher surface that is typical of the lead-free alloy. In places the surface looks quite similar to the one assumed by the microfacet model, which simplifies reality for the purpose of mathematical manageability; in other words, small semi-spherical formations can be observed side by side with each other. On other parts of the picture, however, areas with no unevenness are also visible; and we have taken the electron microscope image of an area that gives a good illustration of a particular property of lead-free, non-eutectic solder alloys (in this case a tincopper-silver alloy), namely that due to the unevenness of the surface it reflects the light more diffusely (in other words, it scatters the light more) than the smoother surface of a eutectic solder. In the applied Cook-Torrance model, the roughness of the surface is described by a single parameter, which describes the surface in average terms.
The above picture shows an SEM image of a cross-section in which the tin (light) and lead (dark) phases of the eutectic alloy are clearly differentiated.
The above image was made at a lower magnification (400x rather than 1500x), but the phase boundaries can still be made out, and the smooth meniscus surface typical of lead-based solder  The fracture in the solder meniscus seen in the above image is probably due to a contaminant, but within the fracture is a particularly clear example of just how uneven a surface can be formed by the lead-free solder alloy.
The surface roughness of the lead-free solder meniscus is visibly greater than that of the tinlead solder. Taking the scale bar as a guide we can also estimate that the size of the uneven protrusions that increase the surface roughness, in terms of both their breadth and height, is in the order of 10 μm. It is also worth noting that the simplification of the microfacet model described by the Cook-Torrance model is clearly visible, as a visual inspection reveals that the surface is not closely similar to the surface made up of tiny flat plates that is assumed by the microfacet model. This simplification, however, is more than made up for by the model's simplicity and general ease of use.

Measuring the surface roughness
To measure the surface roughness we used a Tencor Alpha Step 500 surface profilometer. Based on the 10 measurements of each solder, made on the lead-based (Heraeus F816 Sn63-90 B30) and lead-free (Senju Ecosolder M705-GRN360-K1-V) joints, the two solders yielded the following values: At first glance the measured surface roughness values appear realistic; the surface roughness of the lead-free solder turned out to be approximately three times that of the lead-based solder. The distribution of the roughness values for the lead-based solder was below 1%, which is satisfactory because the divergence between the shape of the actual solder and that modelled by the computer showed a greater error (a few percent), and because greater fluctuation than this can be expected to result from the differing heat profiles, printed circuit boards or solderhandling requirements of real production lines. The distribution of the surface roughness values for the lead-free solder was over 5%, which is due to diversity of the size and shape of the surface protrusions that appear with this type of solder, which the Cook-Torrance model handles using statistical simplification, by assuming the surface to be of a consistent roughness.

Simulation created with the computer model
We checked the measured surface roughness values by comparing the images made using optical microscopes with the computer-generated graphic representations. The Surface Evolver software uses finite element analysis to calculate the surface profile at certain points on the surface. In areas with a greater radius of curvature, where the energies are closer to each other in terms of magnitude (in other words none are dominant in comparison to any others) the software uses more measurement points, that is a denser grid, for displaying the graphic representation. Figure 16. Example of a graphic representation generated using the Surface Evolver software Of the models that use a formula based on the Bi-directional Reflectance Distribution Function (BRDF), which is based on a physical approach, the most widely used is the Cook-Torrance model, which has surface roughness as one of its input parameters and is also capable of handling Fresnel distribution. During our simulation we used this, in a Direct3D environment, so when generating the rendered graphics we were able to use the measured roughness values as input parameters.
The majority of optical microscopes -including the Olympus BX51 microscope used by me at the department -are capable of operating in bright field (BF) and dark field (DF) imaging mode.
In bright field imaging, both the incident and reflected light fall almost perpendicularly onto the sample, naturally through a focusing lens. Dark field microscopes, on the other hand, collect beams of light that arrive not perpendicularly but from the side, from below a given angle, through a lens, in the direction of the observer; in other words the beams of light travel in the opposite direction but along the same path as would the beams of light that enter perpendicularly but are diffracted, not reflected.
Dark field microscopy gives a good resolution and microscopes with this capability are usually more expensive, but they are eminently suitable for the detection of phase boundaries or the examination of surface irregularities highlighted by the side illumination. In the case of metals, in which the proportion of diffuse components is smaller and the incident light is reflected much more in accordance with the principle of optical reflection, bright field microscopy results in a darker image and in the case of observation along the z axis (from above), as is typical of microscopes, only the surfaces that are parallel to the horizontal are illuminated. We also modelled both of these different types of illumination using the Direct3D software.
What follows is a comparison of the images made using the optical microscope and the graphic representations rendered with Direct3D that most closely replicated the actual light and surface conditions. Where not indicated separately, the soldered joint (at the SMT resistors) is illuminated with scattered light.

Detailed analysis of AOI systems
As can be seen in the previous section, AOI machines handle several tasks. Much literature is dedicated to the intelligence of these systems, but from a technical point of view we can also examine other aspects. A large number of these AOIs work on high-mix-high-volume SMT lines where the most important key factors are the inspection duration and quality. The attributes of this system relate to the following sections:

software processing part
They work in close relationship to each other, so the speed of each has to be in sync. There are three well-defined mechanical constructions for an AOI system: • without special moving parts / drives inside • with PWB positioning table • with camera-module actuating unit The simplest case is when the working-process of the system does not include special positioning steps. The PWB is positioned/placed in "one step" into the field of camera system, an image is acquired and the PWB is then taken out for the next process. This could be of great benefit because the machine does not need to synchronize any movements during the image acquisition process. The speed affecting factor can thus be ignored. This is used typically in Automatic Final Inspection (AFI) systems. This does not mean that the system has to only contain one camera. For more complex applications the number of cameras can be increased.
More cameras mean more complex image transformation and manipulation tasks so it follows that these systems are only capable of use when looking at pre-defined areas.
In case of larger inspection areas, the systems are mounted with special drives which can move the camera system or the inspected part. The movements of these drives have to be synchronized with the process of illumination and image acquisition. When the system contains small number of cameras and the illumination devices are also built-in, then the module itself should be the moving section. When there are even more cameras, each with its separate illumination (matrix arrangement), then the PWB should lie on a positioning table.
Two big groups of drive systems are commonly used for this purpose. The first is the conventional electromechanical drive. It is used for some 2D paste inspection machines. Here the velocity of the camera can be constant, while in most cases it contains line-CCD sensor. The other type of motion system is the linear drive which is more accurate and faster and therefore in more frequent use.
The directional route of the moving part highly depends on a second factor, that of speed and the properties of image acquisition system. Here also, three main parts can be singled out: • optics / lenses • camera / sensor type • illumination module / lighting source The system has to get the necessary amount of information and resolution from even the smallest components. In the SMT field this means zooming down to a 10μm pixel resolution.
To ensure the constant magnification at all points of the entire Field of View (FOV) the use of telecentric optics is essential. This criterion enables the system to make the required sizemeasurements. On an image seen through traditional lenses, the apparent shape of components changes with the distance from the centre of the FOV, therefore sometimes making shape recognition a hard task.
But it is not just the permanency of magnification that is important, so too is the need to select the correct level. On one hand, the larger detection area of the image sensor can help solve this task, but it also increases the computational resources needed. On the other hand, higher magnification levels give a better resolution but at the expense of reducing the field of view. The best scenario is if the system is capable of optional magnification. Generally, a relative large FOV, between 10-25 cm2, could be used and only in certain cases should dedicated Field of Interests (FOI) should be zoomed out.
In most AOI applications, the LED based lighting is used for illumination purposes. But independent of the type of illumination source used, the amount of illumination should be only as much as is required. The optimum depends on the application. For example, a 2D paste or a through-hole-technology (THT) components solder-joint inspection system needs only just a small amount of illumination. As the number of failure types / inspection tasks increase so too the number of illumination modes also increase. The programmable illumination module is a good tool to develop lighting requirements for dedicated purposes, but it also carries the risk of inhomogeneous and reduced FOV.  Figure 24. Problem of inhomogeneous grey-level by ring-illumination types Fig. 7 illustrates two types of camera-illumination systems. The first system contains 4 cameras, the second only 1 camera. Both have LED-ring illumination modules. The grey-level distribution maps shown above have been measured with the same type of illumination and greyreference flat. The green areas indicate the valuable field of the camera. This example clearly points out the importance of the homogeneity. Of course this phenomenon is also present when the illumination system is multi-coloured.
Most optical inspection / control appliance decisions are based on image-processing methods that have been set experientially. The stress is on the word "experientially". Most of the AOI machines make some kind of template matching. These sample-templates can be colour or greyscale, stand from parts/windows or form a complete pattern. The machine can be 'selflearning or directed by means of an "external trainer". Due to the fact that the overall reliability of these machines is not 100%, the defined limits between good and bad classified patterns are not strict. In some cases it could be that just two pixels differ between the data provided. If the phenomenon which the system needs to detect is not so unambiguous, then it should search for another method to make the gap wider between the 2 classes.

Software questions
One of the most wide-spread criticisms against the principles and methods of automated optical inspection systems stems from a very interesting paradox. As we have mentioned earlier, the introduction of AOI devices in the manufacturing lines was a result of the growth in manufacturing process complexity. These inspection and control devices have to fulfill certain reliability criteria which need to be validated. But unfortunately, these validation processes can only be used to a limited degree because of the high-complex manufacturing process and the equally complex and highly varied appearance of devices under test (DUTs).
This contradiction invokes the conclusion that accuracy and reliability of AOI system depends very much on the competence and working quality of the engineers and operators, the correct management of the setting up and controlling the inspection devices. In reality, this sets out several very serious challenges to experts. The quality inspection algorithms have many parameters -in some cases several hundred -(image processing, region of interest, threshold parameters etc.). Their setup requires experience, intuition and inspiration from the process engineers themselves.
In addition, during parameter tuning, the engineers need to solve the following contradiction, where the difference between images showing correct and faulty components is often only a few pixels which need to be detected by the AOI devices (Fig. 8). In the case of incorrect parameter settings these small signals can disappear and the system classify a bad component as good ("slip-through"). Certainly this false classification is totally intolerable in quality inspection processes; therefore it is necessary to aim for the complete elimination of this possibility by fine tuning the algorithm's parameters. Unfortunately because of this, engineers can easily set the algorithm to be too strict, meaning also that some correct components will be dropped out during the inspection process. Although these "false calls" (also known as pseudo failure) do not cause catastrophic consequences nevertheless they are the source of a very serious problem. Namely, in this instance, the human operators performing the reinspection of components considered "faulty" can easily get used to the repeated mistakes of the AOI system. Therefore they can eventually take the inspection device's decisions out of consideration even where there is a cases of real errors. This implies that the reliability of the inspection device itself would be in doubt; the fact of which would result in one of the biggest catastrophic effects on AOI systems. In addition, it seems insignificant but it is important to note that many bad classifications slow the manufacturing process, decrease productivity and increase the product overall production costs. To avoid false calls, process-engineers need to reduce the strictness of the inspection parameters which -as we have mentioned earlier -is inconsistent with principle used by the parameter settings preventing the slip-through. In addition, AOI engineers need to cope with several other difficulties a major one of which is that the production process changes continuously e.g. the settings of devices on the manufacturing line have to be modified, and this needs to be followed also by modifications to the AOI devices. Therefore the need to monitor the inspection algorithms and adapt to different parameters is a serious challenge to the process engineers.
Furthermore, it is necessary to satisfy some practical requirements when selecting and adjusting the inspection algorithms. Usually, electronic factories manufacture more products in parallel in which several similar or identical components can be located. If all the components were to be inspected with a separate AOI algorithm, the code management, version tracking and fixing etc. would be impossible. Therefore, engineers often use only one inspection method for similar mountings to achieve simpler AOI algorithm version management. Unfortunately, this strategy cannot always be used successfully because of the very heterogeneous appearance of the same components. Fig.9. shows an image sequence of the C0805 capacitor which illustrates the enormous differences between images taken of similar components.
In this varied environment it is very hard to develop an inspection method which results in highly reliable classification of each type of image for the same component. In addition, a parameter setting process that reduces the number of bad classifications in case of one component influences not only the selected manufacturing line but has an effect on the whole factory. Therefore it can happen that whilst a parameter optimization process reduces the number of bad classifications in the first part of the factory, it increases them on other manufacturing lines. This paradox is one of the reasons why the AOI macro optimization process is a very long and "Sisyphean" task of AOI process engineers. A very interesting and important question is the optimization of classification thresholds. One of the most important requirements of an inspection system is high-level robustness, but this condition can hardly be guaranteed if the classification decision (namely whether a component gets "faulty" or "good" label) is dependent on only one pixel. Therefore the quality results close to the decision threshold need to be classified in a separate group ("limit error") and it is necessary to apply a different strategy to them. It follows that AOI experts -apart from the fact that they need to solve the optimization paradox mentioned earlier -have to strive to find such an algorithm parameter setting where during the classification, the number of components classified near the decision threshold are as few as possible. Efficiency of AOI appliances Materials Science -Advanced Topics 420 can be significantly improved with the help of macro optimization. In the first task, the pseudo rate was reduced while slip-throughs remains zero (Table VIII, Fig. 10  Secondly parallel pseudo and slip-through reduction were carried out (Table IX, Fig. 11, Fig.  12).  Another very serious question is about the parameter optimization process, namely how can the AOI engineers validate the new parameter values determined by the optimization process? Certainly a correction of a bad classification cannot be validated only by examination of the specified image, but it is necessary to check several other instances. Therefore, to execute a reliable validation process, the engineers have to collect a large image database ("image base") covering all cases as they occur in the best possible way. Unfortunately, creating a good and usable image base is a long and sometimes impossible task because of several -often contradicting -criteria. A manual image collection by the engineers is very time-consuming and in case of automatic systems (like AOIs) there is only a limited possibility because of the high number and varied type of data. Automatic methods are faster but during the collection, some falsely classified images can be put in the image base which makes the parameter optimization impossible. For example, if an image containing a faulty component is placed into the "good" part of the image base, the optimization process will try to adjust to the parameters that the AOI algorithm has classified the image as "good". As a result, the optimized macro cannot recognize this specified error which can indicate slippages causing the greatest type of inspection catastrophe.
The number of stored images is also a very important factor. If the image base contains too many images, the resources (processor, hard drive, network etc.) become overloaded and the optimization process can only be executed slowly. On the other hand in case of a small image base the algorithm validation is neither reliable nor accurate enough. If we suppose that the optimal size of image base is determined (and which cannot be exceeded) and relevant images are collected resulting in a reasonably good image base. At this point another question arises: how can the engineers update the database with new images? It is very hard to determine which images from the new image-set need to be stored and which images need to be deleted from the current image base. There are several criteria -such as the date created, the number of similar images etc. -which can be used as the basis of the updating decision but a precise numerical factor which shows the usefulness of pictures in the database is much more difficult to determine.
The aspects and concepts mentioned in this section have shown that the usage and perfect operation of automated optical inspection system requires human control and supervision. Although the devices' algorithms are able to execute fast, accurate, efficient, reliable, "assiduous" and continuous inspection (they appear to be much more suitable than human operators as a consequence!) without being fed sufficient intelligence they cannot adapt immediately and independently to changes in manufacturing. Therefore the quality inspection process can hinder the increased spreading of autonomous electronic manufacture.
Several researches and developments are focusing on the problem to redeem the status of the human operators' work and to provide help for AOI engineers. Very interesting research directions are in automatic algorithm parameter optimization methods. The AOI devices on the manufacturing line monitor the quality of the algorithms (number of false calls and slipthrough, if possible) and on occasions they adjust the parameters using the image base to create a better, higher quality algorithm. The engineers only need to take care of special cases like changing the lighting or creating new inspection methods. Although the automatic parameter optimization methods do not have to satisfy high real-time criteria, it is important to determine the optimized parameter values in a relatively short time. It is easy to verify that even in the case of having some dozen parameters; the analysis of all parameter-combinations takes a very long time (years) therefore heuristic search methods have to be used to solve the optimization problem. Certainly the automatic optimization methods also need to collect the relevant images autonomously to create the reference image base. This work sets serious challenges for optimization processes because of the problems and difficulties mentioned earlier.
As a summary, we can establish that AOI systems offer a powerful solution for a complex problem by means of simple principles, but the analysis of details can reveal several problems, difficulties and contradictions. Finding a solution for them is an essential condition for the automated optical inspection systems in the future.

3D Inspection
But the analyze-development is just one route for improving the AOI process. The other is the "extended" optical inspection system with measuring capability. The pioneers of this property are 3D SPI machines. In the last few years, a wide variety of these machines have been developed. The inspection in this application -checking the SMT printing process -means the 3 dimensional measurements of solder paste bumps. These bumps are shaped like cylinders or cubes so that the geometries and surfaces are relative simple. This fact makes the 3D optical techniques a viable option. Several measurement techniques are used for this process, some of these are shown in Fig. 13. The same is true for inspecting component presence, but for solder-joint detection these technologies are in their infancy at present. The shape of different components' solder-joints is complex and the specular surface also makes the task even more difficult. There have been a number of research efforts, optical 3D shape measurement technologies, based on several technologies as shown in Fig.2. Some of these researches can be found in the following studies [181][182][183][184][185][186][187][188][189][190][191][192][193][194][195][196][197][198]. Also some companies are in the development phase such as Koh-Young Technology [180]. So the evaluation of these geometries is as yet more difficult, but with the development of optical metrology there will be more AOI machines with measuring capability.

Further developments, the future of AOI
AOI systems are following the worldwide trends i.e. multi-task integration, adaptivity, speed, etc. There are already appliances that integrate optical inspection with repair functions: Ersa's AOI+R solution or optical and X-ray inspection together. Some suppliers have AOI+AXI or Viscom's AOXI (simultaneous inspection). Another possible area of development is the inspection speed. Faster image capturing (with larger FOV, faster camera positioning etc), parallel inspection of two PWBs are some possible ways for this to be done.
The other important area is adaptivity. Mainly adaptive illumination is the future of AOI systems. It would help to drastically reduce pseudo-failures rates and eliminate slip-through failures.
A third area is image processing. 3D inspection, neural networks, fuzzy systems, intelligent algorithms which will help to increase the efficiency and reliability of these systems.

Conclusion
Inspection systems are widely used to determine the quality of electronics modules after assembly sequences. Nowadays this is usually the automatic, non-contact and non-destructive process, such as automatic optical inspection (AOI), supplemented with automatic X-ray inspection (AXI) if necessary. These appliances inspect the ready or the incomplete printed wiring boards to determine the quality of it's given property in any technological sequence, such as paste printing, component placement or soldering. The rapid development of electronics module assembly manufacturing requiring parallel development of test procedures. The automatic optical inspection is potential multi-disciplinary research area, because from image acquiring, (illumination, the detection of the reflected light etc.) through image processing, to the evaluation each area can be optimized to reach to goal, that the qualification of the inspected object in the field of interest (FOI) by the used appliance, matches the specifications as stated. Most manufacturers agree that, from a strategic point of view, the optical inspection after soldering should not be ignored. As a consequence, this is the most important part of an AOI inspection. The quality of solder joints is determined from geometric and optical properties of the solder meniscus. These parameters determine the reflection properties of the