There are targeted machine vision edge devices that have AI accelerators embedded in them to implement AI at the edge. And, of course, there are implementations of AI in the cloud as well as in the fog (the local cloud).

What does all this mean? It means that AI is here to stay. It gives the ability to address previously difficult or intractable applications such as flaw detection. It also tends to lure people into addressing applications using AI that are solved, perhaps better, by conventional image processing approaches.

For example, if the application needs to recognize a manufactured part, there are well established processes to perform recognition, and these processes are very effective. While AI can also recognize a manufactured part, it may actually take more engineering effort than using an existing image processing process.

On the other hand, a flaw, say a scratch, has a unique shape for which there is no template. Programming flaw detection can be a very tedious and time-consuming effort with lots of testing and refinement of the code. AI can address finding flaws in a rather straightforward manner. However, there are challenges in getting the training right including collecting a suitably representative set of sample images for training.

Because there are so many companies, established and new, promoting their AI products in the market, competition is fierce for sales. Promises are rampant. Not all promises will be realized. Soon, there will be places where AI is less welcome because of past failures. The market descends into what is known as the valley of disillusionment. Eventually, as the market aligns with fewer players establishing traction and building track records, the benefits of AI will be realized, and the market will grow.

Expect new AI technologies to emerge that enhance the efficiency and effectiveness of AI.

Figure 2 -- NVIDIA Orin Nano Edge Processor. Image Source: NVIDIA

Cameras and Image Sensing

Cameras continue to expand the available offerings. One trend is toward ever higher resolution image sensors with more pixels. 100 megapixel cameras are readily available with cameras up to 250 megapixels also available. As the image resolution of cameras exceeds 20 megapixels, the camera cost rises rapidly.

Fortunately, the trend toward smaller pixels has slowed, for the moment. Smaller pixels challenged optical design for lenses and delivered poorer signal-to-noise performance. A threshold on pixel size around 2.35µm seems to be the lower limit at present. Perhaps there will be technical breakthroughs that address the optical challenges and the poorer signal-to-noise of the smaller pixels. If those challenges are met, then the trend to smaller pixels is again inevitable. One would expect image sensor cost to decrease with smaller pixels leading to less expensive cameras. But the cost of the technical advances (e.g., in optics design and fabrication) may offset any camera cost savings.

Another camera trend is SWIR (short wave infrared) cameras becoming more available and less expensive. While SWIR cameras have existed for some time, Sony’s introduction of the IMX990 and IMX991 image sensors launched a new generation of SWIR cameras that are less expensive than previous SWIR cameras. These sensors are based on indium phosphide (InP) and indium-gallium-arsenide (InGaAs) semiconductors rather than silicon like most common image sensors. They have a spectral response range from 400nm down to 1,700nm; much wider than silicon’s 200nm to 1,000nm range.

The lowering cost of cameras imaging into the SWIR range are enabling new applications where the formerly unexplored SWIR wavelengths reveal useful information such as the oft-cited bruising on fruit or imaging through colored plastic that is opaque in visible light.

Three-dimensional imaging is still growing with classical techniques of structured lighting, laser profilometry, and stereo being the dominant methodologies. Some novel techniques such as time-of-flight (TOF) imaging are in limited use. Their resolution limitations restrains where they will work.

Hyperspectral and multispectral imaging continue to show promise in machine vision for applications in recycling plastics and textiles, sorting food, and monitoring agricultural crop health. The expense of cameras and the large amounts of data to transmit and process are limiting their more widespread adoption.

Figure 3 -- Allied Vision Goldeye SWIR Camera. Image Source Allied Vision

Camera interfaces

Although there are many interfaces for machine vision cameras, the big five still dominate: Camera Link, GigE Vision, USB3 Vision, CoaxPress, and Camera Link HS.

Camera Link, although an older technology, is still popular. No further development of this interface is expected.

GigE Vision is progressing with speeds increasing to 5 Gbps, 10 Gbps, 25 Gbps, and even 100 Gbps. As the speed increases, cable lengths may need to be shorter, but at 100 meters originally, it is still one of the longest camera interfaces. At speeds over 10GigE, the protocol will change from UDP which has no data packet resend ability, to RDMA which is more efficient in the processor and does include packet resend capability to ensure reliability at the higher speeds.

Since its inception, USB3 Vision has gone through two doubling of speed from 6 Gbps to 20 Gbps. USB4 is coming and will give even higher speed up to 40 Gbps. However, the standard will still be called USB3 Vision.

The current version of CoaxPress, version 2.1, provides a bandwidth of up to 12.5 Gbps over one cable. Four parallel cables give 50 Gbps speed. Version 3 is in the planning stages and will likely extend to even higher speeds.

For embedded systems, the MIPI and SVLS interfaces are growing in popularity.

Figure 4 -- Ruggedized Lens. Image Source: Moritex, a Cognex Company

Lenses

Lenses advance less rapidly than other elements of machine vision mainly due to the laws of physics and the glasses available for design. Still there have been notable advances including more lenses ruggedized for applications in moist environments where mechanical shock and vibration are concerns.

Lenses have advanced some to handle the smaller pixels and to handle larger image sensor sizes. Principally, these advances have come with more advanced lens designs and more precision in the lens manufacturing.

Other lenses that have become available to enable new applications include those that image visible through near infrared (NIR) over the range of 400nm to 1,000nm maintaining good resolution and minimum chromatic aberration. This range corresponds well to the response range of silicon allowing even more applications that require imaging in the NIR.

The range of lenses supporting SWIR imaging is also growing along with SWIR cameras that facilitate imaging in this range. The price of SWIR lenses is decreasing as the volume of these lenses increases.

Offerings of telecentric lenses are increasing due to interest in their advantages for making precision measurements.

To help keep vision systems more compact, there is a growing interest in wide angle lenses. Effort is underway to mitigate the lens distortion common in wide angle lenses. Both optical and software techniques are under development.

Interest in liquid lenses is increasing as they open up more opportunities to adjust focus, either programmatically or automatically.

Figure 5 -- Line of Chip-on-Board LEDs. Image Source: ProPhotonix

Lighting

Lighting is making some evolutionary progress. While very few machine vision applications now use halogen lighting, the availability of halogen lamps will become more difficult as Europe bans them in a couple of years except for certain exempt applications.

LED illumination continues to dominate machine vision lighting and should continue to do so into the foreseeable future. LEDs though are getting more efficient allowing more light output for a given amount of input power. Chip-on-board (CoB) is becoming more popular with light source manufacturers allowing higher packing density of LEDs and better thermal management. This also leads to higher light output for LED light sources.

Software

Machine vision software continues its bifurcation between ease of use and maximum flexibility. For ease of use, the software packages provide a guided form to allow building a machine vision application with low-code or no-code solutions. This frees the developer from needing to know details of image processing or how to write computer code. Many machine vision software packages also allow access to the software development kit (SDK), a library of image processing functions that can be linked with a user-written program, to address an application’s needs.

As new technologies, such as AI, are introduced, there will be refinement through tools created for specific applications using these technologies. Vision system developers will be relieved from significant work in applying these new technologies when these application specific tools are available.

Computational imaging

An emerging field of machine vision image processing is computational imaging. Computational imaging is a relatively new arm of machine vision. It involves combining multiple images algorithmically to create a single more useful image.

One technique that meets this description, yet is not considered computational imaging, is structured light imaging for 3D surfaces. The common approach to structured light is to project a series of patterns, usually bars of light, onto the scene and take images of each pattern. Combining all these images can give a profile of the imaged surface as a single image.

An emerging use of computational image is photometric stereo that can detect surface artifacts not easily imaged with other techniques. It uses three or more, typically four, images each acquired with illumination from a different direction to compute an image showing reflectance or surface height change revealing fine structure such as defects.

Another instance of computational imaging is super-resolution in which multiple images of the same scene but from slightly different positions are combined to create very high-resolution images. The different images can come from individual cameras that take images simultaneously from different positions, or the images can come from the same camera with the image sensor shifted a fraction of a pixel in different directions between each exposure.

Summary

While machine vision has matured since coming onto the scene in the later 1970’s, there is still much progress being realized. With the ability to borrow technology from photography, military reconnaissance, medical imaging, remote sensing, and other imaging applications, machine vision will continue to evolve new and better approaches to solving the need for industrial automation imaging.