The automotive industry has used machine vision technology longer, better and more creatively than any industrial sector.

“Vision is a mature technology that has been widely embraced in the auto
industry,” says David Dechow, principal vision systems architect at Integro Technologies Corp. “Because many automakers and suppliers have been at the forefront of using vision systems, there are few production processes today that are not touched by the technology. It’s an integral part of most assembly lines.

“The automotive industry is also at the leading edge of applying state-of-the-art vision technology, such as color inspection systems, deep learning, smart cameras and 3D systems,” explains Dechow.

“Robotic vision is increasingly being used for inspection of parts, welding and material handling,” adds Toni Roda, global product manager at ABB Robotics. “Increasing demand for customization, labor shortages and cost pressures are major drivers for the use of vision in the auto industry.”

Automotive OEMs and their suppliers use machine vision to inspect such diverse components as air bags, batteries, brakes, cooling modules, door panels, engine valves, fuse boxes, wheels and windshields.

Advanced machine vision technology plays a key role in the daily operation of Ford Motor Co.’s Van Dyke transmission plant, the recipient of the 2018 Assembly Plant of the Year award. The high-mix, high-volume facility produces 70 different automatic transmission models used in a wide variety of Ford and Lincoln vehicles that feature front- and all-wheel drive.

Vision-guided robots ensure high-quality assembly of critical components such as gears and clutches. The Van Dyke plant has more than 500 inline machine vision applications that address error proofing, gauging and complexity issues.

One application involves robots and vision systems that dispense sealant. A six-axis robot applies a bead of room-temperature vulcanizing silicone at variable quantities, speeds and angles. The application is validated with a two-camera vision inspection system that ensures the correct path and width of the bead.

Vision systems are also used at Ford’s Van Dyke plant for a variety of gauging and inspection applications. For instance, clutch packs consist of three different components that require a particular sequence or stack up. Vision technology measures height and sequence based on complexity requirements.

In addition, vision systems are used on all external transmission electrical connections to ensure that, after end-of-line testing, pins are within a tolerance frame for straightness in vertical and horizontal position.

Unique Challenges

In the automotive industry, the main goal of machine vision technology is to inspect for defects and confirm that a part or subassembly is correct before moving on to the next step in the production process.

“Automotive applications can be large in scale and have harsh environments, so having a vision system that can withstand this environment is key,” says Amanda Nelson, product manager at Banner Engineering Corp. “Depending on where the vehicle is in the assembly cycle, different inspections will need to be performed.

“Having a flexible system that can detect for very different things, such as presence of welding seams, dispensing of glue beads or correct part placement, will increase throughput and allow for standardization on one system,” explains Nelson.

“The automotive industry is continually becoming more automated,” Nelson points out. “Machine vision is replacing the need for human visual inspection, reducing errors, creating a vehicle history trail and improving quality. In many cases, it helps speed up the inspection process required at many steps within the automotive assembly process.”

“Automotive manufacturers are increasingly relying on machine vision to make sure their products meet stakeholder’s expectations,” notes Fernando Callejon, vision product manager at Omron Automation Americas. “Vision has primarily been used for quality inspection purposes, but instead of simply finding and rejecting defective parts, it’s also starting to be used for tracking important data on all steps of the manufacturing process as customers demand greater operational visibility.

“In many cases, machine vision is seen as a non-value-added activity inside automotive plants, because it’s not modifying anything on the final product,” claims Callejon. “But, these plants are starting to take advantage of the fact that inspection via machine vision significantly improves their operations, [enabling them to] become more repeatable, reproducible, efficient and effective.”

According to Callejon, many automotive engineers now prefer to use multiple smart cameras rather than the complex, centralized vision systems that used to be common. “Today’s smart cameras are getting smaller, more capable and more affordable, so it’s getting easier to introduce machine vision into areas where it was cost-prohibitive in the past,” he points out.

“As manufacturers inspect more and different targets, the typical resolution of cameras has increased to be able to see more,” adds Nelson. “2D vision is still the go-to vision technology today because of its flexibility, robustness and repeatability.”

But, the automotive industry was one of the first adopters of 3D vision technology. And many engineers are intrigued by its long-term potential.

“3D vision is playing a key role in the automation of final trim and assembly,” says ABB’s Roda. The company’s FlexVision 3D system is already used in a variety of body-in-white and power train assembly applications.

“[Traditionally, this part of automotive] assembly has a low degree of automation compared to stamping, body-in-white and paint processes where robots [have been used for a] long time,” explains Roda. “Vision is key for the automation of applications such as cockpit installation and door assembly.”

“We’ve been working on a project with an automotive manufacturer that involves using machine vision to inspect the joint quality of interior components,” adds Dechow. “However, the multiple variations are proving to be a big challenge.

“Things such as armrests, buttons, door handles, panels and speakers all need to be inspected with the same system,” says Dechow. “And, they need to be handled with the same precision and reliability regardless of part location and variation.”

Bin picking is another challenging application that can benefit from 3D vision systems.

“A 2D camera is unreliable in bin picking, because  its accuracy is affected by changes in ambient lighting and there are problems detecting low-contrast or reflective objects,” claims Jason Niu, marketing manager at Bluewrist Inc., which has developed a hardware agnostic bin-picking system. FlexiPick enables a robot to pick up parts randomly placed in a bin and present them to an assembly line.

“As the bin becomes empty, 2D vision systems have difficulty maintaining reliability and accuracy that may lead to collisions,” explains Niu. “However, demand for vision systems is complex, because manufacturers need one type of product that can serve different car models, vehicle platforms and multiple variations.

“In the past, due to the large amount of data that 3D sensors generate, it was not always feasible to apply the technology to inline inspection processes, because it would slow down the entire assembly line,” notes Niu. “Applications that involved shiny metallic surfaces were not possible.

“Today, things have changed,” Niu points out. “Sensor accuracy and speed have improved, enabling them to keep up with the cycle time requirements of the auto industry.”

New Technology

Automotive engineers can choose a variety of innovative products that make it easier than ever to use advanced machine vision technology on assembly lines.

Banner’s iVu Color sensor is designed for several applications. “In addition to the traditional inspection tools also found in the standard iVu, the iVu Color adds the ability to detect if a specific color part is present,” says Nelson. “With our device, it is possible to detect that a door handle is present and in the correct location and that the interior is the correct shade.

“The sensor also offers the ability to setup in only minutes without a PC,” claims Nelson. “This makes the iVu Color a powerful, timesaving solution for automotive applications.”

Canon developed its RV series of image sensors specifically for robotic bin picking. Products range from the RV300 to the RV1100.

“They are capable of high-speed, high-accuracy three-dimensional recognition of objects,” says Grant Zahorsky, machine vision engineer at Canon USA Inc. “The system is applicable for parts with curved surfaces, parts with few distinguishing features and even parts with more complex shapes.

“[Our product] projects recognition patterns onto randomly piled parts and analyzes the projected images,” explains Zahorsky. “Based on the analysis of the differences between images of the parts and the multiple projected patterns, the system is able to recognize targeted objects in three dimensions.”

Engineers at Omron recently developed the FHV7 smart camera. It can be used for numerous applications, such as accurate inspection of parts mounting points on different automobile models without moving cameras.

“[The camera provides] a modular concept that lets users select their
configuration while it adapts flexibly to changes in the manufacturing environment,” says Callejon. “Its multicolor light module, for instance, mixes different colors (red, green, blue and infrared) in a single device to compensate for product variations, while providing enough contrast for stable inspection.

“The FHV7’s dual-core CPU provides enough power to eliminate the need for expensive vision systems in smaller applications,” claims Callejon. “It also allows the system to log inspection data for traceability purposes without slowing down the process.

“High-resolution image sensors of up to 12 megapixels make it possible for users to avoid multi-camera systems that require image stitching operations to successfully perform quality inspection and precision measurements on large parts,” adds Callejon. “High-speed autofocus liquid lenses adapt quickly to various working distances and object heights without the need to have multiple cameras dedicated to different inspection needs.”

A new vision system recently unveiled in Europe automates the time-consuming and error-prone process of manual fuse inspection. The EVT Car Fuse Inspector makes it easier to check various fuses installed in a fuse panel.

“Depending on the size of the panel, the system uses one or more color cameras,” says Michael Beising, CEO of EVT Eye Vision Technology GmbH. “It analyzes the color images for the position of the fuses to ensure that the right fuse is in the right position.

“With the Car Fuse Inspector, it’s easy and cheap to do an inspection during the final assembly of the fuse panel and be sure everything is OK,” claims Beising. “And, you can quickly change between different panels of different types of cars and configurations.

“The system can inspect at full production speed so you can check a different fuse panel with each production step,” explains Beising. “That’s important, because nearly every car now gets a unique fuse panel, due to all of the multiple features and options that are available to customers today.”