Machine-vision applications increasingly make use of one-package-does-all cameras.

Leland Teschler
Editor

A smart camera/vision sensor: The recently developed DVT 535C from Cognex Corp. is an example of a smart sensor that comes bundled with imaging software. Color tools for the device include those for sorting, color match, and defect detection.

Visitors to a recent trade show were able to get a firsthand look at what smart cameras can do via a foosball game demo. Vision Components GmbH programmed one of its VC4038E cameras to monitor the movements of the foosball and calculate the optimal position for a goalkeeper. A stepmotor moved the goalie around at the command of a camera-controlled positioning system. Many show goers tried to shoot the ball past the automated goalkeeper, but few succeeded.

The Iris P-Series from Matrox Electronic Systems Ltd. Is an example of a smart camera that runs under Windows CE.NET. Developers typically write application code for the camera on a PC and download it to the Iris over an Ethernet link. Matrox engineers say the release of Microsoft's Vista operating system won't impact development efforts for the company's industrial cameras.

Print-quality verification is one area where smart cameras are frequently applied. In this printing demonstration, two Vision Components GmbH smart cameras (left) check print quality. A third controls print registration. A fourth (right) verifies date imprints.

Start with an industrial camera and a frame grabber. Then add an image processor. Put it all in an enclosure you can hold in your hand. You have just assembled what today is commonly called a smart camera.

The idea behind smart cameras is to make something small enough to be positioned near rapidly moving machinery or on assemblies that themselves move within an automated system. This lets industrial vision be a candidate for applications at several points throughout a manufacturing or packaging process.

Moreover, smart cameras carry enough software to serve as a complete vision system. They can generate I/O signals in response to such problems as a bad label, a mispositioned part, or a missing feature. So they can be deployed throughout a production line to detect problems and defects as they arise. This sort of quick feedback helps keep manufacturing processes healthy.

The working definition of a smart camera has broadened over the years. It used to be that a typical device in this category included a CCD or CMOS imaging sensor capable of recording a scene, an associated lens system, an embedded computer, and a capacity for managing a few industrial I/O connections. Smart cameras were strictly high-end devices targeting fast production lines or complicated industrial vision problems.

Now, the smart-camera moniker also gets applied to less-sophisticated industrial-imaging products with more modest application targets. Smart cameras on the low end, for example, might contain just a color or a proximity sensor tuned to recognize a few key events transpiring in their sensing window.

Whatever the application, the sensing technology has an important bearing on price. Suppliers of high-end smart cameras, for example, say the sensor accounts for 30 to 40% of their costs. These devices can run into the $10,000 range. They typically include a CCD imager providing up to 2 megapixels (1,600 × 1,200) of resolution.

CCDs are a more-expensive imaging technology than CMOS but they make possible such features as short exposure times on the order of microseconds. CCDs also excel in applications characterized by high frame rates of 100 fps or more. Typical applications include reading small registration marks on pharmaceutical packaging whizzing by at perhaps 10 m/sec.

Smart cameras employing CMOS sensors may be able to hit a few of the same individual performance metrics as those with CCDs, but typically not in combination. The reason is that optimizing a CMOS sensor chip for one or two parameters tends to compromise its performance in other categories.

For example, short exposure times and the electronic shuttering that makes them possible come at the expense of fill factor, the amount of CMOS chip area available for use as light-sensitive elements. This is because shutter electronics must occupy what would otherwise be light-sensitive areas on the chip. CMOS cameras also have less dynamic range (Specifically, the ratio of pixel saturation level to signal threshold.) than CCDs. This is partly because CCDs have less on-chip circuitry that contributes to noise levels.

In general, the higher noise levels degrade image quality somewhat. This may eliminate CMOS cameras from consideration where precise imaging is important.

Both CCD and CMOS-based smart cameras digitize sensor data and transfer it via direct-memory access to a main memory, usually a synchronous dynamic random-access memory. The onboard processor found in smart cameras is typically a digital signal processor. The DSP typically processes image data and delivers typical cycle times of as little as 0.1 sec/part in high-end cameras running sophisticated image-analysis software.

Unlike other kinds of camera systems, smart cameras typically do not generate a video image for viewing in normal operation. They normally communicate with outside devices via 24-V outputs or similar industrial I/O for connecting to PLCs, actuators, and so forth. Smart cameras also typically carry Ethernet connections for transmitting images to outside computers, frequently as a means of archival storage or record keeping.

COMPARING APPROACHES
It is interesting to compare the smart-camera approach to industrial stand-alone cameras used in conjunction with PCs or an operator workstation. In the latter case, cameras typically connect to frame grabbers on PCs through FireWire, Camera Link, USB, or some other broadband connection standard. But bandwidth limitations of these connections can pose problems in high-speed applications. The speed of the connection may be such that the camera must first compress an image before sending it back to the frame grabber. For example, Camera Link provides the highest bandwidth of these standards and hits 1.6 Gbps in its base configuration. At the PC, the frame grabber decompresses and manages the image. The compress/decompress cycle necessary for each image limits the reaction time available for controlling processes or machines.

The fact that smart cameras bundle an image processor with a frame grabber eliminates this potential bottleneck. Close proximity of camera and frame grabber also make possible other operations that come in handy for special imaging tasks. For example, smart cameras can set shutter time for individual frames if necessary. Similarly, they can set the gain and offset for each frame because there is no delay in sending information between the frame grabber and the camera.

Similarly, close proximity between the camera and the DSP handling the image lets Fast Fourier Transforms and other calculations common in image analysis progress quickly. For example, the VC44666 smart camera from Vision Components GmbH carries a Texas Instruments DSP operating at 1 GHz. Its 8,000-Mips computing capacity lets it hit cycle times on the order of 0.1 sec/part depending on feature complexity.

Many of the applications for high-end smart cameras of this nature get programmed by users in C and C++ with help from development tools. Camera makers say this sort of do-it-yourself development affords users some anonymity. For example, a company might devise niche applications in-house this way to avoid divulging information about its internal processes to outside vendors or integrators.

Application programs typically run on the camera through a real-time operating system. In the case of the VC4466, for example, programmers get access to camera functions through a Linux-like RTOS called VCRT. Though most cameras in this class use a DSP, some use some type of Pentium and may run Windows CE or a similar embedded OS.

THIRD-PARTY SOFTWARE
Many camera makers provide canned routines that can be brought to bear on vision applications. The typical user interfaces for such programs are based on spreadsheet inputs, flowchart or icon-based programming languages, scripting, or menus. Users then draw common vision routines from libraries for such tasks as blob analysis, matching colors, or correcting distortions.

Camera vendors have different policies about software bundling. Some offer a software and camera combo. For example, vision supplier Cognex Corp. typically bundles its Intellect vision software with its DVT sensors and cameras. Intellect includes tools such as multiple-image analysis to check for product flaws such as scratches, porosity, and chipped edges. Also included are location algorithms for edges and objects.

However, more-sophisticated software tools tend to be a la carte. Allowing users to get special tools separately keeps down the cost of the overall system, particularly for applications that are relatively simple.

For example, Cognex puts out a separate pattern recognition package called PatMax. It targets complicated applications in areas such as integrated circuit manufacturing because it can, among other things, recognize patterns while tolerating changes in object appearance, detect partially hidden objects, and provide more accuracy than ordinary object location techniques.

MAKE CONTACT
Cognex Corp., cognex.com
Matrox Electronic Systems Ltd., matrox.com
Omron Corp., omron.com
Vision Components GmbH, visioncomp.com