Matrox Imaging Library (MIL) Tools

3D vision

Situations sometime arise where classical 2D vision techniques are unable to perform the required localization, recognition, inspection, or measurement tasks. These circumstances range from an inability to obtain the necessary consistent contrast from conventional illumination to needing the pose of an object with six degrees of freedom. This is where 3D vision tools step in—whether alone or in combination with 2D vision tools—to carry out the job.

MIL X has a rich set of tools for performing 3D processing and analysis. These tools work on the 3D data produced by profile and snapshot sensors as well as stereo and time-of-flight (ToF) cameras. Consult the Camera Interfacing section for a list of qualified makes. The 3D data supported by MIL X can also come from a Stanford Polygon Format (PLY) or stereolithography (STL) file.

Point-cloud (meshed) filtering

3D processing tools

The 3D processing tools in MIL X operate on—and in between—point clouds, depth maps, and/or elementary objects. The latter can be a box, cylinder, line, plane, or sphere. Operations on a point cloud include filtering (denoising)⁴, rotation, scaling, translation, cropping/masking, re-sampling, and meshing into surfaces; computing normal vectors; projecting to a depth map; and extracting a cross-section. Operations on a depth map include addition, subtraction/distance, and minimum/maximum; filling gaps (i.e., caused by invalid or missing data); and extracting a profile. Additional operations on both a point cloud and depth map include establishing a bounding box, computing the centroid, counting the number of points, and calculating the distances to the nearest neighboring point.

A depth map can subsequently be analyzed using MIL X's 2D vision tools like Pattern Recognition—without being affected by illumination variations or surface texture—and Character Recognition, when the alphanumeric code to read protrudes from, but has the same color as, the background. A profile or cross section can be analyzed using Metrology.

Pattern Recognition on depth map

Character Recognition on depth map

Metrology on cross section

3D surface matcher

MIL X includes a tool for finding a surface model – including multiple occurrences – at wide-ranging orientations in a point cloud. A surface model is defined from a point cloud obtained from a 3D camera or sensor, or from a CAD (PLY or STL) file. Various controls are provided to influence the search accuracy, robustness, and speed. Search results include the number of occurrences found and for each occurrence, the score, error, number of points, center coordinates, and estimated pose.

Surface Matcher

3D shape finding tool

MIL X provides a tool for locating specific shapes – cylinders, (hemi)spheres, rectangular planes, and boxes – in a point cloud. The shape to find is specified either numerically or from one or two previously defined elementary objects.. Several settings are provided to tune the finding process accuracy, robustness, and speed. Results include the number of occurrences found and for each occurrence, the score, error, number of points, and center coordinates. Additional results include the radius for spheres and cylinders, length(s) for cylinders, rectangular planes, and boxes, central axis and bases for cylinders, normal unit vector for rectangular planes, and number of visible faces for boxes.

Cylinder Finder

Sphere Finder

Rectangular Plane Finder

Box Finder

3D Blob Analysis

3D blob analysis tool

The 3D Blob Analysis tool in MIL X is available to locate and inspect objects in a point cloud. It makes it possible to segment a point cloud into blobs, calculate numerous blob features, filter and sort blobs by features, as well as select and combine blobs. Multiple segmentation methods can be selected: point distance, color distance, or similarity of normal vectors.

3D metrology tools

MIL X includes a toolset for 3D Metrology. Within this toolset, one tool fits a point cloud or depth map to a cylinder, line, plane, or sphere. Additional tools compute various distances and statistics between point clouds, depth maps, and fitted or user-defined elementary objects. Another tool is available to determine volume in a variety of ways.

3D Metrology

3D alignment for comparative analysis: Model, target, and difference

3D registration tool

An additional 3D Registration tool in MIL X establishes the fine alignment of two or more point clouds and merges them together if required. This tool provides the means to perform high-accuracy comparative analysis between a 3D model and target, do full object reconstruction from multiple neighboring 3D scans, and align the data from multiple 3D camera or sensors.

3D profilometry tools

MIL X also contains tools for 3D Profilometry using a discrete sheet-of-light source (i.e., laser) and a conventional 2D camera. A calculator is included to establish the camera, lens, and alignment needed to achieve the desired measurement resolution and range. MIL X provides straightforward calibration methods and associated tools to produce a point cloud or depth map. The calibration carried out in MIL X can combine multiple sheet-of-light sources and 2D camera pairs to work as one, thus avoiding the need for post alignment and merger. Such configurations are useful to limit occlusion, increase scan density, and image the whole volume of an object. Moreover, MIL X makes use of a unique derivative-based algorithm for beam extraction or peak detection, which is both more accurate and robust than traditional ones based on the center of gravity.

3D Profilometry

3D (eye-in-hand) Calibration

3D calibration tools

In addition, MIL X provides the necessary calibration services to position and orient a camera or sensor in 3D space and determine the coordinate transformation matrices between it and the rest of a robotic cell. The camera or sensor can either be fixed overhead and/or to the side of the robotic cell (i.e., eye-to-hand calibration) or mounted at the end of the robot arm (i.e., eye-in-hand calibration). The calibration enables to relate what a camera or sensor sees—the object pose established using other analysis MIL X tools—to a robot controller for guidance.

More tools