The team at Scorpion Vision Software has been working with the transition from 2D machine vision to 3D machine vision since 2006. Our goal is to make this transition transparent for the user. Scorpion Vision Software is a very flexible, rapid application development framework to create the most reliable, most robust and most accurate industrial 2D and 3D machine vision solutions. I believe that 3D machine vision is the most important development in machine vision software at the moment
Machine Vision Development from 1990 – 2010
When I was first introduced to machine vision in 1997 we evaluated Checker one of the first smart cameras from Image Industries in UK. The potential of vision sensors were obvious when seeing this system running in real-time processing more than 50 images / seconds. The setup was menu driven and required only a clever engineer to handle. The image processing was only pixel based. I have worked with machine vision since this introduction. After moving to PC based machine vision with excellent lens calibration procedures, I realised that pixel based processing is very limited with respect to accuracy. With lens calibration and sub-pixel processing one can achieve 10 to 20 times the accuracy than with pixel based algorithms.
Fundamental performance gain and cost saving with sub-pixel measurement
The fundamental performance gain and potential cost savings in lens calibration and sub-pixel measurements techniques are immense. It took me many years to really understand and take advantage of this fact. The importance of sub-pixel measurement means that with the right software it is possible to get a better, more accurate and more robust system with a web camera resolution 640×480 and a plastic lens than with a 4 mega-pixel solution with high quality lens. This is of course only true in the calibrated 2D plane as is the limitation of a 2D machine vision system.
What will the future bring?
Four years ago in 2006 when we added 3D camera calibration to our own Scorpion Vision Software I was intrigued by the fact that 3D camera calibration contains information about how every camera pixel moves in space and the power of this.
More robust 3D Robot Vision
We implemented basic stereo vision with two and three cameras and had the ability to measure distance or Z with one camera – mono pose or two or three cameras using stereo vision. We used these basic 3D capabilities for robot vision and upgraded a number of only partial successful 2D robot vision solutions to 3D. This significantly reduced the stop rate caused by the machine vision solution being able to estimate the distance to the objects to be picked.
2D robot vision is being obsolete
In 2010 we do not recommend 2D robot vision anymore. I believe that 3D camera calibration is the next fundamental advance in machine vision after the invention of pixel based processing and sub-pixel measurement. This makes 2D robot vision obsolete when you know better.
Faster and more accurate 3D Camera Calibration
There will be advances in 3D camera calibration. The procedures will be automated to improve the accuracy and the reliability. This automation will also significantly reduce the time to recalibrate a system. The different methods for calibration will also be improved. A popular method used in academia and research is based on planar checkerboard patterns. While simple in concept we do not always find it practical in the industrial context. Here, it may be better to have a fixed 3D reference object or to use the robot itself for the calibration.
Benefits in 3D Robot Vision
The benefits in 3D robot vision are obvious:
- Works in physical coordinates
- easy to verify – can verify against cad drawings
- Object location does not degrade when the object position changes
- It is possible to detect that something is wrong
- in 2D machine vision there is normally no quality measure
- The systems are more accurate
- The system should never fail
3D Stereo Vision will start to replace standard 2D machine vision solutions
3D Stereo Vision is my favorite method for creating 3D images or extracting 3D information from a scene. Other methods like laser triangulation, stripe-light, time of flight and other structure light approaches are of course also important but I will leave these for now. The properties of 3D stereo vision are:
- Low cost due to inexpensive high quality digital cameras
- Fast 3D data acquisition
- Progressive scan cameras can capture 3D images of moving objects
- It is fast to extract 3D data with stereo vision
- 3D information can be measured in the 2D images
- High accuracy – 3D feature location with sub-pixel accuracy in x,y and z
The successful stereo vision software implementations will be based on the most accurate 2D sub-pixel feature location and have a true 3D reference system concept. With 3D references one can re-sample or work in any object plane in the 2D images. This means that one seamlessly can move back and forth between 2D and 3D space and combine 2D and 3D machine vision algorithms. It is easy to verify how good a 3D solution is, just extract the values from the 3D model of the measured part. When everything is right it just fits within the accuracy of the system typically within 0.1 or 0.01 mm.
Easy to use 3D Visualisation
To be able to work efficient in 3D machine vision it is important to have the following functions:
- Draw 3D objects in any 2D Image
- True 3D Image visualisation to show
- lines
- objects
- planes
To verify camera calibration or your 3D scene, visualisation of camera positions in the same model can be useful. Calibration errors and other errors can then easily be detected by inspecting the 3D image.
Future Innovations in 3D Stereo Vision
We believe that the value of 3D stereo vision systems will be even higher when someone invents the ability to convert a 2D edge into a 3D polygon model or uses the multiple cameras to extend the 3D field of view of a 3D stereo vision system in contrast to the standard concept of only being able to locate objects limited by the intersection of the 2D camera’s field of view.
3D never fails
The best 3D machine vision systems will never crash or fail. This means that they will never output a result which is wrong. Based on the redundancy in a 3D stereo vision system there should be reliable quality data available. When the quality is bad the system will not output void data but indicate that something is wrong to enable the master system to act in accordance.
Reduced cost of ownership with state-of-the-art 3D solutions
In 3D robot vision there are great productivity gains from a more reliable and accurate machine vision infrastructure. With a 3D system 99.9 % location rate or better can be guaranteed. Improving the accuracy 10x with 3D sub-pixel measurements can increase the value of a system or lower the hardware cost significantly or maybe make new innovative designs feasible.
All in all the value of a good 3D solution can be much greater and in many cases the total cost of ownership is significantly lower than a traditional 2D solution.
TordivelBlog 