Computer Vision
Mission
Section titled “Mission”The Computer Vision Team empowers Mujin’s robots to perceive the world. In unstructured environments like logistics warehouses, robots need to handle millions of different SKUs, variable lighting, and dynamic obstacles. We build the eyes that make this possible.
Core Capabilities
Section titled “Core Capabilities”3D Perception
Section titled “3D Perception”We utilize high-end 3D cameras and custom sensors to generate accurate point clouds of the workspace. This allows the robot to know exactly where objects are in 3D space.
Object Detection & Pose Estimation
Section titled “Object Detection & Pose Estimation”Using advanced machine learning and geometric algorithms, we identify items to pick—whether they are boxes, sacks, or loose parts—and determine their precise orientation for grasping.
Machine Learning
Deep learning models for segmentation and classification of novel goods.
Geometric Vision
Model-based matching for precise industrial parts handling.
Innovation: Interactive Calibration Unique
Section titled “Innovation: Interactive Calibration ”One of our key innovations is autonomous calibration. Traditionally, calibrating a camera to a robot is a tedious manual process. Our system can automatically calibrate itself by having the robot move a target around, ensuring sub-millimeter accuracy in minutes, not hours.
Challenges We Solve
Section titled “Challenges We Solve”- Transparent/Reflective Objects: Handling difficult materials that confuse standard depth sensors.
- Occlusion: Figuring out how to pick an object that is partially hidden by others (bin picking).
- Speed: Processing massive point clouds in real-time to keep up with the robot’s cycle time.
Tech Stack
Section titled “Tech Stack”- Languages: C++, Python, CUDA
- Libraries: OpenCV, PCL (Point Cloud Library), PyTorch, TensorFlow
- Hardware: Industrial 3D Cameras, GPU computing