Sensor to Robot Calibration

Guide to the calibration process of Sensor with respect to the Robot.

The Sensor to Robot calibration effectively aligns the robot's digital twin with the real-world view captured by the sensor.

Steps for Calibration

Prerequisites

Ensure the robot and sensor are correctly set up, and the Charuco Board is available.

Grab or place the calibration board

If your camera is mounted in a stationary way:

• Select the Interaction panel, switch to the Robot Control Mode, turn on the Grip Mode and grab the calibration board with the robotic arm

Tip: Instead of holding the calibration board in the gripper, use Velcro strips to temporarily attach it to the arm during the calibration process.

If your camera is mounted on the robot gripper/end-effector and moving with the robot arm

• Place the calibration board close to the camera so that it is visible to the camera. For example, on the table with the robotic arm)

Open the calibration panel

• Click the Configuration tab in the wrist panel.

• Click on the dropdown button next to the Cortex button, which will open Robots and Sensors buttons.

• Press on the Sensors button, which will open the Active Sensors panel

• Select the Sensor you want to calibrate in the list of Active Sensors; this will open the Modify Sensor panel.

• In Modify Sensor panel, click on the Calibrate button at the bottom of the right-hand side panel.

• Select Sensor to Robot calibration.

All the other sensor feeds will get disabled to help the operator to focus on the feed from the sensor, which is getting calibrated and removes confusion from the feeds of other cameras.

Note : This option will be disabled if the sensor is not attached to a robot. In this scenario, go to the Modify Sensor Panel and attach the sensor to a robot and mount type using the Robot Linkage button. Save the setting, and then try again.

• The type of mount selected when attaching the sensor to the robot will automatically select a suitable calibration type.

If the mount type of sensor with respect to the robot is:

Base : hand-eye calibration is selected, and the camera is mounted in a stationary way with respect to the robot

End Effector : eye-in-hand calibration is selected, the camera is mounted on the gripper/end-effector and moving with the robot

End Effector with Pose : eye-in-hand calibration is selected, like above but with camera data sent including synchronized pose data, currently only available in custom Extend Robotics setups.

Note: To change the type of calibration or the robot the calibration is performed with respect to then, go to the Modify Sensor Panel and attach the sensor to the required robot and mount type using the Robot Linkage button, save the setting, and then try again.

Performing the Calibration

You should see the Sensor Calibration panel with:

• Information such as: Calibration Type, Sample Size, Position Error and Rotation Error.

• Buttons: Take Sample,Reset,Save and Cancel.

The Position and Rotation Errors will be shown as zero until the fourth sample is taken, as the software doesn't have enough data to calculate the error previously.

Make sure Robot Control Mode is enabled in the Interaction panel to move the arm.

1. Move the end effector position and rotation of the joint to change the angle at which the camera observes the calibration board, and take your first sample by clicking on Take Sample.

2. Repeat Step 1 three more times to reach a sub-optimal point where the Sensor will partially align with the robot.

3. Press Save if you're happy with the first result, which will take you back to the Modify Sensor of the Sensor.

4. Finally, press Save to save the calibration done so far.

Usually, the first four samples only achieve a partial calibration with high positional and rotational errors. We recommend repeating Step 25 to 20 times to fine-tune the calibration and increase the accuracy between the digital twin and the physical robot.

In case the calibration gets broken during the process, you can click Reset to reset to the last save and try Step 1 again.

For best results, ensure the calibration board is visible to the camera during the whole process. While the Charuco calibration board supports partial views and occlusions, this limits the number of markers that can be detected and potentially decreases the calibration accuracy.

Alternatively, fine-tuning can also be achieved using manual calibration. See Manual Calibration for further instructions.

Improving calibration accuracy

Calibration accuracy may be enhanced by:

• increasing the number of robot movements

• maximizing the angular spread between robot movements

• minimizing the distance between the camera and the calibration target

• minimizing the distance moved by the robot arm between two positions

For a detailed technical description of the problem, see example:

A Comparative Review of Hand-Eye Calibration Techniques for Vision Guided Robots