Following are the steps required to add a new SenseKit:
Open the Configuration tab in the wrist panel.
Press on the SenseKits button, which will open the Active SenseKits panel.
Press on Add New button, which will open the panel with input fields.
Input the IP and port (default port is 9091) of the SenseKit.
Press Connect this will try to connect to the SenseKit device and find any configurations saved on the SenseKit device.
Previously stored configurations on the SenseKit device will be shown in the panel, and can be selected to load the configuration.
If no configurations are available, click on Create New Config button on the panel and define new configuration
SenseKit Configuration Elements
Name : This field is name of the SenseKit configuration
Mode : This field is used to the view option of the RGBD streaming
Pointcloud Only (default) : show 3D point cloud
Video Only : show 2D video
Pointcloud + Video : show 3D point cloud and 2D video
Depth Codec :
zdepth (default) : lossless depth compression with sensor based accuracy
yuvdepth : lossy low bandwidth depth compression, not supported on Jetson
Color Codec :
MJPEG : color compression with high bandwidth requirement
H.264 (default) : recommended color compression codec to use
Fusion Offset (m unit) : when fusing multiple sensors treat sensor as if data was reported with larger distance to avoid noisy data fighting for view, only small values recommended (0.01 - 0.1) as it may have side effects with VR scene
Cutoff Distance (m unit) : define to remove from 3D view data with depth larger than value, 0 to disable
Cutoff Replacement (m unit) : define to replace data at depth larger then cutoff distance and missing data with the value, typically set to 0 (remove) or matches Cutoff Distance to perform 2D-3D Fusion
If the SenseKit Firmware Specific Settings are not configured during the OTA Firmware Update Process or require an update, click on the Sensekit Firmware Specific Settings button.
Sensekit Firmware Specific Settings panel contains the following fields:
SenseKit Firmware Specific Settings
ROS Port (default 9091) : the port used for connecting the AMAS VR app to the SenseKit
Data Port (default 12345) : the port used for connecting the AMAS VR app to the RGBD streaming pipeline of the Sensekit
Color Compression Config : hardware acceleration to use. In typical scenarios default is set per sensor when installing firmware and should not be changed. When installing firmware on non Extend Robotoics hardware choose method supported by hardware.
nvenc_18_04.yaml : NVidia NVENC for Ubuntu 18.04 Bionic firmware (x86-64)
nvenc_motioncam.yaml : NVidia NVENC optimized for Photoneo MotionCam-3D (x86-64)
nvenc_zed_2i.yaml : NVidia NVENC optimized for StereoLabs ZED 2i (x86-64)
vaapi.yaml : Intel VAAPI (x86-64)
vaapi_amd.yaml : AMD VAAPI (x86-64)
vaapi_low_power.yaml : alternative Intel VAAPI low power encoding path (x86-64)
vaapi_motioncam.yaml : Intel VAAPI optimized for Photoneo MotionCam-3D (x86-64), may fail with non-standard MotionCam resolutions with firmware side iHD VAAPI driver and will work with i965 VAAPI driver (if hardware supports it)
Enable Audio : enable/disable audio support on SenseKit, this setting should be only enabled for SenseKits with audio hardware. When enabled, please follow the instructions at AudioKit Addition and Configuration to configure audio support.
Pose Sync Settings : toggle this option to send moving camera pose from reference to relative frame with the data (only available in setups with robot and sensor on single embedded computer). Do not enable this option if you have no intention to use pose data as waiting for it may add small delay.
Reference Frame : typically set to base of the arm link_base
Relative Frame : set to camera mounting point, for example end-effector camera mounted on arm with 7 joints would use link7
FPS : desired framerate of the sensor, currently supported only for Kinect4A, FemtoBolt, MotionCam-3D, ZED-X and ZED-XM. Note that typically sensors support only limited set of framerates and may fail, fallback to default or not reach non-standard framerate. Decreasing framerate may help if bandwidth is limited. Increasing framerate may improve real-time performance at the cost of computational resources and bandwidth.
