SenseKit Addition and Configuration

Guide to add a new SenseKit and configure it.

Adding a New SenseKit

Following are the steps required to add a new SenseKit:

1. Open the Configuration tab in the wrist panel.

2. Press on the SenseKits button, which will open the Active SenseKits panel.

3. Press on Add New button, which will open the panel with input fields.

4. Input the IP and port (default port is 9091) of the SenseKit.

   
5. Press Connect this will try to connect to the SenseKit device and find any configurations saved on the SenseKit device.

6. If any configurations is stored on the SenseKit device it will be shown in the panel, which could be selected to load the configuration available.

7. If no configurations are available click on Create New Config button on the panel and define new configuration

SenseKit Configuration Elements

• Name : This field is name of the SenseKit configuration

• Mode : This field is used to the view option of the RGBD streaming

  • Pointcloud Only (default) : show 3D point cloud

  • Video Only : show 2D video

  • Pointcloud + Video : show 3D point cloud and 2D video

• Depth Codec :

  • zdepth (default) : lossless depth compression with sensor based accuracy

  • yuvdepth : lossy low bandwidth depth compression, not supported on Jetson

• Color Codec :

  • MJPEG : color compression with high bandwidth requirement

  • H.264 (default) : recommended color compression codec to use

• Fusion Offset (m unit) : when fusing multiple sensors treat sensor as if data was reported with larger distance to avoid noisy data fighting for view, only small values recommended (0.01 - 0.1) as it may have side effects with VR scene

• Cutoff Distance (m unit) : define to remove from 3D view data with depth larger than value, 0 to disable

• Cutoff Replacement (m unit) : define to replace data at depth larger then cutoff distance and missing data with the value, typically set to 0 (remove) or matches Cutoff Distance to perform 2D-3D Fusion

1. If the SenseKit Firmware Specific Setting are not configured during OTA Firmware Update Process or requires update, click on the Sensekit Firmware Specific Settings button. Sensekit Firmware Specific Settings Panel if selected contains the following fields:

SenseKit Firmware Specific Settings

• ROS Port (default 9091) : the port used for connecting the AMAS VR app to the SenseKit

• Data Port (default 12345) : the port used for connecting the AMAS VR app to the RGBD streaming pipeline of the Sensekit

• Color Compression Config : hardware acceleration to use. In typical scenarios default is set per sensor when installing firmware and should not be changed. When installing firmware on non Extend Robotoics hardware choose method supported by hardware.

  • jetson_nvmpi.yaml : NVidia Jetson NVMPI (Jetson only)

  • nvenc.yaml : NVidia NVENC (x86-64)

  • nvenc_18_04.yaml : NVidia NVENC for Ubuntu 18.04 Bionic firmware (x86-64)

  • nvenc_motioncam.yaml : NVidia NVENC optimized for Photoneo MotionCam-3D (x86-64)

  • nvenc_zed_2i.yaml : NVidia NVENC optimized for StereoLabs ZED 2i (x86-64)

  • vaapi.yaml : Intel VAAPI (x86-64)

  • vaapi_amd.yaml : AMD VAAPI (x86-64)

  • vaapi_low_power.yaml : alternative Intel VAAPI low power encoding path (x86-64)

  • vaapi_motioncam.yaml : Intel VAAPI optimized for Photoneo MotionCam-3D (x86-64), may fail with non-standard MotionCam resolutions with firmware side iHD VAAPI driver and will work with i965 VAAPI driver (if hardware supports it)

• FPS : desired framerate of the sensor, currently supported only for Kinect4A, FemtoBolt and MotionCam-3D. Note that typically sensors support only limited set of framerates and may fail, fallback to default or not reach non-standard framerate. Decreasing framerate may help if bandwidth is limited.

• Camera ID (ZED-X/ZED-XM sensors only) : ZED GMSL hardware id of the sensor

1. Press Save Local or Save Local and Device to finish the configuration and connect to the SenseKit.

2. The Active SenseKits panel will now show this SenseKit.

Adding a New SenseKit

Modifying a Currently Active SenseKit

Follow the steps given below to modify a currently active SenseKit:

1. Open the Configuration tab in the wrist panel.

2. Press on the SenseKits button.

3. Press on the SenseKit name you would like to modify.

4. Edit the configuration of the SenseKit accordingly.

5. Press Save. This will update the SenseKit, and will update the stored configuration