Unmanned Systems Lab | Research

Online Multi-Camera and IMU Calibration

2022-09-15T00:00:00+00:00

This research focuses on the implementation of an online multi-camera IMU calibration filter that is based on the work of Mirzaei and Roumeliotis. This work expands on what has been previously done by incorporating the fiducial marker detectors provided by OpenCV and manipulating the update equations to utilize the quaternion measurement provided by these detectors.

The main challenge was in developing the Jacobians for the quaternion measurement, such that the updates would be stable even with large body rotations. To overcome this challenge, the derivations and work by Joan Sola which can be found on ArXiv were used extensively.

Additionally, work was performed on monitoring the calibration for shifts in extrinsic parameters, such as would occur if a sensor is bumped in the middle of use. Through the use of a sliding window T-test, online monitoring of calibration extrinsic parameters was implemented.

Experimentation

Testing was performed using a bar of 8020 with multiple camera mounts and a IMU/INS. This allowed for multiple sensor configurations and rapid recalibration testing with different sensor configurations. The specific sensors tested were:

IMUs:
- Vectornav VN300
- Vectornav VN100
- Wheeltec FDISystems N100N
- Pixhawk PX4
- WitMotion WT901C485
Cameras:
- Flir Blackfly BFLY-PGE-20E4C-CS
- Basler Ace acA1920-40uc

Code

The code has been open-sourced and is available on Github. Feel free to fork, update and create pull requests where you see fit, so long as you adhere to the GPL3 license.

Preprint

A preprint of the published paper can be found on ArXiv.

Citation:

@INPROCEEDINGS{9564981,
  author={Hartzer, Jacob and Saripalli, Srikanth},
  booktitle={2022 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR)}, 
  title={Online Multi Camera-IMU Calibration}, 
  year={2022}

Roadside LiDAR Dataset

2021-07-03T00:00:00+00:00

A roadside LiDAR dataset both in urban and in highway environments with annotated vehicles. This dataset corresponds to the dataset described in “Building a Smart Work Zone Using Roadside LiDAR”. Our dataset contains two ~10 minute segments on a urban (45 mph) and highway segment (75 mph), consisting of >1000 frames labeled of measurements taken with a Velodyne VLP-16. The dataset format is as follows:

Raw (unprocessed ROI filtering) rosbags
Bird’s Eye View projection images of LiDAR data
Labels in .csv format for each image

Full Dataset Link

The table below summarizes the dataset:

Citation:

@misc{darwesh2021SWZ,
      title={Building a Smart Work Zone Using Roadside LiDAR
      author={Darwesh, Amir and Wu, Dayoung and Le, Minh and Saripalli, Srikanth},
      year={2021},
      journal={IEEE Transactions on Intelligent Transportation Systems},
      publisher={IEEE}
}

SemanticUSL: A Dataset for LiDAR Semantic Segmentation Domain Adaptation

2020-11-25T00:00:00+00:00

SemanticUSL was collected on a Clearpath Warthog robotics with an Ouster OS1-64 Lidar. The data collection location includes the campus site and off-road research facility of Texas A& M University. The data include the traffic-road scene, walk-road scene, and off-road scene. Our dataset has 16578 unlabeled scans for domain adaptation training and 1200 labeled scans for evaluation. The data uses the same format and ontology as SemanticKITTI; therefore, it can be easily used for domain adaptation research between SemanticKITTI and SemanticPOSS.

Download

Example Data link

Full Data link

LiDARNet: A Boundary-Aware Domain Adaptation Model for Lidar Point Cloud Semantic Segmentation

RELLIS-3D: A Multi-modal Dataset for Off-Road Robotics

Citation

@misc{jiang2020lidarnet,
      title={LiDARNet: A Boundary-Aware Domain Adaptation Model for Lidar Point Cloud Semantic
      author={Peng Jiang and Srikanth Saripalli},
      year={2020},
      eprint={2003.01174},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

RELLIS-3D: A Multi-modal Dataset for Off-Road Robotics

2020-11-25T00:00:00+00:00

Peng Jiang¹, Philip Osteen², Maggie Wigness² and Srikanth Saripalli¹
1. Texas A&M University; 2. CCDC Army Research Laboratory
[Website] [Paper] [Github]

Overview

Semantic scene understanding is crucial for robust and safe autonomous navigation, particularly so in off-road environments. Recent deep learning advances for 3D semantic segmentation rely heavily on large sets of training data; however, existing autonomy datasets represent urban environments or lack multimodal off-road data. We fill this gap with RELLIS-3D, a multimodal dataset collected in an off-road environment containing annotations for 13,556 LiDAR scans and 6,235 images. The data was collected on the Rellis Campus of Texas A\&M University and presents challenges to existing algorithms related to class imbalance and environmental topography. Additionally, we evaluate the current state of the art deep learning semantic segmentation models on this dataset. Experimental results show that RELLIS-3D presents challenges for algorithms designed for segmentation in urban environments. Except for the annotated data, the dataset also provides full-stack sensor data in ROS bag format, including RGB camera images, LiDAR point clouds, a pair of stereo images, high-precision GPS measurement, and IMU data. This novel dataset provides the resources needed by researchers to develop more advanced algorithms and investigate new research directions to enhance autonomous navigation in off-road environments.

Recording Platform

Clearpath Robobtics Warthog

Sensor Setup

64 channels Lidar: Ouster OS1
32 Channels Lidar: Velodyne Ultra Puck
3D Stereo Camera: Nerian Karmin2 + Nerian SceneScan
RGB Camera: Basler acA1920-50gc + Edmund Optics 16mm/F1.8 86-571
Inertial Navigation System (GPS/IMU): Vectornav VN-300 Dual Antenna GNSS/INS

Annotated Data:

Ontology:

With the goal of providing multi-modal data to enhance autonomous off-road navigation, we defined an ontology of object and terrain classes, which largely derives from the RUGD dataset but also includes unique terrain and object classes not present in RUGD. Specifically, sequences from this dataset includes classes such as mud, man-made barriers, and rubble piles. Additionally, this dataset provides a finer-grained class structure for water sources, i.e., puddle and deep water, as these two classes present different traversability scenarios for most robotic platforms. Overall, 20 classes (including void class) are present in the data.

Images Statics:

LiDAR Scans Statics:

Benchmarks

Image Semantic Segmenation

LiDAR Semantic Segmenation

ROS Bag Raw Data

Data included in raw ROS bagfiles:

Topic Name	Message Tpye	Message Descriptison
/img_node/intensity_image	sensor_msgs/Image	Intensity image generated by ouster Lidar
/img_node/noise_image	sensor_msgs/Image	Noise image generated by ouster Lidar
/img_node/range_image	sensor_msgs/Image	Range image generated by ouster Lidar
/imu/data	sensor_msgs/Imu	Filtered imu data from embeded imu of Warthog
/imu/data_raw	sensor_msgs/Imu	Raw imu data from embeded imu of Warthog
/imu/mag	sensor_msgs/MagneticField	Raw magnetic field data from embeded imu of Warthog
/nerian_stereo/left_image	sensor_msgs/Image	Left image from Nerian Karmin2
/nerian_stereo/right_image	sensor_msgs/Image	Right image from Nerian Karmin2
/odometry/filtered	nav_msgs/Odometry	A filtered local-ization estimate based on wheel odometry (en-coders) and integrated IMU from Warthog
/os1_cloud_node/imu	sensor_msgs/Imu	Raw imu data from embeded imu of Ouster Lidar
/os1_cloud_node/points	sensor_msgs/PointCloud2	Point cloud data from Ouster Lidar
/os1_node/imu_packets	ouster_ros/PacketMsg	Raw imu data from Ouster Lidar
/os1_node/lidar_packets	ouster_ros/PacketMsg	Raw lidar data from Ouster Lidar
/vectornav/GPS	sensor_msgs/NavSatFix	INS data from VectorNav-VN300
/vectornav/IMU	sensor_msgs/Imu	Imu data from VectorNav-VN300
/vectornav/Mag	sensor_msgs/MagneticField	Raw magnetic field data from VectorNav-VN300
/vectornav/Odom	nav_msgs/Odometry	Odometry from VectorNav-VN300
/vectornav/Pres	sensor_msgs/FluidPressure
/vectornav/Temp	sensor_msgs/Temperature
/velodyne_points	sensor_msgs/PointCloud2	PointCloud produced by the Velodyne Lidar

Data Download:

Access Link

Citation

@misc{jiang2020rellis3d,
      title={RELLIS-3D Dataset: Data, Benchmarks and Analysis},
      author={Peng Jiang and Philip Osteen and Maggie Wigness and Srikanth Saripalli},
      year={2020},
      eprint={2011.12954},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

Collaborator

License

All datasets and code on this page are copyright by us and published under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 License.

SemanticUSL: A Dataset for Semantic Segmentation Domain Adatpation

LiDARNet: A Boundary-Aware Domain Adaptation Model for Lidar Point Cloud Semantic Segmentation

LiDARNet: A Boundary-Aware Domain Adaptation Model for Point Cloud Semantic Segmentation

2020-11-25T00:00:00+00:00

Introduction

We present a boundary-aware domain adaptation model for LiDAR scan full-scene semantic segmentation (LiDARNet). Our model can extract both the domain private features and the domain shared features with a two branch structure. We embedded Gated-SCNN into the segmenter component of LiDARNet to learn boundary information while learning to predict full-scene semantic segmentation labels. Moreover, we further reduce the domain gap by inducing the model to learn a mapping between two domains using the domain shared and private features. Besides, we introduce a new dataset (SemanticUSL). The dataset has the same data format and ontology as SemanticKITTI. We conducted experiments on real-world datasets SemanticKITTI, SemanticPOSS, and SemanticUSL, which have differences in channel distributions, reflectivity distributions, diversity of scenes, and sensors setup. Using our approach, we can get a single projection-based LiDAR full-scene semantic segmentation model working on both domains. Our model can keep almost the same performance on the source domain after adaptation and get an 8%-22% mIoU performance increase in the target domain.

Updates:

The paper is released on arXiv

The code is released on Github

Approach

Generally, we expect a model that can complete a task for data from similar domains. However, feature difference between two similar domains causes a model, which learns from one domain (called source domain (S)), can not perform well on another domain (called target domain (T)). Therefore, we expect a method that can adapt a model from one domain to another domain. If the target domain does not provide ground truth, the problem is called unsupervised domain adaptation. In this paper, the task is full-scene semantic segmentation for Lidar scan. In this problem, we use (X_S) denotes source data, (Y_S) denotes source labels, and (X_T) denotes target data, but target labels are not accessible.

Based on the intuition that two similar domains should contain shared information across the two domains and private information to each domain. And the adaptable information should be contained in shared information of two domains. Therefore, we designed an end-to end trainable model that splits input data into domain shared and private features. The model then utilizes the extracted shared features to perform semantic segmentation. Fig.1 shows the components of the model and the information flow in the model. The model contains two extractors: a shared feature extractor (f_P) and a private feature extractor (f_D).

To induce the two extractors to produce such split information, we add a loss function that encourages the independence of these parts, and connect the private feature extractor to a classifier (f_C) to differentiate the data from two domains. Besides, we feed the output of the domain shared extractor to a segmenter to complete the same task (predict semantic labels (\hat{Y})). To ensure that the private features are still useful and to further reduce the domain gap, we introduce the CycleGAN mechanism \cite{Zhu2017} to induce the models to learn two mappings between two domains. The domain private and shared features are fed into domain converters to convert the data from one domain to another domain: ((f_{S\to T}) converts source data into target domain, (f_{T\to S}) converts target data into source domain). The conversion is learning through an adversarial learning procedure. Therefore, the converted data are separately fed into domain discriminators ((D_{T}) and (D_{S})). Meanwhile, we add Gated-SCNN \cite{Takikawa2019} on the side of the segmenter to extract boundary maps (B) while learning to predict semantic segmentation. We utilize the output boundaries to penalize the label predictions from the target domain. To further penalize the label output, we add a boundaries discriminator (D_{B}), and a labels discriminator (D_{Y}) to penalize the output label and boundary.

Results

Experiment I: From SemanticKITTI to SemanticPOSS and SemanticUSL

Experiment II: From SemanticPOSS to SemanticKITTI and SemanticUSL

Experiment III: From SemanticUSL to SemanticPOSS and SemanticKTTI

Citation

@misc{jiang2020lidarnet,
      title={LiDARNet: A Boundary-Aware Domain Adaptation Model for Point Cloud Semantic Segmentation
      author={Peng Jiang and Srikanth Saripalli},
      year={2020},
      eprint={2003.01174},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

SemanticUSL: A Dataset for Semantic Segmentation Domain Adaptation

RELLIS-3D: A Multi-modal Dataset for Off-Road Robotics

Ultra-Wideband Localization

2020-10-05T00:00:00+00:00

Introduction

There are numerous industries that are in the process of developing autonomous passenger vehicles. Such vehicles have the capacity to reduce risk and free time of riders. In the development of an autonomous vehicles, a key consideration is reliable localization in order to plan for and react to the environment. Typically,h igh position accuracy is achieved through the use of Global Navigation Satellite Systems (GNSS). The use of GNSS with an Inertial Measurement Unit (IMU) or odometry measurements can lead to an accurate positioning system.

Reliable GNSS measurements are often not available. Because GNSS receivers must have line of sight with at least four satellites, environments that include obstructions pose significant challenges. Urban canyons, tunnels, or indoor environments degrade or fully obstruct GNSS signals, forcing the system to rely on dead reckoning. These measurements are subject to drift, and therefore cannot be trusted after continued interruption of GNSS. As such, it would be advantageous to the continued development of autonomous vehicles to use a exteroceptive sensor for navigation through environments where GNSS is typically denied.

Ultra-Wideband (UWB) radio, utilizes low-energy pulse communication typically for short-range, high-bandwidth applications. By measuring time of flight across various frequencies, it is possible to measure distance between modules while overcoming multipath errors. This has allowed UWB modules to be applied towards localization and tracking problems. With the continued development of UWB technology and roll out in commercial products, it’s ability to aid in navigation should continue to be explored.

By using UWB ranging, autonomous vehicles can generate ranging estimates to other independent vehicles. Just like measurements to landmarks, these measurements are not subject to drift when GNSS measurements are obscured, and can be made available in both indoor and outdoor environments. Known as collaborative localization, these measurements can leverage the growing number of intelligent vehicles on the road, and can increase the accessible area of autonomous vehicles.

Background

Ultra-Wideband, with it’s previously mentioned benefits, is a very promising technology for localization. There are many examples of using UWB ranging modules for high-accuracy indoor localization of ground or aerial systems. These systems typically measure with respect to fixed landmark anchors.

There are also examples of using UWB modules for outdoor vehicular environments. Such a framework has the capacity to increase localization accuracy of existing systems, or provide accurate localization in environments where GNSS is not reliable, such as tunnels or urban canyons.

Given a network of vehicles, it is possible to leverage relative measurements to provide better localization accuracy as a group, than as a collection of individuals, generally referred to as collaborative localization (CL). These methods are generally classified in two groups: centralized methods, and decentralized methods.

Centralized CL

Centralized CL methods use a single or multiple fusion centers, to which every vehicle communicates measurement information. These state estimators are capable of producing optimal linearized state estimates and have been used in a number of applications with success. A major issue with centralized networks is sensitivity to failure. As the number of measurements made can increase on the order of O(n^2), centralized networks can meet constraints when large networks of nodes are implemented with complex measurements or update functions.

Decentralized CL

Decentralized CL (DCL) methods are defined by distributing the state estimation computation across every agent in the network. Unlike centralized methods, DCL methods are not susceptible to single point failures. These generic, recursive approximations of the centralized method can extend the framework to decrease convergence time or computational cost without loss of accuracy.

Contributions

This research seeks to expand upon previous work through the following contributions

Expanded previous UWB EKF and DCL simulation environments through the incorporation of vehicle and sensor models and the framework for Monte Carlo simulations.
Validated the zero-mean normally distributed error assumption for a UWB ranging module.
Showed EKF localization improvements using real-time UWB measurements to landmarks in GNSS denied environments.
Showed DCL accuracy improvements by utilizing real-time UWB relative ranging measurements.

Decentralized Collaborative Localization

The decentralized collaborative localization algorithm is a form of a Kalman filter that approximates the centralized filter through distributed computation. The estimation of the entire state of a network is broken down into smaller filters where each vehicle has private controls and measurements that are used internally and relative measurements that require the sharing of state and covariance information.

For a network of N vehicles, each vehicle initializes its own 6-DOF state and assumes zero cross-correlations between the vehicles until a relative measurement is made. As such, the number of vehicles does not need to be known.

Initialization

It is assumed that at the beginning of any network, the vehicles positions are uncorrelated. Therefore, the network state can be initialized with the initial beliefs of each vehicle and the cross correlation is set as a zero matrix.

When the vehicles come into sensing range, generally the cross correlation is no longer equal to zero and therefore the cross-correlation term can be decomposed to allow each vehicle to maintain estimates of cross correlation terms that can be combined at the next relative measurement. The decomposition sets the cross-correlation of the sensing vehicle equal to the true cross-correlation and the sensed vehicle estimate equal to the identity matrix.

Control

It is assumed that the vehicles follow the motion model g(U) where control input U is an IMU measurement. The prediction step for a vehicle is given by the standard EKF equations.

Private Update

It is assumed that the private update measurements are functions of the state of a single vehicle with a Gaussian error disturbance h(x). These updates come from GNSS positioning and UWB landmark ranging measurements.

Relative Update

It is assumed that the relative update measurement to be a function of the state of two vehicles with a Gaussian error disturbance. These updates come from vehicle-to-vehicle relative UWB ranging measurements. The cross-correlation estimates are combined to be used in the best state estimate of the approximated system.

Simulation

In order to simulate various collaborative localization algorithms, a simulation framework that would be flexible in number of cars and networking was created in MatLab: collab_localization repository. The simulation was designed to handle an indeterminate number of cars and configurations. Additionally, to facilitate the testing of collaborative localization, UWB tags can be treated as either fixed landmarks or mobile units on other vehicles.

Sensing Models

The UWB sensing model was implemented with Gaussian normal randomly distributed errors with a measured standard deviation of 0.31 meters.

The GNSS sensing model takes a Circular Error Probable (CEP) error value and converts into a distance root mean square (DRMS), which is approximately 84.93\% of CEP. This DRMS value is input to a circularly symmetric Rayleigh distribution to perturb the measurement.

Output

The resulting output of this software package is a visual animation of the simulation, position error summaries for each vehicle, and the final state and covariance. When compiled in Monte Carlo simulations, these data can be used to evaluate filter performance improvements.

Results

Using the simulation framework, it was possible to test a wide variety of vehicle environments and sensor configurations. These configurations included vehicles moving parallel to each other, vehicles moving perpendicular in street crossings, and moving in groups in tunnels without GNSS measurements. Additionally, UWB landmarks were added to each of these situations.

Examples of the Monte Carlo simulations performed are shown. These figures show the crossing of two vehicles at an intersection using various estimation algorithms.

The simulations showed that UWB measurements offered improvements to localization accuracy across all experiments. Using more vehicles improved accuracy, and using additional landmarks offered significant improvements. This was especially the case in situations where GNSS data was not available such as tunnels and urban canyons.

Experimentation

The filter framework was tested in Bryan, Texas using the Unmanned Systems Lab autonomous trolley. The Decwave EVK1000 evaluation boards were mounted alongside a VectorNav VN-300 and ArduSimple RTK GNSS. The trolley itself includes PACMod, a by-wire kit prepared by Autonomous Stuff, which gives access to wheel odometry and steering data.

Using tripods, a second set of UWB ranging modules were placed in various experimental setups to either represent road landmarks, or a second vehicle.

As in the simulations, the addition of the two UWB sensors on the vehicle offered improvements to the localization accuracy of the vehicle in all scenarios. The improvement was even more pronounced when GNSS data to the VN-300 was denied. The quality of data coming from the VN-300 was underestimated, leading to much higher location accuracy than simulated. As such, the percent improvement from simply adding UWB measurements was lower than in the simulations, with the smallest percent improvement being 2.9% and the largest being 9.3%. However, removing the GNSS updates showed results much more in line with the simulation, with the addition of UWB measurements showing a 83.3% improvement in localization accuracy in a tunnel environment.

Future Work

Future work for this reserach includes

Experiemnts with additional tags and multiple moving vehicles
Explore transitional space between GPS and GPS-denied environments
Compare the decentralized approximation to the centralized method
Simulate the effects of delays in the system

In the future, testing will include more in-person experiments with multiple moving vehicles and a greater number of UWB tags. This would allow for better evaluation of truly cooperative localization using of vehicles using UWB and will validate the algorithm used in simulation. This will also explore the use of real-time varying covariance estimations in the update equations.

Preprint

A preprint of the paper can be found here.

@INPROCEEDINGS{9564981,
  author={Hartzer, Jacob and Saripalli, Srikanth},
  booktitle={2021 IEEE International Intelligent Transportation Systems Conference (ITSC)},
  title={Vehicular Teamwork: Collaborative localization of Autonomous Vehicles},
  year={2021},
  pages={1077-1082},
  doi={10.1109/ITSC48978.2021.9564981}}

L. Yao, Y. A. Wu, L. Yao, and Z. Z. Liao, “An integrated IMU and UWB sensor based indoor positioning system,” in 2017 International Conference on Indoor Positioning and Indoor Navigation (IPIN), pp. 1–8, 2017.
L. Luft, T. Schubert, S. I. Roumeliotis, and W. Burgard, “Recursive decentralized localization for multi-robot systems with asynchronous pairwise communication,” The International Journal of Robotics Research, vol. 37, no. 10, pp. 1152–1167, 2018.
S. I. Roumeliotis and G. A. Bekey, “Distributed multirobot localization,”IEEE Transactionson Robotics and Automation, vol. 18, no. 5, pp. 781–795, 2002
S. Tanwar and G. Gao, “Decentralized collaborative localization with deep gps coupling for UAVs,” in 2018 IEEE/ION Position, Location and Navigation Symposium, PLANS 2018 - Proceedings, (United States), pp. 767–774, June 2018.

Improving Occlusion Occupancy Prediction

2020-07-24T00:00:00+00:00

Safe planning for autonomous vehicles requires accurate prediction of future vehicle motion. For observable vehicles, future motion is limited by physical constraints as well as some logical (rules of the road) constraints. For sensor occlusion, the assumption has been that a vehicle may exit at any time, at a speed up to and slightly exceeding the speed limit. This approach does not consider the passage of time, or any spatial perception. Our research focuses on how this prediction can be improved to more closely reflect the true possibility of a vehicle hidden inside the occlusion.

Related Papers

Improving Bounds on Occluded Vehicle States for use in Safe Motion Planning, G. Neel and S. Saripalli, Submitted to 2020 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), pdf soon.

Extrinsic Calibration of Multiple Sensors

2020-07-09T00:00:00+00:00

Cross calibration of multiple sensors is a classic estimation problem in robotics and with the advent of multi-sensor perception and state estimation techniques, this problem has gained paramount importance. We at USL are working on both target-based and targetless approaches of multi sensor calibration. Having already studied and implemented a few targetbased approaches we are currently exploring targetless aproaches which can be used when the robot is operating in real-time. The long term goal is to integrate a targetless motion based approach into a multi-sensor SLAM pipeline which allows tracking the calibration parameters online and detects changes which might occur during robot operation, thus ensuring robustness, miscalibration detection and recalibration.

Here are our recent research papers on target-based calibration of cameras and lidars.

Related Papers

Experimental Evaluation of 3D-LIDAR Camera Extrinsic Calibration, S. Mishra, P. Osteen, G. Pandey and S. Saripalli, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS, 2020), pdf
Extrinsic Calibration of a 3D-LIDAR and a Camera, S. Mishra, G. Pandey and S. Saripalli, IEEE Intelligent Vehicles Symposium (IV, 2020) pdf

Teleoperation for Autonomous Shuttles

2020-04-07T00:00:00+00:00

Our research is investigating shared driving operations between autonomy, and a remote operator. In challenging urban scenarios, such as traffic jams and accidents, the autonomy may require human intervention. Currently, teleoperation for our shuttles is manual control via four separate 4G LTE connections. Current research goals focus on reducing teleoperator stress by exploring different methods of remote vehicle guidance and control for ground vehicles in city environments.

Autonomous Cone Placement

2020-03-07T00:00:00+00:00

The Auto Cone project is an effort to develop cones that are capable of localizing and placing themselves to improve safety conditions for highway workers. These cones utilize RTK GPS and onboard localization filtering to produce decimeter-level accuracy in placing themselves in road conditions. Additionally, they are capable of transitioning through GPS-denied environments such as under bridges or overpasses. Pictured are two of the cones and the real-time kinematic (RTK) base station.

Problem:

The goal of this project is to develop a robotic platform capable of automatically placing cones in a defined wedge shape behind the work vehicle within the starting lane. Specifically the cones shall:

Place three cones in 40 foot increments in a wedge
Begin the wedge 80 feet from the end of the vehicle
Operate on highway surfaces unaffected by small debris
Remain within the lane despite road curvature
Not rely on magnetic or road-embedded sensors
Have a speed greater than 0.3 m/s
Cost less than $1,500 per cone unit
Be easy to use and require little training

Kinematics:

The omnidirectional platform makes the system holonomic, which means that with only three motors, the system can smoothly and directly move between any two states. This allows orientation to be independently controlled from position, and makes the system unconstrained by initial conditions. This is very advantageous for pick and place when deploying. The image outlines the kinematics used to drive the cone’s motion.

Sensor Fusion:

Using a system of RTK GPS and a ground base station, the cone’s achieve a much lower position error than typical GPS for a relatively small increase in cost.

Fusing these corrected GPS measurements with the higher rate encoders on the wheel motors provides the system with a high rate localization estimate that is robust in handling drift and sensor noise.

Results:

The results of this project were a functioning prototype system that is capable of deploying multiple cones without collision to a wedge shape formation while remaining within lane lines. The system is also capable of operating in GPS-denied environments.

Videos

Deployment:

IV2020 Presentation:

Preprint

A preprint of the paper can be found here.

Citation:

@INPROCEEDINGS{AutoCone,
  author={Hartzer, Jacob and Saripalli, Srikanth},
  booktitle={2020 IEEE Intelligent Vehicles Symposium (IV)},
  title={AutoCone: An OmniDirectional Robot for Lane-Level Cone Placement},
  year={2020},
  pages={1663-1668},
  doi={10.1109/IV47402.2020.9304683}
}

Unmanned Systems Lab | Research

Online Multi-Camera and IMU Calibration

Experimentation

Code

Preprint

Citation:

Roadside LiDAR Dataset

Citation:

SemanticUSL: A Dataset for LiDAR Semantic Segmentation Domain Adaptation

Download

Related Work

Citation

RELLIS-3D: A Multi-modal Dataset for Off-Road Robotics

Overview

Recording Platform

Sensor Setup

Annotated Data:

Ontology:

Images Statics:

LiDAR Scans Statics:

Benchmarks

Image Semantic Segmenation

LiDAR Semantic Segmenation

ROS Bag Raw Data

Data Download:

Citation

Collaborator

License

Related Work

LiDARNet: A Boundary-Aware Domain Adaptation Model for Point Cloud Semantic Segmentation

Introduction

Approach

Results

Citation

Related Work

Ultra-Wideband Localization

Introduction

Background

Centralized CL

Decentralized CL

Contributions

Decentralized Collaborative Localization

Initialization

Control

Private Update

Relative Update

Simulation

Sensing Models

Output

Results

Experimentation

Future Work

Preprint

Related Works

Improving Occlusion Occupancy Prediction

Extrinsic Calibration of Multiple Sensors

Teleoperation for Autonomous Shuttles

Autonomous Cone Placement

Problem:

Kinematics:

Sensor Fusion:

Results:

Videos

Preprint

Citation: