COSTER: Collision Snapshot Guided Time-Reversed Safety-Critical Scenario Generation

Anonymous Author(s)
Submitted to ICRA 2027
arXiv Supplementary Video Code
Overview of COSTER

The generation of safety-critical traffic scenarios is essential for the training and evaluation of autonomous vehicles. Earlier approaches generally depended on oversimplified adversarial objectives and narrow agent modifications, thereby limiting the plausibility and diversity of generated scenarios.

This work introduces COllision Snapshot guided TimE Reversed safety-critical scenario generation (COSTER), which identifies collision timesteps and locations based on learned traffic priors and constructs a collision snapshot by inserting a vehicle in contact with the existing vehicle in the real world traffic scenario. A time-reversed rollout based on a conditional variational autoencoder (CVAE) is then employed to reconstruct the trajectory of the inserted vehicle toward prior timesteps from the collision snapshot.

Our experimental results demonstrate that COSTER outperforms existing methods in plausibility, diversity, and data efficiency. Furthermore, agents trained on COSTER-generated scenarios reduce collision rates by 31% in real-world out-of-distribution scenarios, leading to improved ego task completion.

Highlights

The two videos illustrate how COSTER and several baseline methods modify ordinary driving scenarios to induce collisions.

For methods that perturb existing vehicles, such as SEAL and CAT, collision generation can become unrealistic when inappropriate surrounding vehicles are selected for perturbation. Furthermore, the use of simplified, scenario-agnostic adversarial objectives can lead to collisions that occur outside the drivable map, resulting in implausible behaviors.

Adv-BMT takes a step further by introducing adversarial vehicles into the scene. However, its sampling- and heuristic-based strategy often generates collisions that are not map-compliant, and the resulting trajectories are frequently unrealistic.

In contrast, COSTER incorporates scenario-specific context into collision generation through a learned traffic prior. By adapting to the unique characteristics of each scenario, COSTER produces plausible and multimodal collision trajectories. These realistic collision scenarios can be effectively leveraged for both the training and evaluation of autonomous driving systems.

Method

Time-reversed trajectory generation 1 Time-reversed trajectory generation 2

Collision time determination
This figure demonstrates the advantages of selecting collision locations based on velocity entropy. Compared with randomly selecting a collision timestep or using the final timestep, the velocity entropy-based approach tends to identify locations where collisions are more plausible and where a greater diversity of collision scenarios can occur. This suggests that the method preferentially selects regions that are both behaviorally realistic and rich in interaction complexity.

.

Collision snapshot determination
This figure illustrates the process of partitioning the map into local regions and determining the placement of a newly inserted adversarial vehicle with respect to the target vehicle. At a given collision timestep in the scenario, the local region with the highest velocity entropy among those overlapping the elliptical area centered on the target vehicle is selected. The adversarial vehicle in the collision snapshot is then initialized by sampling its position, velocity, and heading from the vehicle distribution within the selected local region.

Generated safety-critical scenario

Time-reversed trajectory generation
Combined with Dynamic Surrounding Attention (DSA), the CVAE-based time-reversed rollout enables the generation of plausible and diverse trajectory modalities from a collision snapshot.

Video Presentation