DisasterWorld

A Benchmark for Survivor Search
in Immediate Post-Disaster Response

Krrish Jain*1, Muyang Yan*1, Steven Brewer2, Wenshan Wang1, Seungchan Kim†1, Sebastian Scherer†1
*Equal Contribution     †Equal Advising
1Robotics Institute, Carnegie Mellon University   2Software Engineering Institute, Carnegie Mellon University
PaperComing soon arXivComing soon CodeComing soon DatasetComing soon Docs
Simulated hurricane sceneHurricane
Hurricane Ida aftermathReal
Simulated earthquake sceneEarthquake
Turkey-Syria earthquake damageReal
Simulated fire sceneFire
Building fire in downtown St. PaulReal
Simulated tornado sceneTornado
Tornado damage in Nebraska and IowaReal

Photographs: Hurricane Ida (ABC News), Turkey–Syria earthquake (Reuters), St. Paul building fire (MPR News), Nebraska and Iowa tornadoes (PBS NewsHour).

4Disaster typesHurricane · Tornado · Fire · Earthquake
2LocalitiesUrban · Suburban
3Damage levelsLow · Medium · High
120Environments24 scenes · 5 survivor configurations each

4 disasters × 2 localities × 3 damage levels24 scene geometries × 5 survivor configurations120 environments1 km × 1 km per scene

Abstract

After a large-scale disaster such as an earthquake or fire, responders must locate survivors across areas that are too dangerous to search directly or too large to search quickly. Robotic systems could provide this capability at scale, yet full-system development and evaluation in large-scale disaster environments are challenging because realistic physical trials are dangerous and difficult to reproduce.

We present DisasterWorld, a benchmark for multi-target survivor search by a team of robots during the initial response phase. It provides procedurally generated scenes crossing 2 localities (urban and suburban), 4 disaster types (hurricane, tornado, fire, earthquake), and 3 damage levels, each paired with 5 sampled survivor configurations, for 120 released environments in total. Additional survivor configurations can be generated on demand without rebuilding the underlying scene. Scenes include physics-settled debris with active collision geometry and evidence-informed survivor locations, together with exact ground truth for survivors and damaged structures. By enabling repeatable multi-target search missions under a fixed mission budget across this range of settings, DisasterWorld evaluates complete autonomy systems using time-sensitive measures of survivor discovery and path efficiency.

Environments

Task Formulation

Objective: Locate and report as many survivors as possible within time budget T.

  1. 1Observations

    Simulated suburban tornado scene with collapsed buildings, debris, and surrounding streetsExample scene view

    Onboard RGB-D
    Estimated state & pose

    Initially: starting pose
    Optional search boundary

  2. 2Team policy

    Next navigation objective
    for each robot

    Independent, distributed,
    or centralized coordination

    Observe → act → observe

  3. 3Detection reports

    JSON report
    Timestamp
    t
    World location
    (x, y, z)

    Visually detect a survivor
    and report its location

  4. 4Evaluation

    Match reports to survivors

    Credit the earliest valid
    report per survivor

Policy restrictionsNo prior maps, scene geometry, damage annotations, survivor counts/locations, or other simulator-only state. All coordination information comes from permitted inputs.

Ground-truth annotationsEvaluator only
Robots share a global coordinate frame. Each survivor is credited once; duplicate reports and false positives receive no discovery credit.

Scene Generation

Each 1 km × 1 km scene begins as a seeded 2-D layout of roads, parcels, buildings, vegetation, vehicles, and public spaces, generated from summary statistics of OpenStreetMap data rather than copied from a single real location. A disaster model decides where damage occurs, the damage is applied to buildings and other scene elements, and physics simulation settles displaced objects and debris before the geometry is fixed. Scenes are assembled from Unreal Engine/Fab and Objaverse-XL assets and run in NVIDIA Isaac Sim, with a ready-to-use aerial-robot configuration built on AirStack.

Dataset generator

Assets

Building asset House asset Tree and street-furniture assets Car asset

Damaged assets

Burning building Collapsed house Fallen tree and toppled street furniture Overturned car

Layout generation

Urban and suburban layouts
Disaster Intensity

Disaster model

Damage fields over urban and suburban layouts

Human placement

Survivor beside a fallen tree Survivor under roof panels Survivor in an overturned car Survivor at a burning window

Final scenes

Urban scene from above Suburban scene from above Urban scene at street level Suburban scene at low altitude

Simulator

Isaac Sim / AirStack

Scene running in Isaac Sim

Baseline trajectories

Robot search trajectories over a scene

Disaster Models

Each disaster model defines a spatial damage intensity over an otherwise intact layout. Local intensity is combined with object type, construction, and environmental conditions to assign structural damage states, informed by published damage scales and post-event studies.

  • Fire spreads from an ignition region as a wind-oriented ellipse whose elongation grows with wind, following wildfire-growth models, leaving burned structures, vegetation damage, smoke, and active flames.
  • Tornado damage is concentrated along a meandering track and falls off smoothly across it, with damage categories following the Enhanced Fujita Scale.
  • Hurricane damage is broad wind exposure with small spatial variation, plus inundation behind an irregular flood boundary.
  • Earthquake intensity decreases radially from an epicenter and is combined with construction type and site conditions to produce facade failure, soft-storey failure, collapse, and settlement.
Hurricane Earthquake Fire Tornado Suburban Hurricane damage, suburban Earthquake damage, suburban Fire damage, suburban Tornado damage, suburban Urban Hurricane damage, urban Earthquake damage, urban Fire damage, urban Tornado damage, urban
Damage
  1. None
  2. Light
  3. Moderate
  4. Heavy
  5. Severe
  6. Destroyed
Flood depth
  1. Dry
  2. <0.5 m
  3. 0.5–1 m
  4. 1–2 m
  5. >2 m
Overlays
  • Strong shaking
  • Weak shaking
  • Soft ground
  • Flooded house

Survivor Placement

Survivors are added after the damaged geometry is fixed, so new placements can be sampled without rebuilding the scene. Locations, poses, and occlusions are sampled near damaged structures, debris, evacuation routes, and shelters, informed by post-event studies and agency reports. Each task is exported as a USD scene with JSON annotations giving the exact 3-D location, extent, pose, semantic label, and attributes of every survivor and scene element.

Hurricane

Pinned by a fallen tree
Lying among broken glass
Lying among broken glass
Sitting among broken glass

Earthquake

Trapped under slabs
Pinned by fallen debris
Pinned by fallen debris
Pinned under roof panels
Lying on rubble pile
Pinned beneath rubble

Fire

At a window
On a balcony
Sheltering in a pool
In a vehicle

Tornado

Inside an overturned van
In a vehicle
Pinned by a streetlight
Pinned under a roof panel
Lying among debris
Pinned under a roof panel

Evaluation

Because finding survivors sooner matters, the primary metric is Time-Integrated Progress (TIP), the normalized area under the curve of survivors found over time:

TIPi = 1 − 1⁄Ni ∑j tij / T

where tij is the time survivor j is first found (T if never). We also report Progress, the fraction of survivors found by the end of the mission, and Progress-weighted Path Length (PPL), which also rewards finding them with shorter routes.

We evaluate 5 search policies spanning systematic coverage, geometric exploration, and semantic object-goal navigation on 24 tasks (one survivor arrangement per scene, 1,122 survivors in total), with teams of 8 aerial robots running AirStack under a 600-second mission budget.

MethodTIP (%) ↑Progress (%) ↑PPL ↑
Lawnmower coverage0.2730.9380.000393
Frontier exploration2.2863.7210.001782
VLFM1.9152.7020.001515
Co-NavGPT0.0770.1050.000045
RAVEN3.5045.3600.001443

Use DisasterWorld

Code Coming soonProcedural scene generation and survivor configuration tools. Dataset Coming soonUSD scenes and JSON annotations for 120 search environments. AirStack DocumentationInstallation, ROS 2 interfaces, and aerial-robot simulation in Isaac Sim. DocsSetup, mission configuration, survivor generation, and evaluation.

BibTeX

@article{jain2026disasterworld,
  title   = {DisasterWorld: A Benchmark for Survivor Search in
             Immediate Post-Disaster Response},
  author  = {Jain, Krrish and Yan, Muyang and Brewer, Steven and
             Wang, Wenshan and Kim, Seungchan and Scherer, Sebastian},
  journal = {arXiv preprint arXiv:XXXX.XXXXX},
  year    = {2026},
}