Skip to content

Running carla headless evaluation on determined AI cluster and failed due to segmentation fault(core dumped) #199

@WeiZhen-grr

Description

@WeiZhen-grr

I run the bash file run_evaluation_debug.sh and get outputs as the following
_(simlingo) zhenwei@4dc93ea82e7a:/workspace$ bash b2d_scripts/run_evaluation_debug.sh
/workspace/software/carla0915/CarlaUE4.sh -RenderOffScreen -nosound -carla-rpc-port=30000 -graphicsadapter=0 None
4.26.2-0+++UE4+Release-4.26 522 0
Disabling core dumps.
sh: 1: xdg-user-dir: not found
WARNING: lavapipe is not a conformant vulkan implementation, testing use only.
load_world success , attempts=0
traffic_manager init success, try_time=0

========= Preparing RouteScenario_2286 (repetition 0) =========

Loading the world
Setting up the agent
CUDA initialized. GPU: NVIDIA GeForce RTX 4090
path to conf file: /workspace/checkpoints/epoch=013.ckpt/pytorch_model.pt+RouteScenario_2286_rep0_Town12_HighwayCutIn_1_18_09_23_10_44_35
Config path: /workspace/checkpoints/epoch=013.ckpt/pytorch_model.pt
Save path root: RouteScenario_2286_rep0_Town12_HighwayCutIn_1_18_09_23_10_44_35
Loading model...
Using /workspace/pretrained/InternVL2-1B as the image encoder.
Using PEFT model
trainable params: 17,596,416 || all params: 647,260,288 || trainable%: 2.7186
Loading model weights...
Model loaded successfully
===== Debug mode is ON. Saving images to: ./eval_v1/RouteScenario_2286_rep0_Town12_HighwayCutIn_1_18_09_23_10_44_35/debug_viz/workspace/iter_013.ckpt/Bench2Drive/bench2drive220_2025_09_23_10_46_10/images =====
Running the route
Segmentation fault (core dumped)_

I suppose it might not be mainly due to not enough memory because my command "gpustat" comes
"(simlingo) zhenwei@4dc93ea82e7a:~$ gpustat
4dc93ea82e7a Tue Sep 23 10:46:39 2025 535.183.01
[0] NVIDIA GeForce RTX 4090 | 38°C, 0 % | 4896 / 24564 MB |
"
and all the memory used was for loading my model.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions