[AISPEECH_ASR] Finetuning script does not load dev_scp_file_path (validation data)

[helloseraphina]

System Info

1 NVIDIA Ada Lovelace L40S GPU

Information

• The official example scripts
• My own modified scripts

🐛 Describe the bug

Hi! Thank you for creating the AISPEECH_ASR recipe.

I compiled separate multitask.jsonl files for my training and validation data and have tried to run scripts/finetune_torchrun.sh. I modified the paths (train_scp_file_path and dev_scp_file_path) to the directories containing the multitask.jsonl files in the .sh file:

train_scp_file_path=/multitask/train/
dev_scp_file_path=/multitask/dev/

...
++dataset_config.train_scp_file_path=$train_scp_file_path \
++dataset_config.dev_scp_file_path=$dev_scp_file_path \

The training progresses but will fail when reaching the validation stage, giving a "FileNotFoundError" error despite the validation multitask.jsonl file existing.

However, it runs once I replace "dev_scp_file_path" to "test_scp_file_path" in the .sh file

Error logs

File "/SLAM-LLM/src/slam_llm/utils/train_utils.py", line 186, in train
[rank0]: eval_ppl, eval_epoch_loss, *rest = evaluation(model, train_config, eval_dataloader, local_rank, tokenizer)
[rank0]: File "/SLAM-LLM/src/slam_llm/utils/train_utils.py", line 424, in evaluation
[rank0]: for step, batch in enumerate(eval_dataloader):
[rank0]: File "/home/.local/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 701, in next
[rank0]: data = self._next_data()
[rank0]: File "/home/.local/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 1465, in _next_data
[rank0]: return self._process_data(data)
[rank0]: File "/home/.local/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 1491, in _process_data
[rank0]: data.reraise()
[rank0]: File "/home/.local/lib/python3.10/site-packages/torch/_utils.py", line 715, in reraise
[rank0]: raise exception
[rank0]: FileNotFoundError: Caught FileNotFoundError in DataLoader worker process 0.
[rank0]: Original Traceback (most recent call last):
[rank0]: File "/home/.local/lib/python3.10/site-packages/torch/utils/data/_utils/worker.py", line 351, in _worker_loop
[rank0]: data = fetcher.fetch(index) # type: ignore[possibly-undefined]
[rank0]: File "/home/.local/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 42, in fetch
[rank0]: data = next(self.dataset_iter)
[rank0]: File "/SLAM-LLM/src/slam_llm/datasets/speech_dataset_large.py", line 245, in iter
[rank0]: for elem in self.dp:
[rank0]: File "/SLAM-LLM/src/slam_llm/datasets/speech_dataset_large.py", line 84, in iter
[rank0]: with open(multitask_task_path) as f_task:
[rank0]: FileNotFoundError: [Errno 2] No such file or directory: ''

Expected behavior

The script should load the validation data from the validation file path (dev_scp_file_path) when running validation during training.

[teamtee]

I will test this problem tomorrow.

[teamtee]

Strangely, I didn't encounter this problem.

[teamtee]

The error above shows that the variable dev_scp_file_path is an empty string. It should be that your bash script has spaces or something like that, which causes this variable to not be assigned successfully.

[helloseraphina]

Thank you for the suggestion, but unfortunately the error still occurs for me even if the file exists.

FileNotFoundError: [Errno 2] No such file or directory: 'multitask.jsonl'

When I add some print statements into speech_dataset_large.py (under the MultiTaskDataset class and get_speech_dataset function), it shows me that it is the test split that is being instantiated even when putting the corresponding dev_scp_file_path in the bash script.

[2025-05-15 10:44:02][root][INFO] - dataset_config: {'dataset': 'multitask_dataset', 'train_max_frame_length': 1500, 'eval_max_frame_length': 3000, 'multitask_prompt_path': 'workspace/prompts_jsonl/multitask_train_prompts.jsonl', 'prompt_style': '{}', 'append_info_tasks': ['hotword'], 'file': 'workspace/SLAM-LLM/src/slam_llm/datasets/speech_dataset_large.py:get_speech_dataset', 'train_scp_file_path': 'workspace/multitask/train/', 'dev_scp_file_path': 'workspace/multitask/dev/', 'test_scp_file_path': '', 'train_split': 'train', 'dev_split': 'dev', 'test_split': 'test', 'pad_or_trim': True, 'prompt': 'Transcribe speech to text.', 'use_ocr': True, 'inference_mode': False, 'lower': False, 'fix_length_audio': -1, 'input_type': 'mel', 'mel_size': 128, 'normalize': False}
===Initializing dataset for split: train===
===Initializing MultiTaskDataset with split: train===
===Initializing dataset for split: test===
===Initializing MultiTaskDataset with split: test===
[2025-05-15 10:44:03][slam_llm.utils.config_utils][INFO] - Using batching strategy: dynamic
[2025-05-15 10:44:03][slam_llm.utils.config_utils][INFO] - Using batching strategy: dynamic
...

[teamtee]

In my test code, the second split is val

[rookie0607]

在我的测试代码中，第二个 split 是 val

我在测试中遇到了一些问题方便给您发邮件吗 @teamtee

[teamtee]

Ok,2624071330@qq.com

[rookie0607]

Ok,2624071330@qq.com

@teamtee 感谢回复，推理时https://github.com/X-LANCE/SLAM-LLM/blob/440ece648d0ae3686a1b770c0f54c46f49326073/src/slam_llm/datasets/speech_dataset_large.py#L180 处会报错，添加 以下代码就可以work了

if isinstance(samples[0], list):
    new_samples = []
    for sublist in samples:
        for s in sublist:
            new_samples.append(s)
    samples = new_samples