rnaseq_collapse:run_rnaseq_collapse (2)

[tiantian-chen]

I have used the latest version 4.1, but there are still errors. The RNA-SEQ I used has also been renamed to the following format.

SRR20250813.1.1
NTTTGCTTTTCGTTAACTCAGCAGACGGAGGAGTAAATGAGTAGATTGGGTGGCAGGTGGAGGGGAATGGGGTTGTCCGGCCATCTCCTACCTCTGGCGGCTGCAACTTCGAATCGATAGAGCATTCGAGCTTGGGGAGAAGAGATTCTA
SRR20250813.2.1
NCAGAAGCGACTCTACAACTTCACGGGCGCCATGGACCAGGACCCGTCGCTGGGCGCCCGGTACGCGAGGCAGCTGAGGAAGCTCTGCCCGCGCCGCCACGGCGGCCGGGGCAGGAGGACCAAGGTGCCCATGGACCCCGGCAGCGGCTT
SRR20250813.3.1

`executor > local (3799)
[93/1154a0] ega…tup_genome:get_genome_info | 1 of 1 ✔
[89/a81b11] ega…sk_stats:run_winmask_stats | 1 of 1 ✔
[0e/60bb4f] ega…:winmask_wnode:gpx_qsubmit | 1 of 1 ✔
[d3/1a5de8] ega…node:run_winmask_wnode (9) | 16 of 16 ✔
[2e/4cdc9d] ega…ask_wnode:gpx_make_outputs | 1 of 1 ✔
[60/9a0121] ega…ask_wnode:combine_blast_db | 1 of 1 ✔
[d1/815599] ega…pare_masks:gp_annot_format | 1 of 1 ✔
[25/d2cee3] ega…re_masks:convert_rfam_mask | 1 of 1 ✔
[93/43445f] ega…:combine_blast_db_softmask | 1 of 1 ✔
[87/e607b9] ega…bine_blast_db_alt_softmask | 1 of 1 ✔
[48/d8c982] ega…are_masks:convert_softmask | 1 of 1 ✔
[80/961eea] ega…blastdb:run_gc_makeblastdb | 1 of 1 ✔
[7a/26af20] ega…_stats:run_mask_assm_stats | 1 of 1 ✔
[f3/bbe0d1] ega…_proteins:convert_proteins | 1 of 1 ✔
[ac/76a4a8] ega…ne:miniprot:split_proteins | 1 of 1 ✔
[88/18d480] ega…miniprot:run_miniprot (12) | 27 of 27 ✔
[65/4651dc] ega…e:paf2asn:run_paf2asn (16) | 27 of 27 ✔
[38/7f1e25] ega…prot:run_best_aligned_prot | 1 of 1 ✔
[f6/f68b86] ega…ter_sa:run_align_filter_sa | 1 of 1 ✔
[84/bd96d3] ega…ign_sort_sa:run_align_sort | 1 of 1 ✔
[dd/170763] ega…ane:star_index:build_index | 1 of 1 ✔
[ef/c6c8b3] ega…rt_plane:star:run_star (1) | 1 of 1 ✔
[3a/25f65e] ega…seq_divide_by_strandedness | 1 of 1 ✔
[ee/cb6437] ega…d_sort:calc_assembly_sizes | 1 of 1 ✔
[e6/1c476a] ega…m_bin_and_sort:bam_bin (1) | 1 of 1 ✔
[40/85f128] ega…bin_and_sort:merge_prepare | 1 of 1 ✔
[e3/4eab6c] ega…bam_bin_and_sort:merge (4) | 9 of 9 ✔
[1c/02f6c4] ega…_plane:bam2asn:convert (6) | 9 of 9 ✔
[ca/942704] ega…seq_collapse:generate_jobs | 1 of 1 ✔
[db/852101] ega…se:run_rnaseq_collapse (6) | 0 of 10, retries: 3585
[26/b46e56] ega…2_index:run_minimap2_index | 1 of 1 ✔
[62/ce03db] ega…plane:rename_fasta_ids (1) | 1 of 1 ✔
[bc/42e4f4] ega…e:minimap2:gpx_qsubmit (1) | 1 of 1 ✔
[cc/182d81] ega…nimap2:minimap2_wnode (13) | 80 of 80 ✔
[b2/519fb9] ega…g_plane:minimap2:gpx_qdump | 1 of 1 ✔
[ee/15d228] ega…ter_est_align:align_filter | 1 of 1 ✔
[- ] ega…ilter_est_align:align_sort | 0 of 1
[4d/6a5c2b] ega…plane:fetch_swiss_prot_asn | 1 of 1 ✔
[f7/a70a83] ega…n_plane:get_swiss_prot_ids | 1 of 1 ✔
[32/ad6d5d] ega…_plane:print_fake_lxr_data | 1 of 1 ✔
[78/516ac2] ega…:fetch_ortholog_references | 1 of 1 ✔
[5a/57dc1b] ega…ext_genome:get_genome_info | 1 of 1 ✔
[28/c1a047] ega…_proteins:convert_proteins | 1 of 1 ✔
[5a/d7bcde] ega…ogy_plane:get_prot_ref_ids | 1 of 1 ✔

/ERROR ~ Error executing process > 'egapx:rnaseq_short_plane:rnaseq_collapse:run_rnaseq_collapse (2)'

Caused by:
Process egapx:rnaseq_short_plane:rnaseq_collapse:run_rnaseq_collapse (2) terminated with an error exit status (3)

Command executed:

njobs=wc -l <job.001
if [ $njobs -lt 16 ]; then
threads=$njobs
else
threads=16
fi

  echo "PanRefncpclean1-bin1.align_counts.txt

PanRefncpclean1-bin2.align_counts.txt
PanRefncpclean1-bin3.align_counts.txt
PanRefncpclean1-bin4.align_counts.txt
PanRefncpclean1-bin5.align_counts.txt
PanRefncpclean1-bin6.align_counts.txt
PanRefncpclean1-bin7.align_counts.txt
PanRefncpclean1-bin8.align_counts.txt
PanRefncpclean1-bin9.align_counts.txt" > scaffold_list.mft
echo "PanRefncpclean1-bin1.align.asnb.gz
PanRefncpclean1-bin2.align.asnb.gz
PanRefncpclean1-bin3.align.asnb.gz
PanRefncpclean1-bin4.align.asnb.gz
PanRefncpclean1-bin5.align.asnb.gz
PanRefncpclean1-bin6.align.asnb.gz
PanRefncpclean1-bin7.align.asnb.gz
PanRefncpclean1-bin8.align.asnb.gz
PanRefncpclean1-bin9.align.asnb.gz" > align.mft
echo "sra_metadata.dat" > metadata.mft

  # HACK: derive start_job_id from job file extension
  filename=$(basename -- "job.001")
  extension="${filename##*.}"
  # NB: for successful gather phase all job id should be unique,
  # so we must supply start_job_id.
  (( start_job_id = ((10#$extension) * 43) + 1 ))

  # make the local LDS of the genomic sequences
  lds2_indexer -source ./genome -db ./genome_lds  

  mkdir -p interim
  rnaseq_collapse -backlog 1 -max-jobs 1 -support-non-sra -O interim -nogenbank -lds2 ./genome_lds -sorted-vols align.mft -scaffold-list scaffold_list.mft -sra-metadata-manifest metadata.mft -start-job-id $start_job_id -input-jobs job.001 -workers $threads

  mkdir -p output
  cat interim/* > output/rnaseq_collapse.2.gpx-job.asnb
  rm -rf interim

Command exit status:
3

Command output:
(empty)

Command error:
00555/007/0058/R 0970022B8A520D41 0357/0011 2025-08-20T01:12:10.122913 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse Error: LIB "wn_worker_thread.cpp", line 267: ncbi::CWorkerThread::x_DoJob() --- error processing job: (CException::eUnknown) Unknown run: SRR20250813 alignments
00555/007/0058/RE 0970022B8A520D41 0358/0012 2025-08-20T01:12:10.122999 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse request-stop 500 3.277693033 0 40
00555/009/0059/R 0970022B8A520D41 0359/0009 2025-08-20T01:12:10.384958 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse extra num_skipped=960217
00555/009/0059/R 0970022B8A520D41 0360/0010 2025-08-20T01:12:10.385128 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse Error: RNASEQ(CException::eUnknown) "collapse_group.cpp", line 185: SMemberAlignment::SMemberAlignment() --- Unknown run: SRR20250813 alignments
00555/009/0059/R 0970022B8A520D41 0361/0011 2025-08-20T01:12:10.385137 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse Error: LIB "wn_worker_thread.cpp", line 267: ncbi::CWorkerThread::x_DoJob() --- error processing job: (CException::eUnknown) Unknown run: SRR20250813 alignments
00555/009/0059/RE 0970022B8A520D41 0362/0012 2025-08-20T01:12:10.385219 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse request-stop 500 3.444211006 0 40
00555/011/0076/R 0970022B8A520D41 0363/0015 2025-08-20T01:12:10.428967 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse extra num_skipped=564144
00555/011/0076/R 0970022B8A520D41 0364/0016 2025-08-20T01:12:10.429073 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse Error: RNASEQ(CException::eUnknown) "collapse_group.cpp", line 185: SMemberAlignment::SMemberAlignment() --- Unknown run: SRR20250813 alignments
00555/011/0076/R 0970022B8A520D41 0365/0017 2025-08-20T01:12:10.429081 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse Error: LIB "wn_worker_thread.cpp", line 267: ncbi::CWorkerThread::x_DoJob() --- error processing job: (CException::eUnknown) Unknown run: SRR20250813 alignments
00555/011/0076/RE 0970022B8A520D41 0366/0018 2025-08-20T01:12:10.429137 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse request-stop 500 2.028527021 0 40
00555/014/0075/R 0970022B8A520D41 0367/0009 2025-08-20T01:12:10.484734 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse extra num_skipped=523419
00555/014/0075/R 0970022B8A520D41 0368/0010 2025-08-20T01:12:10.484901 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse Error: RNASEQ(CException::eUnknown) "collapse_group.cpp", line 185: SMemberAlignment::SMemberAlignment() --- Unknown run: SRR20250813 alignments
00555/014/0075/R 0970022B8A520D41 0369/0011 2025-08-20T01:12:10.484911 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse Error: LIB "wn_worker_thread.cpp", line 267: ncbi::CWorkerThread::x_DoJob() --- error processing job: (CException::eUnknown) Unknown run: SRR20250813 alignments
00555/014/0075/RE 0970022B8A520D41 0370/0012 2025-08-20T01:12:10.484991 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse request-stop 500 2.091407060 0 40
00555/016/0061/R 0970022B8A520D41 0371/0015 2025-08-20T01:12:10.517775 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse extra num_skipped=992737
00555/016/0061/R 0970022B8A520D41 0372/0016 2025-08-20T01:12:10.517895 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse Error: RNASEQ(CException::eUnknown) "collapse_group.cpp", line 185: SMemberAlignment::SMemberAlignment() --- Unknown run: SRR20250813 alignments
00555/016/0061/R 0970022B8A520D41 0373/0017 2025-08-20T01:12:10.517903 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse Error: LIB "wn_worker_thread.cpp", line 267: ncbi::CWorkerThread::x_DoJob() --- error processing job: (CException::eUnknown) Unknown run: SRR20250813 alignments
00555/016/0061/RE 0970022B8A520D41 0374/0018 2025-08-20T01:12:10.517970 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse request-stop 500 3.569376945 0 40
00555/013/0078/R 0970022B8A520D41 0375/0015 2025-08-20T01:12:10.777666 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse extra num_skipped=589249
00555/013/0078/R 0970022B8A520D41 0376/0016 2025-08-20T01:12:10.777877 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse Error: RNASEQ(CException::eUnknown) "collapse_group.cpp", line 185: SMemberAlignment::SMemberAlignment() --- Unknown run: SRR20250813 alignments
00555/013/0078/R 0970022B8A520D41 0377/0017 2025-08-20T01:12:10.777889 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse Error: LIB "wn_worker_thread.cpp", line 267: ncbi::CWorkerThread::x_DoJob() --- error processing job: (CException::eUnknown) Unknown run: SRR20250813 alignments
00555/013/0078/RE 0970022B8A520D41 0378/0018 2025-08-20T01:12:10.777994 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse request-stop 500 2.128391027 0 40
00555/015/0062/R 0970022B8A520D41 0379/0015 2025-08-20T01:12:10.912787 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse extra num_skipped=1051974
00555/015/0062/R 0970022B8A520D41 0380/0016 2025-08-20T01:12:10.912948 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse Error: RNASEQ(CException::eUnknown) "collapse_group.cpp", line 185: SMemberAlignment::SMemberAlignment() --- Unknown run: SRR20250813 alignments
00555/015/0062/R 0970022B8A520D41 0381/0017 2025-08-20T01:12:10.912958 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse Error: LIB "wn_worker_thread.cpp", line 267: ncbi::CWorkerThread::x_DoJob() --- error processing job: (CException::eUnknown) Unknown run: SRR20250813 alignments
00555/015/0062/RE 0970022B8A520D41 0382/0018 2025-08-20T01:12:10.913052 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse request-stop 500 3.799286127 0 40
00555/003/0080/RB 0970022B8A520D41 0383/0001 2025-08-20T01:12:10.913120 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse request-start internal_thread=output&processing_cycle=1&ncbi_app_version=0.0.28058&ncbi_app_sc_version=29&ncbi_app_vcs_revision=699892&ncbi_app_revision=699892
00555/003/0080/R 0970022B8A520D41 0384/0002 2025-08-20T01:12:10.913168 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse extra success_jobs=0&fail_jobs=40
00555/003/0080/R 0970022B8A520D41 0385/0003 2025-08-20T01:12:10.913592 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse extra attempts=1
00555/003/0080/RE 0970022B8A520D41 0386/0004 2025-08-20T01:12:10.913602 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse request-stop 200 0.000511169 0 0
00555/008/0063/R 0970022B8A520D41 0387/0009 2025-08-20T01:12:11.080214 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse extra num_skipped=1085457
00555/008/0063/R 0970022B8A520D41 0388/0010 2025-08-20T01:12:11.080393 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse Error: RNASEQ(CException::eUnknown) "collapse_group.cpp", line 185: SMemberAlignment::SMemberAlignment() --- Unknown run: SRR20250813 alignments
00555/008/0063/R 0970022B8A520D41 0389/0011 2025-08-20T01:12:11.080403 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse Error: LIB "wn_worker_thread.cpp", line 267: ncbi::CWorkerThread::x_DoJob() --- error processing job: (CException::eUnknown) Unknown run: SRR20250813 alignments
00555/008/0063/RE 0970022B8A520D41 0390/0012 2025-08-20T01:12:11.080502 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse request-stop 500 3.915654897 0 40
00555/018/0064/R 0970022B8A520D41 0391/0015 2025-08-20T01:12:11.254611 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse extra num_skipped=1121133
00555/018/0064/R 0970022B8A520D41 0392/0016 2025-08-20T01:12:11.254830 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse Error: RNASEQ(CException::eUnknown) "collapse_group.cpp", line 185: SMemberAlignment::SMemberAlignment() --- Unknown run: SRR20250813 alignments
00555/018/0064/R 0970022B8A520D41 0393/0017 2025-08-20T01:12:11.254841 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse Error: LIB "wn_worker_thread.cpp", line 267: ncbi::CWorkerThread::x_DoJob() --- error processing job: (CException::eUnknown) Unknown run: SRR20250813 alignments
00555/018/0064/RE 0970022B8A520D41 0394/0018 2025-08-20T01:12:11.254952 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse request-stop 500 4.040262937 0 40
00555/019/0065/R 0970022B8A520D41 0395/0015 2025-08-20T01:12:11.379773 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse extra num_skipped=1162086
00555/019/0065/R 0970022B8A520D41 0396/0016 2025-08-20T01:12:11.379946 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse Error: RNASEQ(CException::eUnknown) "collapse_group.cpp", line 185: SMemberAlignment::SMemberAlignment() --- Unknown run: SRR20250813 alignments
00555/019/0065/R 0970022B8A520D41 0397/0017 2025-08-20T01:12:11.379956 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse Error: LIB "wn_worker_thread.cpp", line 267: ncbi::CWorkerThread::x_DoJob() --- error processing job: (CException::eUnknown) Unknown run: SRR20250813 alignments
00555/019/0065/RE 0970022B8A520D41 0398/0018 2025-08-20T01:12:11.380049 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse request-stop 500 4.161355018 0 40
00555/003/0081/RB 0970022B8A520D41 0399/0005 2025-08-20T01:12:11.380240 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse request-start internal_thread=output&processing_cycle=2&ncbi_app_version=0.0.28058&ncbi_app_sc_version=29&ncbi_app_vcs_revision=699892&ncbi_app_revision=699892
00555/003/0081/R 0970022B8A520D41 0400/0006 2025-08-20T01:12:11.380262 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse extra success_jobs=0&fail_jobs=3
00555/003/0081/R 0970022B8A520D41 0401/0007 2025-08-20T01:12:11.380275 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse extra attempts=1
00555/003/0081/RE 0970022B8A520D41 0402/0008 2025-08-20T01:12:11.380283 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse request-stop 200 0.000060081 0 0
00555/000/0000/P 0970022B8A520D41 0403/0088 2025-08-20T01:12:12.372696 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse Error: LIB(CException::eUnknown) "wn_app_core.cpp", line 256: ncbi::CGPX_WorkerAppCore::Run() --- 43 jobs failed
00555/000/0000/P 0970022B8A520D41 0404/0089 2025-08-20T01:12:12.372721 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse Error: CORELIB(106.16) "ncbiapp.cpp", line 725: ncbi::CNcbiApplicationAPI::x_TryMain() --- Application's execution failed (CException::eUnknown) 43 jobs failed
00555/000/0000/PE 0970022B8A520D41 0405/0090 2025-08-20T01:12:12.428209 c01n0046 UNK_CLIENT UNK_SESSION rnaseq_collapse stop 3 24.203124046
++ kill 36

Container:
/share/org/YZWL/egapx.0.4.1-alpha.sif

`

[etvedte]

This might be similar to #161

I'm assuming you are running this in offline mode because you are using SRA data stored locally? Can you post your input YAML and egapx.py command?

Have you tried using the example data?

[tiantian-chen]

Sure, I'm running in offline mode. Below are the contents of the two files.

genome: /share/org/cau_genome/PanRefncpclean1.fasta
taxid: 4565
locus_tag_prefix: PanRef
long_reads:
- /share/org/cau_t2t_rna_seq/cleanSRR28004297.fa
short_reads:
- /share/org/cau_anno/dup/fq_fa/SRR20250813_R1.fa
- /share/org/cau_anno/dup/fq_fa/SRR20250813_R2.fa 

egapx.py

[etvedte]

Do the short reads come from SRA, or is it your own local data?

If it is SRA data, the ids SRR20250813 seem like a mistake.

If it is local data, do you have the same error if you don't rename the read ids?

[tiantian-chen]

My data is local. If I don't rename it, will I be able to test it directly using the original fq files?

[etvedte]

Please do try, using the method to explicitly set the library name and read paths.

short_reads:
 - - paired_end_library_name1   # set name  
   - - path/to/pe1_reads_R1.fq  # file name for paired-end R1 reads
     - path/to/pe1_reads_R2.fq  # file name for paired-end R2 reads

[tiantian-chen]

Thank you. I'll run it again this way first.