Add BBTools Java implementation for fqcnt benchmark#25
Open
bbushnell wants to merge 1 commit intolh3:masterfrom
Open
Add BBTools Java implementation for fqcnt benchmark#25bbushnell wants to merge 1 commit intolh3:masterfrom
bbushnell wants to merge 1 commit intolh3:masterfrom
Conversation
Adds fqcnt_java_bbtools.sh wrapper for BBTools FastqScan. BBTools is a comprehensive suite of bioinformatics tools developed at JGI. FastqScan uses multithreaded SIMD-accelerated parsing for high performance on modern hardware. Tested with M_abscessus_HiSeq.fq (5,682,010 reads). Requirements: - Java 18+ (for jdk.incubator.vector SIMD support) - Java 25 recommended for optimal performance - BBTools: git clone --depth=1 https://github.com/bbushnell/BBTools
Author
Performance Note: FastqScan is fastest with larger files and BGZF compressionJVM Startup OverheadJava has ~0.25s startup/JIT compilation overhead that dominates benchmarks on small files (like the 5.6M read test case). This overhead is:
BGZF Multithreaded DecompressionFastqScan is actually faster on BGZF-compressed files than plaintext due to parallel decompression, if there are sufficient cores (~20). On 80M reads:
Performance comparison showing FastqScanMT at 13.5x faster than Rust needletail on BGZF files |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds BBTools FastqScan as a Java implementation for the fqcnt benchmark.
Implementation Details
fqcnt_java_bbtools.sh<records>\t<bases>\t<qualities>Testing
Tested with
M_abscessus_HiSeq.fq(5,682,010 reads):Requirements
git clone --depth=1 https://github.com/bbushnell/BBToolsAbout BBTools
BBTools is a comprehensive suite of bioinformatics tools developed at the Joint Genome Institute (JGI). FastqScan provides high-performance FASTQ parsing optimized for modern hardware.
Repository: https://github.com/bbushnell/BBTools