Release 1.20

Author: daviesrob

INSTALL

@@ -38,6 +38,7 @@ a development ('-dev' or '-devel') package separate from the main library.
     liblzma    (required, unless configured with --disable-lzma)
     libcurl    (optional, but strongly recommended)
     libcrypto  (optional for Amazon S3 support; not needed on MacOS)
+    libdeflate (optional, but strongly recommended for faster gzip)
 
 Disabling libbzip2 and liblzma will make some CRAM files unreadable, so
 is not recommended.
@@ -252,14 +253,14 @@ Debian / Ubuntu
 ---------------
 
 sudo apt-get update  # Ensure the package list is up to date
-sudo apt-get install autoconf automake make gcc perl zlib1g-dev libbz2-dev liblzma-dev libcurl4-gnutls-dev libssl-dev
+sudo apt-get install autoconf automake make gcc perl zlib1g-dev libbz2-dev liblzma-dev libcurl4-gnutls-dev libssl-dev libdeflate-dev
 
 Note: libcurl4-openssl-dev can be used as an alternative to libcurl4-gnutls-dev.
 
 RedHat / CentOS
 ---------------
 
-sudo yum install autoconf automake make gcc perl-Data-Dumper zlib-devel bzip2 bzip2-devel xz-devel curl-devel openssl-devel
+sudo yum install autoconf automake make gcc perl-Data-Dumper zlib-devel bzip2 bzip2-devel xz-devel curl-devel openssl-devel libdeflate-devel
 
 Note: On some versions perl FindBin will need to be installed to make the tests work.
 
@@ -271,13 +272,19 @@ Alpine Linux
 doas apk update  # Ensure the package list is up to date
 doas apk add autoconf automake make gcc musl-dev perl bash zlib-dev bzip2-dev xz-dev curl-dev openssl-dev
 
+Ideally also install a copy of libdeflate-dev for faster (de)compression.
+This can be found in the Alpine community repository.
+
 Note: some older Alpine versions use libressl-dev rather than openssl-dev.
 
 OpenSUSE
 --------
 
 sudo zypper install autoconf automake make gcc perl zlib-devel libbz2-devel xz-devel libcurl-devel libopenssl-devel
 
+Also install libdeflate-devel, available on OpenSUSE Leap 15.4 onwards
+or directly via git releases above.
+
 Windows MSYS2/MINGW64
 ---------------------
 

Makefile

@@ -150,8 +150,8 @@ LIBHTS_SOVERSION = 3
 # is not strictly necessary and should be removed the next time
 # LIBHTS_SOVERSION is bumped (see #1144 and
 # https://developer.apple.com/library/archive/documentation/DeveloperTools/Conceptual/DynamicLibraries/100-Articles/DynamicLibraryDesignGuidelines.html#//apple_ref/doc/uid/TP40002013-SW23)
-MACH_O_COMPATIBILITY_VERSION = 3.1.19
-MACH_O_CURRENT_VERSION = 3.1.19
+MACH_O_COMPATIBILITY_VERSION = 3.1.20
+MACH_O_CURRENT_VERSION = 3.1.20
 
 # Force version.h to be remade if $(PACKAGE_VERSION) has changed.
 version.h: $(if $(wildcard version.h),$(if $(findstring "$(PACKAGE_VERSION)",$(shell cat version.h)),,force))

NEWS

@@ -1,3 +1,91 @@
+Noteworthy changes in release 1.20 (15th April 2024)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Updates
+-------
+
+* When working on named files, bgzip now sets the modified and access times
+  of the output files it makes to match those of the corresponding input.
+  (PR #1727, feature request #1718.  Requested by Gert Hulselmans)
+
+* It's now possible to use a -o option to specify the output file name in
+  bgzip.
+  (PR #1747, feature request #1726.  Requested by Gert Hulselmans)
+
+* Improved error faidx error messages.
+  (PR #1743, thanks to Nick Moore)
+
+* Faster reading of SAM array (type "B") tags.  These often turn up
+  in ONT and PacBio data.
+  (PR #1741)
+
+* Improved validity checking of base modification tags.
+  (PR #1749)
+
+* mpileup overlap removal now works where one read has a deletion.
+  (PR #1751, fixes samtools/samtools#1992.  Reported by Long Tian)
+
+* The S3 plugin can now find buckets via S3 access point aliases.
+  (PR #1756, thanks to Matt Pawelczyk;
+   fixes samtools/samtools#1984.  Reported by Albert Li)
+
+* Added a --threads option (and -@ short option) to tabix.
+  (PR #1755, feature request #1735.  Requested by Dan Bolser)
+
+* tabix can now index Graph Alignment Format (GAF) files.
+  (See https://github.com/lh3/gfatools/blob/master/doc/rGFA.md)
+  (PR #1763, thanks to Adam Novak)
+
+Bug fixes
+---------
+
+* Security fix: Prevent possible heap overflow in cram_encode_aux() on
+  bad RG:Z tags.
+  (PR #1737)
+
+* Security fix: Prevent attempts to call a NULL pointer if certain URL
+  schemes are used in CRAM @SQ UR: tags.
+  (PR #1757)
+
+* Security fix: Fixed a bug where following certain AWS S3 redirects could
+  downgrade the connection from TLS (i.e. https://) to unencrypted http://.
+  This could happen when using path-based URLs and AWS_DEFAULT_REGION
+  was set to a region other that the one where the data was stored.
+  (PR #1762, fixes #1760. Reported by andaca)
+
+* Fixed arithmetic overflow when loading very long references for CRAM.
+  (PR #1738, fixes #1738.  Reported by Shane McCarthy)
+
+* Fixed faidx and CRAM reference look-ups on compressed	fasta where the .fai
+  index file was present, but the .gzi index of compressed offsets was not.
+  (PR #1745, fixes #1744.  Reported by Theodore Li)
+
+* Fixed BCF indexing on-the-fly bug which produced invalid indexes when
+  using multiple compression threads.
+  (PR #1742, fixes #1740.  Reported by graphenn)
+
+* Ensure that pileup destructors are called by bam_plp_destroy(), to
+  prevent memory leaks.
+  (PR #1749, PR #1754)
+
+* Ensure on-the-fly index timestamps are always older than the data file.
+  Previously the files could be closed out of order, leading to warnings
+  being printed when using the index.
+  (PR #1753, fixes #1732.  Reported by Gert Hulselmans)
+
+* To prevent data corruption when reading (strictly invalid) VCF files
+  with duplicated FORMAT tags, all but the first copy of the data
+  associated with the tag are now dropped with a warning.
+  (PR #1752, PR #1761, fixes #1733.  Reported by anthakki)
+
+* Fixed a bug introduced in release 1.19 (PR #1689) which broke variant
+  record data if it tried to remove an over-long tag.
+  (PR #1752, PR #1761)
+
+* Changed error to warning when complaining about use of the CG tag
+  in SAM or CRAM files.
+  (PR #1758, fixes samtools/samtools#2002)
+
 Noteworthy changes in release 1.19.1 (22nd January 2024)
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -33,6 +121,7 @@ Updates
 
   In future work, the library will instead convert such tags into their
   local alternatives (see https://github.com/samtools/hts-specs/pull/434).
+  (PR #1689)
 
 * New program. Adds annot-tsv which annotates regions in a destination file with
   texts from overlapping regions in a source file.

annot-tsv.1

@@ -1,5 +1,5 @@
 '\" t
-.TH annot-tsv 1 "22 January 2024" "htslib-1.19.1" "Bioinformatics tools"
+.TH annot-tsv 1 "15 April 2024" "htslib-1.20" "Bioinformatics tools"
 .\"
 .\" Copyright (C) 2015, 2017-2018, 2023 Genome Research Ltd.
 .\"

bgzf.c

@@ -2,7 +2,7 @@
 
    Copyright (c) 2008 Broad Institute / Massachusetts Institute of Technology
                  2011, 2012 Attractive Chaos <[email protected]>
-   Copyright (C) 2009, 2013-2022 Genome Research Ltd
+   Copyright (C) 2009, 2013-2023 Genome Research Ltd
 
    Permission is hereby granted, free of charge, to any person obtaining a copy
    of this software and associated documentation files (the "Software"), to deal
@@ -1959,6 +1959,11 @@ int bgzf_flush(BGZF *fp)
         return ret;
     }
 #endif
+
+    if (!fp->is_compressed) {
+        return hflush(fp->fp);
+    }
+
     while (fp->block_offset > 0) {
         int block_length;
         if ( fp->idx_build_otf )

bgzip.1

@@ -1,10 +1,10 @@
-.TH bgzip 1 "22 January 2024" "htslib-1.19.1" "Bioinformatics tools"
+.TH bgzip 1 "15 April 2024" "htslib-1.20" "Bioinformatics tools"
 .SH NAME
 .PP
 bgzip \- Block compression/decompression utility
 .\"
 .\" Copyright (C) 2009-2011 Broad Institute.
-.\" Copyright (C) 2018, 2021-2023 Genome Research Limited.
+.\" Copyright (C) 2018, 2021-2024 Genome Research Limited.
 .\"
 .\" Author: Heng Li <[email protected]>
 .\"
@@ -50,6 +50,8 @@ bgzip \- Block compression/decompression utility
 .IR index_name ]
 .RB [ -l
 .IR compression_level ]
+.RB [ -o
+.IR outfile ]
 .RB [ -s
 .IR size ]
 .RB [ -@
@@ -71,7 +73,9 @@ otherwise when compressing bgzip will write to a new file with a .gz
 suffix and remove the original.  When decompressing the input file must
 have a .gz suffix, which will be removed to make the output name.  Again
 after decompression completes the input file will be removed. When multiple
-files are given as input, the operation is performed on all of them.
+files are given as input, the operation is performed on all of them. Access
+and modification time of input file from filesystem is set to output file. 
+Note, access time may get updated by system when it deems appropriate.
 
 .SH OPTIONS
 .TP 10
@@ -128,6 +132,10 @@ Do not delete input file during operation.
 Compression level to use when compressing.
 From 0 to 9, or -1 for the default level set by the compression library. [-1]
 .TP
+.BI "-o, --output " FILE
+Write to a file, keep original files unchanged, will overwrite an existing
+file.
+.TP
 .B "-r, --reindex"
 Rebuild the index on an existing compressed file.
 .TP
@@ -136,7 +144,7 @@ Decompress INT bytes (uncompressed size) to standard output.
 Implies -c.
 .TP
 .B "-t, --test"
-Test the intregrity of the compressed file.
+Test the integrity of the compressed file.
 .TP
 .BI "-@, --threads " INT
 Number of threads to use [1].