Skip to content

Releases: ipfs/boxo

v0.41.0

Choose a tag to compare

@github-actions github-actions released this 08 Jun 11:35
9e8010b

Note

This release was brought to you by the Shipyard team.

What's Changed

Added

  • ipld/merkledag/traverse: added the Visited interface and Options.Visited field, so callers can plug in their own set for skipping duplicates when SkipDuplicates is on. *go-cid.Set already works as one; a bounded or disk-backed set keeps memory low on very large DAGs.

Changed

Fixed

  • routing/http/server: GET /routing/v1/ipns/{name} no longer gives a cache a window that outlasts the record. It caps max-age to the record's remaining validity and sizes the stale window (stale-while-revalidate/stale-if-error) to the time left after it, so the two never cross the record's EOL. An expired record, or one whose ValidityType is not EOL (unknown expiration), returns Cache-Control: no-store, and a negative TTL no longer yields a negative max-age. #1166
  • gateway: serving a raw IPNS record (GET /ipns/{name}?format=ipns-record) now caps max-age to the record's remaining validity and never lets it go negative, so a cache cannot reuse the record past its EOL. #1166
  • namesys: the IPNS resolver now floors a negative record TTL at zero, so a malformed record can no longer surface a negative TTL through Result.TTL. #1166
  • namesys: a cache hit now reports the TTL remaining in the cache entry rather than the record's original TTL, so a late hit near a record's EOL can no longer advertise a freshness lifetime that outlives the record. #1166
  • ipns: NewRecord floors a negative TTL at zero and Validate rejects records carrying one. #1166
  • bitswap/network/bsnet: stop marking a peer unresponsive on a single failed send attempt. send() is retried by multiAttempt(), which already marks the peer once all retries are exhausted; marking on the first failure could permanently sideline a peer that had just reconnected (the disconnect notification being suppressed), hanging fetches from it until it fully disconnected. #1164

Security

  • tracing: bumped OpenTelemetry OTLP exporters to v1.43.0, which caps the HTTP exporter's response body at 4 MiB. A hostile or man-in-the-middle collector could otherwise exhaust its memory (CVE-2026-39882). The gRPC exporter is unaffected.

Full Changelog: v0.40.0...v0.41.0

v0.40.0

Choose a tag to compare

@github-actions github-actions released this 26 May 20:06
71d6e88

What's Changed

Added

  • retrieval: added State.Snapshot, State.Apply, and State.Notify so consumers can stream State across a process boundary, e.g. to drive a live progress bar in Kubo's cat, get, or dag export. #1153
  • 🛠 pinning/pinner: added Pinner.Close() error. Close cancels every in-flight operation's context, including streaming goroutines from RecursiveKeys, DirectKeys, and InternalPins, and waits for them to return. A scalar method that observes the cancellation may return context.Canceled; a stream interrupted by Close may surface ErrClosed on the channel before it closes. After Close returns, every other method returns the new ErrClosed sentinel; streaming methods deliver it as StreamedPin.Err on a single entry, then close the channel. Close is idempotent and goroutine-safe. Action required: downstream Pinner implementations must add Close. #1150
  • pinning/pinner/dspinner: implements Close. Close cancels the contexts of in-flight operations, so snapshot iteration in RecursiveKeys/DirectKeys and DAG fetches in Pin bail out promptly instead of draining to completion. Close returns as soon as those operations honor their ctx. Hosts owning the datastore should call Close on the pinner before closing the datastore to avoid the use-after-close panic path in stores such as pebble. #1150
  • routing/http/types/iter: added Limit, an iterator that caps another iterator at a fixed number of values. #1157
  • routing/providerquerymanager: new WithFindPeerFallback option. When set, a one-shot FindPeer fallback runs if the first dial to a provider fails; the manager retries the dial with the fresh AddrInfo, but only if FindPeer surfaced at least one address that wasn't already in the routing-record set just tried. Rescues providers whose routing-record snapshot is thin or stale but whose actual addresses are still reachable, without wasting a retry on a duplicate address set. Disabled by default; pass WithFindPeerFallback(myDHT) to enable.

Changed

  • upgrade to go-libp2p-kad-dht v0.40.0

  • 🛠 files: the File interface no longer embeds io.Seeker. Seekability is now explicit in the type system instead of implied by an always-present Seek method that returned ErrNotSupported at runtime for non-seekable inputs. Callers that need to seek type-assert to io.Seeker; the assertion is an honest capability check rather than a guess. Seekable implementations (ReaderFile wrapping a seekable reader, Symlink, UnixFS files returned from the gateway and importer) still satisfy the assertion. Non-seekable implementations (HTTP multipart streams, WebFile) now fail the assertion at compile-aware sites instead of producing runtime errors deep in third-party code. The previous behavior forced downstream workarounds: e.g. ipfs/kubo#11253 had to wrap files.File in a plain io.Reader/io.Closer to strip Seek and force go-car's forward-only fallback, because go-car's NewBlockReader trusted the interface and called Seek, which failed with "operation not supported" on CARv2 imports over the HTTP API. With this change the trap stops existing.

    Action required. Replace direct Seek calls on files.File with a type assertion:

    // before
    n, err := f.Seek(offset, io.SeekStart)
    
    // after
    seeker, ok := f.(io.Seeker)
    if !ok {
        return fmt.Errorf("file does not support seeking")
    }
    n, err := seeker.Seek(offset, io.SeekStart)

    See ipfs/kubo#11254 for a worked example of the call-site update. #1128

  • routing/http/server: the Delegated Routing server now passes limit=0 (unbounded) to DelegatedRouter.FindProviders/FindPeers and applies the configured records limit itself, after filtering. Filtered requests now return a full page of results instead of fewer than requested. The server reads the delegate's iterator lazily and closes it once it has enough records. Action required: delegate implementations should return results lazily and stop work on Close. A delegate that previously used the limit argument to end its walk early should now end the walk on Close instead. #1157

  • path/resolver: ResolveToLastNode, ResolvePath, and ResolvePathComponents now populate retrieval.State on the request context when one is attached. They advance the state to PhasePathResolution, record the root CID from the input path, and record the terminal CID once resolution completes. Until now only the gateway backends populated these fields, leaving non-gateway callers (CLIs, custom tools) without phase or CID diagnostics on retrieval errors. The new calls are idempotent with the existing gateway-side ones, so behavior on the gateway path is unchanged.

Fixed

  • files: now builds under GOOS=js GOARCH=wasm and GOOS=wasip1 GOARCH=wasm. #935
  • routing/http/server: filtered /routing/v1/providers and /routing/v1/peers requests now return up to the configured records limit. Previously the limit was applied before filter-addrs/filter-protocols ran, so records dropped by the filters shrank the response below the limit. The limit now applies after filtering. #1157
  • routing/http/types/iter: Filter.Next now iterates instead of recursing on rejected values, so the goroutine stack stays flat even when a long run of records is filtered out. This matters now that the server pulls unbounded results from the delegate. #1157

Full Changelog: v0.39.0...v0.40.0

Note

This release was brought to you by the Shipyard team.

v0.39.0

Choose a tag to compare

@github-actions github-actions released this 23 Apr 18:44
23c380b

Note

This release was brought to you by the Shipyard team.

What's Changed

Added

  • gateway: Config.MaxDeserializedResponseSize allows setting a maximum file/directory size for deserialized gateway responses. Content exceeding this limit returns 410 Gone, directing users to run their own IPFS node. Trustless response formats (application/vnd.ipld.raw, application/vnd.ipld.car) are not affected. The size is read from the UnixFS root block, so no extra block fetches are needed for the check. #1138
  • gateway: Config.MaxUnixFSDAGResponseSize allows setting a maximum content size applied to all response formats (deserialized, raw blocks, CAR, TAR). Content exceeding this limit returns 410 Gone. For most handlers the check reuses size information already available in the request path; for CAR responses a lightweight Head call is made only when the limit is configured. #1138

Changed

  • bitswap/server: the default peer comparator now schedules peers fairly. A peer that has never been served, or has waited longer than 10s, outranks non-starved peers. Pending counts cap at 16 for ordering purposes, so peers with small wantlists no longer wait behind peers with large ones. The final tiebreak uses a per-process salted hash of peer.ID, so no peer can craft an ID that permanently outranks everyone. Engines built with WithTaskComparator keep their existing behavior. #1141
  • upgrade to go-libp2p-kad-dht v0.39.1

Fixed

  • bitswap/network/bsnet: SendMessage and handleNewStream now close streams in a background goroutine. Previously, stream.Close could hold the caller for up to DefaultNegotiationTimeout (10s) while lazyClientConn.Close waited for the remote peer to complete the multistream handshake. This saturated the bitswap TaskWorkerCount pool when peers were unresponsive and stopped bitswap from serving blocks to other peers. As a side effect, SendMessage no longer returns errors from stream.Close; close failures are logged at Debug. #1142
  • bitswap/server: a peer with a single pending want no longer waits behind peers with large wantlists. #1141
  • pinner/dspinner: RecursiveKeys and DirectKeys now snapshot the pin index under the read lock and release it before emitting pins, so a slow consumer (e.g. the reprovider draining the channel at DHT speed under Provide.Strategy=pinned*) can no longer starve Pin/Unpin/Flush writers. #1140

Full Changelog: v0.38.0...v0.39.0

v0.38.0

Choose a tag to compare

@github-actions github-actions released this 09 Apr 17:56
a696a54

Note

This release was brought to you by the Shipyard team.

What's Changed

Added

  • ipns: NewRecord accepts WithMetadata(map[string]any) option for storing custom scalar key-value pairs (string, []byte, int64, int, bool) in the signed DAG-CBOR data of IPNS records. Metadata can be read back via Record.Metadata (returns typed MetadataValue with Kind() discriminator) and iterated with Record.MetadataEntries. Reserved IPNS field names, empty keys, and unsupported value types are rejected. #1085
  • dag/walker: new package for memory-efficient DAG traversal with deduplication. VisitedTracker interface with BloomTracker (scalable bloom filter chain, ~4 bytes/CID vs ~75 bytes for a map) and MapTracker (exact, for tests). WalkDAG provides iterative DFS traversal with integrated dedup, supporting dag-pb, dag-cbor, raw, and other registered codecs. ~2x faster than the legacy go-ipld-prime selector-based traversal. WalkEntityRoots emits only entity roots (files, directories, HAMT shards) instead of every block, skipping internal file chunks. #1124
  • pinner: NewUniquePinnedProvider and NewPinnedEntityRootsProvider log and skip corrupted pin entries instead of aborting the provide cycle, allowing remaining pins to still be provided. #1124
  • routing/http/client: WithProviderInfoFunc option resolves provider addresses at provide-time instead of client construction time. This only impacts legacy HTTP-only custom routing setups that depend on IPIP-526 and were sending unresolved 0.0.0.0 addresses in provider records instead of actual interface addresses. #1115
  • chunker: added Register function to allow custom chunkers to be registered for use with FromString.
  • mfs: added Directory.Mode() and Directory.ModTime() getters to match the existing File.Mode() and File.ModTime() API. #1131

Changed

  • provider: NewPrioritizedProvider now continues to the next stream when one fails instead of stopping all streams. NewConcatProvider added for pre-deduplicated streams. #1124
  • chunker: FromString now rejects malformed size- strings with extra parameters (e.g. size-123-extra was previously silently accepted).
  • gateway: compliance with gateway-conformance v0.13
  • upgrade to go-libp2p v0.48.0
  • 🛠 mfs: replaced RootOption with a unified Option functional options pattern (e.g. WithCidBuilder, WithChunker, WithMaxLinks). NewRoot, NewEmptyRoot, MkdirWithOpts, and NewEmptyDirectory now accept ...Option. Mkdir takes a MkdirOpts struct (narrowed to Mkparents and Flush flags) followed by ...Option for directory configuration. #1125

Removed

  • gateway: removed dead DoH resolver for .crypto TLD (Unstoppable Domains) #772
  • cmd/boxo-migrate: removed code for go-ipfs migration -- no longer needed.
  • cmd/deprecator: removed code to deprecare relocated ipfs packages -- no longer needed.

Fixed

  • bitswap/server: incoming identity CIDs in wantlist messages are now silently ignored instead of killing the connection to the remote peer. Some IPFS implementations naively send identity CIDs, and disconnecting them for it caused unnecessary churn. #1117
  • bitswap/network: ExtractHTTPAddress now infers default ports for portless HTTP multiaddrs (e.g. /dns/host/https without /tcp/443). #1123
  • mfs: FileDescriptor operations are serialized with a mutex, preventing data races on the underlying DagModifier when FUSE mounts or Kubo RPC commands dispatch concurrent Read, Write, Seek, Truncate, Flush, or Close calls. Flush after Close returns ErrClosed. #1131 #1133
  • mfs: preserve CidBuilder and SizeEstimationMode across setNodeData(), Mkdir() and NewRoot(). #1125
  • mfs: closing a file descriptor after its directory entry was removed (e.g. FUSE RELEASE racing with RENAME) no longer re-adds the stale entry to the parent directory. #1134
  • mfs: SetMode and SetModTime no longer drop file content links when updating UnixFS metadata. #1134

Full Changelog: v0.37.0...v0.38.0

v0.37.0

Choose a tag to compare

@github-actions github-actions released this 16 Feb 10:57
2d36cd3

Note

This release was brought to you by the Shipyard team.

What's Changed

Added

  • ipld/unixfs/io: added SizeEstimationMode for configurable HAMT sharding threshold decisions. Supports legacy link-based estimation (SizeEstimationLinks), accurate block-based estimation (SizeEstimationBlock), or disabling size-based thresholds (SizeEstimationDisabled). #1088, IPIP-499
  • ipld/unixfs/io: added UnixFSProfile with UnixFS_v0_2015 and UnixFS_v1_2025 presets for CID-deterministic file and directory DAG construction. #1088, IPIP-499
  • files: NewSerialFileWithOptions now supports controlling whether symlinks are preserved or dereferenced before being added to IPFS. See SerialFileOptions.DereferenceSymlinks. #1088, IPIP-499

Changed

  • 🛠 chunker, ipld/unixfs/importer/helpers: block size limits raised from 1MiB to 2MiB to match the bitswap spec. Max chunker size is 2MiB - 256 bytes to leave room for protobuf framing when --raw-leaves=false. IPIP-499 profiles use lower chunk sizes (256KiB and 1MiB) and are not affected.
  • 🛠 chunker: DefaultBlockSize changed from const to var to allow runtime configuration via global profiles. #1088, IPIP-499
  • gateway: 🛠 ✨ IPIP-523 ?format= URL query parameter now takes precedence over Accept HTTP header, ensuring deterministic HTTP cache behavior and allowing browsers to use ?format= even when they send Accept headers with specific content types. #1074
  • gateway: 🛠 ✨ IPIP-524 codec conversions (e.g., dag-pb to dag-json, dag-json to dag-cbor) are no longer performed by default. Requesting a format that differs from the block's codec now returns HTTP 406 Not Acceptable with a hint to fetch raw blocks (?format=raw) and convert client-side. Set Config.AllowCodecConversion to true to restore the old behavior. #1077
  • gateway: compliance with gateway-conformance v0.10.0 (since v0.8: relaxed DAG-CBOR HTML preview cache headers, relaxed CAR 200/404 for missing paths, IPIP-523 format query precedence, IPIP-524 codec mismatch returns 406)
  • upgrade to go-ipld-prime v0.22.0
  • upgrade to go-libp2p-kad-dht v0.38.0

Removed

  • tracing: opentelemetry zipkin exporter (go.opentelemetry.io/otel/exporters/zipkin) is deprecated and has been removed. It is recommended to switch to OTLP. Configure your application to send traces using OTLP and enable Zipkin’s OTLP ingestion support.

Fixed

  • 🛠 ipld/unixfs/io: fixed HAMT sharding threshold comparison to use > instead of >=. A directory exactly at the threshold now stays as a basic (flat) directory, aligning behavior with code documentation and the JS implementation. This is a theoretical breaking change, but unlikely to impact real-world users as it requires a directory to be exactly at the threshold boundary. If you depend on the old behavior, adjust HAMTShardingSize to be 1 byte lower. #1088, IPIP-499
  • ipld/unixfs/mod: fixed sparse file writes in MFS. Writing past the end of a file (e.g., ipfs files write --offset 1000 /file on a smaller file) would lose data because expandSparse created the zero-padding node but didn't update the internal pointer. Subsequent writes went to the old unexpanded node.
  • ipld/unixfs/io: fixed mode/mtime metadata loss during Basic<->HAMT directory conversions. Previously, directories with WithStat(mode, mtime) would lose this metadata when converting between basic and sharded formats, or when reloading a HAMT directory from disk.

Full Changelog: v0.36.0...v0.37.0

v0.36.0

Choose a tag to compare

@github-actions github-actions released this 27 Jan 23:23
63b6a19

What's Changed

Added

  • routing/http: GET /routing/v1/dht/closest/peers/{key} per IPIP-476
  • ipld/merkledag: Added fetched node size reporting to the progress tracker. See kubo#8915
  • gateway: Added a configurable fallback timeout for the gateway handler, defaulting to 1 hour. Configurable via MaxRequestDuration in the gateway config.

Changed

  • keystore: improve error messages and include key file name #1080
  • upgrade to go-libp2p-kad-dht v0.37.1
  • upgrade to go-libp2p v0.47.0

Fixed

  • bitswap/network: Fixed goroutine leak that could cause bitswap to stop serving blocks after extended uptime. The root cause is stream.Close() blocking indefinitely when remote peers are unresponsive during multistream handshake (go-libp2p#3448). This PR (#1083) adds a localized fix specific to bitswap's SendMessage by setting a read deadline before closing streams.

Full Changelog: v0.35.2...v0.36.0

v0.35.2

Choose a tag to compare

@github-actions github-actions released this 06 Nov 18:15
fc4c0e6

What's Changed

  • upgrade to go-libp2p v0.45.0
  • upgrade to go-log/v2 v2.9.0
    • Applications using go-log (>=2.9)+go-libp2p(>=0.45) may need to initialize their application to bridge slog-based libraries to into go-log. See documentation for go-log release and slog integration.

Full Changelog: v0.35.1...v0.35.2

v0.35.1

Choose a tag to compare

@github-actions github-actions released this 29 Oct 23:27
d16a4d5

Note

This release was brought to you by the Shipyard team.

What's Changed

Added

  • new span for the handleIncoming bitswap client getter plus events when blocks are received.
  • mark opentelemetry spans, span attributes, and span events as being used by ProbeLab's analysis scripts

Changed

  • upgrade to go-dsqueue v0.1.0 - Fixes batch reuse that could cause panic.

Fixed

  • gateway: Fixed duplicate peer IDs appearing in retrieval timeout error messages
  • bitswap/client: fix tracing by using context to pass trace and retrieval state to session #1059
    • bitswap/client: propagate trace state when calling GetBlocks #1060
  • bitswap/network/httpnet: improved error detection on HTTP and block fetches:
    • Do not attempt to GET a test CID if the endpoint returns 429 to the test HEAD request.
    • Unify error parsing and handling of http statues and content.

Full Changelog: v0.35.0...v0.35.1

v0.35.0

Choose a tag to compare

@github-actions github-actions released this 01 Oct 01:09
69ec427

Note

This release was brought to you by the Shipyard team.

What's Changed

Added

  • pinning/pinner: Added CheckIfPinnedWithType method to Pinner interface for efficient type-specific pin checks with optional name loading (#1035)
    • Enables checking specific pin types (recursive, direct, indirect) without loading all pins
    • Optional includeNames parameter controls whether pin names are loaded from datastore
    • CheckIfPinned now delegates to CheckIfPinnedWithType for consistency
  • gateway: Enhanced error handling and UX for timeouts:
    • Added retrieval state tracking for timeout diagnostics. When retrieval timeouts occur, the error messages now include detailed information about which phase failed (path resolution, provider discovery, connecting, or data retrieval) and provider statistics including failed peer IDs #1015 #1023
    • Added Config.DiagnosticServiceURL to configure a CID retrievability diagnostic service. When set, 504 Gateway Timeout errors show a "Check CID retrievability" button linking to the service with ?cid=<failed-cid> #1023
    • Improved 504 error pages with "Retry" button, diagnostic service integration, and clear indication when timeout occurs on sub-resource vs root CID #1023
  • gateway: Added Config.MaxRangeRequestFileSize to protect against CDN issues with large file range requests. When set to a non-zero value, range requests for files larger than this limit return HTTP 501 Not Implemented with a suggestion to use verifiable block requests (application/vnd.ipld.raw) instead. This provides protection against Cloudflare's issue where range requests for files over 5GiB are silently ignored, causing excess bandwidth consumption and billing

Changed

  • routing/http: ✨ Delegated Routing V1 HTTP endpoints now return 200 with empty results instead of 404 when no records are found, per IPIP-513 (#1024)
    • Server endpoints (/routing/v1/providers/{cid}, /routing/v1/peers/{peer-id}, /routing/v1/ipns/{name}) return HTTP 200 with empty JSON arrays or appropriate content types for empty results
    • Client maintains backward compatibility by treating both 200 with empty results and 404 as "no records found"
    • IPNS endpoint distinguishes between valid records (Content-Type: application/vnd.ipfs.ipns-record) and no record found (any other content type)
  • verifcid: 🛠 Enhanced Allowlist interface with per-hash size limits (#1018)
    • Expanded Allowlist interface with MinDigestSize(code uint64) and MaxDigestSize(code uint64) methods for per-hash function size validation
    • Added public constants: DefaultMinDigestSize (20 bytes), DefaultMaxDigestSize (128 bytes for cryptographic hashes), and DefaultMaxIdentityDigestSize (128 bytes for identity CIDs)
    • DefaultAllowlist implementation now uses these constants and supports different size limits per hash type
    • Renamed errors for clarity: Added ErrDigestTooSmall and ErrDigestTooLarge as the new primary errors
    • ErrBelowMinimumHashLength and ErrAboveMaximumHashLength remain as deprecated aliases pointing to the new errors
  • bitswap: Updated to use verifcid.DefaultMaxDigestSize for MaximumHashLength constant
    • The default MaximumAllowedCid limit for incoming CIDs can be adjusted using bitswap.MaxCidSize or server.MaxCidSize options
  • 🛠 bitswap/client: The RebroadcastDelay option now takes a time.Duration value. This is a potentially BREAKING CHANGE. The time-varying functionality of delay.Delay was never used, so it was replaced with a fixed duration value. This also removes the github.com/ipfs/go-ipfs-delay dependency.
  • filestore: Support providing filestore-blocks. A new provider.MultihashProvider parameter has been added to filestore.New(). When used, the blocks handled by the Filestore's FileManager will be provided on write (Put and PutMany).

Removed

  • provider: Provide() calls are replaced with StartProviding() to benefit from the Reprovide Sweep improvement. See kubo#10834 and kad-dht#1095.

Fixed

  • routing/http/client:
    • Fixed off-by-one error in routing_http_client_length metric - the metric now correctly reports 0 for empty results instead of 1
    • Added metrics for IPNS operations (GetIPNS and PutIPNS) - these now report latency, status code, and result count (0 or 1 for GetIPNS)
    • Added simple counter metrics to avoid confusing histogram bucket math:
      • routing_http_client_requests_total - total requests including errors
      • routing_http_client_positive_responses_total - requests that returned at least 1 result
  • ipld/unixfs/mod:
    • DagModifier now correctly preserves raw node codec when modifying data under the chunker threshold, instead of incorrectly forcing everything to dag-pb
    • DagModifier prevents creation of identity CIDs exceeding verifcid.DefaultMaxIdentityDigestSize limit when modifying data, automatically switching to proper cryptographic hash while preserving small identity CIDs
    • DagModifier now supports appending data to a RawNode by automatically converting it into a UnixFS file structure where the original RawNode becomes the first leaf block, fixing previously impossible append operations that would fail with "expected protobuf dag node" errors
  • mfs:
    • Files with identity CIDs now properly inherit full CID prefix from parent directories (version, codec, hash type, length), not just hash type (#1018)

Security

  • verifcid: Now enforces maximum size limit of 128 bytes for identity CIDs to prevent abuse (#1018, ipfs/specs#512).
    • 🛠 Attempts to read CIDs with identity multihash digests longer than DefaultMaxIdentityDigestSize will now produce ErrDigestTooLarge error.
    • Identity CIDs can inline data directly, and without a size limit, they could embed arbitrary amounts of data. Limiting the size also protects gateways from poorly written clients that might send absurdly big data to the gateway encoded as identity CIDs only to retrieve it back. Note that identity CIDs do not provide integrity verification, making them vulnerable to bit flips. They should only be used in controlled contexts like raw leaves of a larger DAG. The limit is explicitly defined as DefaultMaxIdentityDigestSize (128 bytes).

Full Changelog: v0.34.0...v0.35.0

v0.34.0

Choose a tag to compare

@github-actions github-actions released this 15 Aug 21:25
d96d57d

What's Changed

Added

  • autoconf: Client library for fetching, caching and expanding IPFS network configurations using "auto" placeholders
  • gateway: Added configurable limits for gateway resource protection:
    • Config.RetrievalTimeout: Maximum duration between writes of non-empty data to HTTP response body (default: 30s). Returns 504 Gateway Timeout when gateway cannot retrieve content within this period.
    • Config.MaxConcurrentRequests: Limits concurrent HTTP requests (default: 4096, suitable for most deployments). Returns 429 Too Many Requests with 60s Retry-After header when exceeded. To restore previous unlimited behavior set both RetrievalTimeout and MaxConcurrentRequests to 0.

      [!IMPORTANT]
      If your gateway returns many HTTP 429 responses while having available resources (CPU, memory), increase MaxConcurrentRequests. If experiencing high load or resource exhaustion, decrease it. See the MaxConcurrentRequests godoc for detailed tuning guidance.

    • Config.MetricsRegistry: Optional Prometheus registry for metrics isolation. When nil, uses the default global registry. Useful for testing and deployments with multiple gateway instances.
    • New middleware with Prometheus metrics:
      • ipfs_http_gw_concurrent_requests: Gauge tracking number of concurrent requests
      • ipfs_http_gw_responses_total{code}: Counter for all HTTP responses by status code
      • ipfs_http_gw_retrieval_timeouts_total{code,truncated}: Counter for retrieval timeout events with details on truncation
  • namesys/IPNSPublisher: option to PublishOptions that allows for setting a custom sequence number for the IPNS record with proper validation to prevent unintentional replay attacks. #962
  • blockstore: Added ValidatingBlockstore wrapper. This replaces the HashOnRead blockstore API.

Changed

  • bitswap/network: The connection event manager now has a SetListeners method. Both bsnet and httpnet now have options to provide the ConnectionEventManager during New(...). This allows sharing the connection event manager when using both. The connection manager SHOULD be shared when using both networks with the network.Router utility.
  • provider: Distribute the responsability of providing new blocks to the places that play a role in the different providing strategies #976. Refactor the logic to perform Provides, when the component has been given a provider:
    • Remove providing.Exchange
    • Provide directly from Blockstore when provider is set (via Option).
    • Provide directly from pinner/merkledag on dag traversal when provider is set (via Option).
    • Provide from MFS whenever there is a call to DAGService.Add and provider is set (via constructor param).
  • upgrade to go-libp2p v0.43.0
  • replace uber-go/multierr with errors.Join #996
  • updated Go in go.mod to 1.24.0 #999

Removed

  • blockstore: Removed HashOnRead API. This is a potentially BREAKING CHANGE for any users of the HashOnRead API. Use the ValidatingBlocksore instead.
    • The HashOnRead function was also removed from Filestore. To use this functionality, provide a ValidatingBlockstore when creating a new Filestore.

Full Changelog: v0.33.1...v0.34.0

This release was brought to you by the Shipyard team.