Kubo v0.38.0 is here, simplifying content announcement, introducing an experimental sweeping DHT provider for large-scale operations, and rolling out significant UI and performance improvements.
This release upgrades the repository from version 17 to 18. The `Provider` and `Reprovider` configuration sections have been merged into a single, streamlined `Provide` section. Your existing settings will be migrated automatically when you run `ipfs daemon --migrate` or manually with `ipfs repo migrate --to=18`.
A new experimental DHT provider is now available, designed for nodes managing hundreds of thousands of CIDs. It offers a more efficient alternative to both the default provider and the resource-intensive accelerated DHT client.
Please note that this feature is experimental. Some commands like `ipfs stats provide` are not yet compatible with sweep mode.
Gateway error pages are now more helpful. When content fails to load with a 504 Gateway Timeout, the error page will display diagnostic information, including:
The WebUI has been updated with a new Diagnostics screen for real-time troubleshooting. You can now adjust log levels for different subsystems without restarting your node and check content retrievability directly from the Files screen. Other improvements include viewing agent versions on the Peers page and new file sorting options.
<a href="https://ipshipyard.com/"><img align="right" src="https://github.com/user-attachments/assets/39ed3504-bb71-47f6-9bf8-cb9a1698f272" /></a> > [!NOTE] > This release was brought to you by the [Shipyard](https://ipshipyard.com/) team. [<img align="right" width="256px" src="https://github.com/user-attachments/assets/cdad8b96-3604-4145-94f6-7f907a59136a" />](https://github.com/user-attachments/assets/cdad8b96-3604-4145-94f6-7f907a59136a) - [Overview](#overview) - [🔦 Highlights](#-highlights) - [🚀 Repository migration: simplified provide configuration](#-repository-migration-simplified-provide-configuration) - [🧹 Experimental Sweeping DHT Provider](#-experimental-sweeping-dht-provider) - [📊 Exposed DHT metrics](#-exposed-dht-metrics) - [🚨 Improved gateway error pages with diagnostic tools](#-improved-gateway-error-pages-with-diagnostic-tools) - [🎨 Updated WebUI](#-updated-webui) - [📌 Pin name improvements](#-pin-name-improvements) - [🛠️ Identity CID size enforcement and `ipfs files write` fixes](#️-identity-cid-size-enforcement-and-ipfs-files-write-fixes) - [📤 Provide Filestore and Urlstore blocks on write](#-provide-filestore-and-urlstore-blocks-on-write) - [🚦 MFS operation limit for --flush=false](#-mfs-operation-limit-for---flush=false) - [📦️ Important dependency updates](#-important-dependency-updates) - [📝 Changelog](#-changelog) - [👨👩👧👦 Contributors](#-contributors) ### Overview Kubo 0.38.0 simplifies content announcement configuration, introduces an experimental sweeping DHT provider for efficient large-scale operations, and includes various performance improvements. ### 🔦 Highlights #### 🚀 Repository migration: simplified provide configuration This release migrates the repository from version 17 to version 18, simplifying how you configure content announcements. The old `Provider` and `Reprovider` sections are now combined into a single [`Provide`](https://github.com/ipfs/kubo/blob/master/docs/config.md#provide) section. Your existing settings are automatically migrated - no manual changes needed. **Migration happens automatically** when you run `ipfs daemon --migrate`. For manual migration: `ipfs repo migrate --to=18`. Read more about the new system below. #### 🧹 Experimental Sweeping DHT Provider A new experimental DHT provider is available as an alternative to both the default provider and the resource-intensive [accelerated DHT client](https://github.com/ipfs/kubo/blob/master/docs/config.md#routingaccelerateddhtclient). Enable it via [`Provide.DHT.SweepEnabled`](https://github.com/ipfs/kubo/blob/master/docs/config.md#providedhtsweepenabled). **How it works:** Instead of providing keys one-by-one, the sweep provider systematically explores DHT keyspace regions in batches. > <picture> > <source media="(prefers-color-scheme: dark)" srcset="https://github.com/user-attachments/assets/f6e06b08-7fee-490c-a681-1bf440e16e27"> > <source media="(prefers-color-scheme: light)" srcset="https://github.com/user-attachments/assets/e1662d7c-f1be-4275-a9ed-f2752fcdcabe"> > <img alt="Reprovide Cycle Comparison" src="https://github.com/user-attachments/assets/e1662d7c-f1be-4275-a9ed-f2752fcdcabe"> > </picture> > > The diagram shows how sweep mode avoids the hourly traffic spikes of Accelerated DHT while maintaining similar effectiveness. By grouping CIDs into keyspace regions and processing them in batches, sweep mode reduces memory overhead and creates predictable network patterns. **Benefits for large-scale operations:** Handles hundreds of thousands of CIDs with reduced memory and network connections, spreads operations evenly to eliminate resource spikes, maintains state across restarts through persistent keystore, and provides better metrics visibility. **Monitoring and debugging:** Legacy mode (`SweepEnabled=false`) tracks `provider_reprovider_provide_count` and `provider_reprovider_reprovide_count`, while sweep mode (`SweepEnabled=true`) tracks `total_provide_count_total`. Enable debug logging with `GOLOG_LOG_LEVEL=error,provider=debug,dht/provider=debug` to see detailed logs from either system. > [!NOTE] > This feature is experimental and opt-in. In the future, it will become the default and replace the legacy system. Some commands like `ipfs stats provide` and `ipfs routing provide` are not yet available with sweep mode. Run `ipfs provide --help` for alternatives. For configuration details, see [`Provide.DHT`](https://github.com/ipfs/kubo/blob/master/docs/config.md#providedht). For metrics documentation, see [Provide metrics](https://github.com/ipfs/kubo/blob/master/docs/metrics.md#provide). #### 📊 Exposed DHT metrics Kubo now exposes DHT metrics from go-libp2p-kad-dht, including `total_provide_count_total` for sweep provider operations and RPC metrics prefixed with `rpc_inbound_` and `rpc_outbound_` for DHT message traffic. See [Kubo metrics documentation](https://github.com/ipfs/kubo/blob/master/docs/metrics.md) for details. #### 🚨 Improved gateway error pages with diagnostic tools Gateway error pages now provide more actionable information during content retrieval failures. When a 504 Gateway Timeout occurs, users see detailed retrieval state information including which phase failed and a sample of providers that were attempted: >  > > - **[`Gateway.DiagnosticServiceURL`](https://github.com/ipfs/kubo/blob/master/docs/config.md#gatewaydiagnosticserviceurl)** (default: `https://check.ipfs.network`): Configures the diagnostic service URL. When set, 504 errors show a "Check CID retrievability" button that links to this service with `?cid=<failed-cid>` for external diagnostics. Set to empty string to disable. > - **Enhanced error details**: Timeout errors now display the retrieval phase where failure occurred (e.g., "connecting to providers", "fetching data") and up to 3 peer IDs that were attempted but couldn't deliver the content, making it easier to diagnose network or provider issues. > - **Retry button on all error pages**: Every gateway error page now includes a retry button for quick page refresh without manual URL re-entry. #### 🎨 Updated WebUI The Web UI has been updated to [v4.9](https://github.com/ipfs/ipfs-webui/releases/tag/v4.9.0) with a new **Diagnostics** screen for troubleshooting and system monitoring. Access it at `http://127.0.0.1:5001/webui` when running your local IPFS node. | Diagnostics: Logs | Files: Check Retrieval | Diagnostics: Retrieval Results | |:---:|:---:|:---:| |  |  |  | | Debug issues in real-time by adjusting [log level](https://github.com/ipfs/kubo/blob/master/docs/environment-variables.md#golog_log_level) without restart (global or per-subsystem like bitswap) | Check if content is available to other peers directly from Files screen | Find out why content won't load or who is providing it to the network | | Peers: Agent Versions | Files: Custom Sorting | |:---:|:---:| |  |  | | Know what software peers run | Find files faster with new sorting | Additional improvements include a close button in the file viewer, better error handling, and fixed navigation highlighting. #### 📌 Pin name improvements `ipfs pin ls <cid> --names` now correctly returns pin names for specific CIDs ([#10649](https://github.com/ipfs/kubo/issues/10649), [boxo#1035](https://github.com/ipfs/boxo/pull/1035)), RPC no longer incorrectly returns names from other pins ([#10966](https://github.com/ipfs/kubo/pull/10966)), and pin names are now limited to 255 bytes for better cross-platform compatibility ([#10981](https://github.com/ipfs/kubo/pull/10981)). #### 🛠️ Identity CID size enforcement and `ipfs files write` fixes **Identity CID size limits are now enforced** Identity CIDs use [multihash `0x00`](https://github.com/multiformats/multicodec/blob/master/table.csv#L2) to embed data directly in the CID without hashing. This experimental optimization was designed for tiny data where a CID reference would be larger than the data itself, but without size limits it was easy to misuse and could turn into an anti-pattern that wastes resources and enables abuse. This release enforces a maximum of 128 bytes for identity CIDs - attempting to exceed this limit will return a clear error message. - `ipfs add --inline-limit` and `--hash=identity` now enforce the 128-byte maximum (error when exceeded) - `ipfs files write` prevents creation of oversized identity CIDs **Multiple `ipfs files write` bugs have been fixed** This release resolves several long-standing MFS issues: raw nodes now preserve their codec instead of being forced to dag-pb, append operations on raw nodes work correctly by converting to UnixFS when needed, and identity CIDs properly inherit the full CID prefix from parent directories. #### 📤 Provide Filestore and Urlstore blocks on write Improvements to the providing system in the last release (provide blocks according to the configured [Strategy](https://github.com/ipfs/kubo/blob/master/docs/config.md#providestrategy)) left out [Filestore](https://github.com/ipfs/kubo/blob/master/docs/experimental-features.md#ipfs-filestore) and [Urlstore](https://github.com/ipfs/kubo/blob/master/docs/experimental-features.md#ipfs-urlstore) blocks when the "all" strategy was used. They would only be reprovided but not provided on write. This is now fixed, and both Filestore blocks (local file references) and Urlstore blocks (HTTP/HTTPS URL references) will be provided correctly shortly after initial add. #### 🚦 MFS operation limit for --flush=false The new [`Internal.MFSNoFlushLimit`](https://github.com/ipfs/kubo/blob/master/docs/config.md#internalmfsnoflushlimit) configuration option prevents unbounded memory growth when using `--flush=false` with `ipfs files` commands. After performing the configured number of operations without flushing (default: 256), further operations will fail with a clear error message instructing users to flush manually. ### 📦️ Important dependency updates - update `boxo` to [v0.35.0](https://github.com/ipfs/boxo/releases/tag/v0.35.0) - update `go-libp2p-kad-dht` to [v0.35.0](https://github.com/libp2p/go-libp2p-kad-dht/releases/tag/v0.35.0) - update `ipfs-webui` to [v4.9.1](https://github.com/ipfs/ipfs-webui/releases/tag/v4.9.1) (incl. [v4.9.0](https://github.com/ipfs/ipfs-webui/releases/tag/v4.9.0)) ### 📝 Changelog <details><summary>Full Changelog</summary> </details> ### 👨👩👧👦 Contributors