- π¬ Discuss
- π¦ Highlights
- Mplex deprecation
- Gateway: meaningful CAR responses on Not Found errors
- Gateway: added
Gateway.DisableHTMLErrors
configuration option - Binary characters in file names: no longer works with old clients and new Kubo servers
- Self-hosting
/routing/v1
endpoint for delegated routing needs - Trustless Gateway Over Libp2p Experiment
- Removal of
/quic
(Draft 29) support - Better Caching of multiaddresses for providers in DHT servers
- Fixed FUSE multiblock structures
- π Changelog
- π¨βπ©βπ§βπ¦ Contributors
Overview
π¦ Highlights
Mplex deprecation
Mplex is being deprecated, this is because it is unreliable and
randomly drop streams when sending data too fast.
New pieces of code rely on backpressure, that means the stream will dynamicaly
slow down the sending rate if data is getting backed up.
Backpressure is provided by Yamux and QUIC.
In case you need compatibility with older implementations that do not ship with
Yamux (like default's JS-IPFS) you can turned it back ON in the config with:
$ ipfs config --json Swarm.Transports.Multiplexers.Mplex 200
We will completely remove Mplex in v0.24 as it makes protocols very bad to implement,
if you are in this situation you need to add yamux support to your other implementation.
Gateway: meaningful CAR responses on Not Found errors
When requesting a CAR from the gateway, the root of the CAR might no longer be
meaningful. By default, the CAR root will be the last resolvable segment of the
path. However, in situations where the path cannot be resolved, such as when
the path does not exist, a CAR will be sent with a root of bafkqaaa
(empty CID).
This CAR will contain all blocks necessary to validate that the path does not
exist without having to trust the gateway.
Gateway: added Gateway.DisableHTMLErrors
configuration option
The Gateway.DisableHTMLErrors
configuration option forces errors to be
displayed in browsers as plain text (text/plain
) rather than HTML error
pages. It's especially beneficial for whitelabel or middleware deployments that
wish to avoid IPFS branding and links on error pages in browsers.
Binary characters in file names: no longer works with old clients and new Kubo servers
In this version, we updated Kubo to support Go 1.20+. In Go 1.20, a regression
regarding multipart headers was introduced.
This only affects ipfs add
when a file name has binary characters in its name.
As a consequence, we had to update the encoding of the file name headers. This is
the compatibility table:
New Client | Old Client | |
---|---|---|
New Server | β | π‘* |
Old Server | β | β |
*Old clients can only send Unicode file paths to the server.
Self-hosting /routing/v1
endpoint for delegated routing needs
The Routing
system configured in Kubo can be now exposed on the gateway port as a standard
HTTP Routing V1 API endpoint. This allows
self-hosting and experimentation with custom delegated routers. This is disabled by default,
but can be enabled by setting Gateway.ExposeRoutingAPI
to true
.
Trustless Gateway Over Libp2p Experiment
In this update, we've introduced an experimental opt-in feature allowing users to
serve a subset of Trustless Gateway responses,
such as blocks and CARs, over libp2p. This enhancement leverages the ongoing
/http/1.1
specification work in libp2p
to make it easier to support HTTP semantics over libp2p streams.
This development means that if users wish to utilize the Trustless Gateway API
for data transport, they can now do so even in scenarios where standard HTTP
might be problematic, such as when the endpoint is behind a firewall or when
attempting to serve data to a browser without a CA certificate.
See HTTP Gateway over Libp2p for details about this experiment.
Removal of /quic
(Draft 29) support
Kubo no longer supports QUIC Draft 29. This means that older nodes aren't able to connect
to newer nodes using QUIC Draft 29. However, they are still able to connect through any other
transport that both nodes talk (such as QUIC RFC 9000, or TCP). QUIC Draft 29 was a preliminary implementation of QUIC before
the official RFC 9000 was published, and it has now been dropped by go-libp2p
and therefore Kubo.
In Kubo 0.18, we shipped a migration
to have listeners for both /quic
(Draft 29) and /quic-v1
(RFC 9000). Similarly, in this
version we are shipping a migration to remove the current /quic
addresses, maintaining
the /quic-v1
addresses only. For more background information, check issue #9496.
Better Caching of multiaddresses for providers in DHT servers
Thanks to probelab.io's RFM17.1 DHT servers will now cache the addresses of content hosts for the lifetime of the provider record.
This means clients who resolve content from theses servers get a responses which include both peer id and multiaddresses.
In most cases this enables skipping a second query which resolves the peer id to multiaddresses for stable enough peers.
This will improve content fetching lantency in the network overtime as servers updates.
Fixed FUSE multiblock structures
ls
ing directories and reading dag-pb files on a fuse volume have been fixed. #9044
Thx a lot @bmwiedemann for debugging this issue.
π Changelog
Full Changelog
- github.com/ipfs/kubo:
- fix: align systemd unit file with default IPFS installation path (#10163) (ipfs/kubo#10163)
- docs: capitalize headers for consistency
- Merge commit '695bf66674931a138862b6fa2cb0b16dc2f6ddd8' into release-v0.23.0
- chore: update version
- changelog: generalize fuse 9044's entry
- changelog: update fuse 9044's entry
- Update go-unixfsnode to 1.8.0 to fix FUSE
- docs(readme): header improvements (#10144) (ipfs/kubo#10144)
- fix(docker): allow nofuse builds for MacOS (#10135) (ipfs/kubo#10135)
- docs: fix typos
- docs: s/ipfs dht/amino dht/
- changelog: mention probelab RFM17.1 dht improvement
- tests: remove sharness ping tests
- perf: make bootstrap saves O(N)
- chore: update go-libp2p-kad-dht
- chore: webui v4.1.1 (#10120) (ipfs/kubo#10120)
- core/bootstrap: fix panic without backup bootstrap peer functions (#10029) (ipfs/kubo#10029)
- feat: add Gateway.DisableHTMLErrors option (#10137) (ipfs/kubo#10137)
- fix(migrations): use dweb.link (#10133) (ipfs/kubo#10133)
- docs: add changelog info for QUIC Draft 29 (#10132) (ipfs/kubo#10132)
- feat: add gateway to http over libp2p (ipfs/kubo#10108)
- migration: update 14-to-15 to v1.0.1
- chore: update to build with Go 1.21
- refactor: stop using go-libp2p deprecated peer.ID.Pretty
- docs(readonly): fix typo
- docs(changelog): link to relevant IPIP
- fix: hamt traversal in ipld-explorer (webui@4.1.0) (#10025) (ipfs/kubo#10025)
- refactor: if statement (#10105) (ipfs/kubo#10105)
- chore: bump repo version to 15
- docs: remove link to deleted #accelerated-dht-client
- feat(gateway): expose /routing/v1 server (opt-in) (#9877) (ipfs/kubo#9877)
- improve error in fuse node failures
- chore: update boxo, go-libp2p, and internalize mplex (#10095) (ipfs/kubo#10095)
- dockerfile: reorder copy order for better layer caching
- refactor: using error is instead of == (#10093) (ipfs/kubo#10093)
- fix: use %-encoded headers in most compatible way
- fix: open /dev/null with read write permissions
- chore: bump to go 1.20
- docs(readme): new logo and header
- docker: change to releases that follow debian's updates
- docker: bump debian version to bookworm
- chore: restore exec perms for t0116-gateway-cache.sh and fixtures (#10085) (ipfs/kubo#10085)
- fix(gw): useful IPIP-402 CARs on not found errors (#10084) (ipfs/kubo#10084)
- feat: add zsh completions (#10040) (ipfs/kubo#10040)
- style: remove commented imports [skip changelog]
- style: gofumpt and godot [skip changelog] (#10081) (ipfs/kubo#10081)
- chore: bump boxo for verifcid breaking changes
- chore: remove outdated comment (#10077) (ipfs/kubo#10077)
- chore: remove deprecated testground plans
- feat: allow users to optin again into mplex
- feat: remove Mplex
- docs(readme): minimal reqs (#10066) (ipfs/kubo#10066)
- docs: add v0.23.md
- docs: get ready for v0.23
- chore: fix link in v0.22 changelog
- github.com/ipfs/boxo (v0.11.0 -> v0.13.1):
- Release v0.13.1 (ipfs/boxo#469)
- Release v0.13.0 (ipfs/boxo#465)
- Release v0.12 (ipfs/boxo#446)
- github.com/ipfs/go-graphsync (v0.14.4 -> v0.15.1):
- v0.15.1 bump
- fix: partial revert of 1be7c1a20; make traverser process identity CIDs
- v0.15.0 bump
- chore: add identity CID parse tests
- fix: traverser should skip over identity CIDs
- fix(ipld): update ipld deps, only slurp LargeBytesNode when matching
- docs(version): update for v0.14.7
- Handle context cancellation properly (#428) (ipfs/go-graphsync#428)
- chore(version.json): update for v0.14.6
- feat: MaxLinks for requests (#420) (ipfs/go-graphsync#420)
- fix(responsemanager): network disconnect reliability (#425) (ipfs/go-graphsync#425)
- Update version to reflect latest fixes (#424) (ipfs/go-graphsync#424)
- Fix shutdown bug in #412 (#422) (ipfs/go-graphsync#422)
- github.com/ipfs/go-ipfs-cmds (v0.9.0 -> v0.10.0):
- chore: version 0.10.0
- fix: panic when calling .SetLength for writerResponseEmitter
- fix!: client with raw abs path option
- doc: clarify flag inheritance explanation
- ci: uci/copy-templates (ipfs/go-ipfs-cmds#242)
- chore: remove dep on github.com/Kubuxu/go-os-helper
- github.com/ipfs/go-unixfsnode (v1.7.1 -> v1.8.1):
- v1.8.1 bump
- testutil: relax DirEntry usage for non-dag-pb
- v1.8.0 bump
- fix: add cross-impl shard test
- files returned from unixfsnode should be traversable back to their substrate
- fix: better import name
- chore: refactor and add tests with fixtures
- fix: proper tsize encoding in sharded files
- rel 1.7.4
- Provide path for getting sizes on directory iteration (ipfs/go-unixfsnode#60)
- tag 1.7.3 (ipfs/go-unixfsnode#57)
- Fail to construct preload hamt shards when traversal fails (ipfs/go-unixfsnode#55)
- fix: large files support io.SeekCurrent (ipfs/go-unixfsnode#56)
- chore(version): update version number
- feat: add entity matcher w/o preload, add matcher fn for consuming bytes (ipfs/go-unixfsnode#52)
- github.com/ipld/go-ipld-prime (v0.20.0 -> v0.21.0):
- v0.21.0 release
- fix(selectors): document ranges in slice matcher
- fix(selectors): update ipld/ipld submodule with latest fixtures
- fix(selectors): more permissive with slice "from" underflow
- chore: extract simpleBytes to testutil package
- feat(selectors): negative values for slice matcher's From and To
- chore: extract MultiByteNote to testutil package
- feat(test): add matcher/slice selector test cases
- feat: remove hard-error when slice matcher reaches non-string/bytes node
- fix: cache offsets for sequential reads
- feat: add inline union representation to schema parser
- fix: basic.NewInt returns pointer (like others)
- fix(bindnode): listpairs value assembly handles complex reprs
- fix(bindnode): listpairs repr assembler handles AssignNode
- fix(schema): handle parsing of "listpairs" in the DSL
- fix: remove _skipAbsent labels
- fix: make listpairs repr [[k1,v1],[k2,v2]...]
- feat(bindnode): support listpairs struct representation
- fix(windows,test): avoid "already exists" error on codegen tests for Windows
- Make traversal.WalkTransforming() work
- doc: clean up and expand on traversal pkg docs
- doc: add lots of notes about using the preloader and the budget
- doc: expand on preloader docs
- fix: inline initialPhase() logic for clarity
- feat: preload walk using phase state, call preloader once per link
- fix: handle Budget & SeenLinks
- chore: remove BufferedLoader
- fix: recurse preloader at block level
- fix: Context->PreloadContext for clarity and consistency with LinkContext
- fix: replace ioutil.ReadAll
- fix: fix tooling complaints
- feat: add BufferedLoader
- feat(traversal): allow preloading functionality
- fix: address dodgy test case variable capture
- stop using the deprecated io/ioutil package
- stop using the deprecated io/ioutil package
- stop using the deprecated io/ioutil package
- fix: make StartAtPath work properly for matching walks
- github.com/libp2p/go-libp2p (v0.29.2 -> v0.31.0):
- release v0.31.0 (#2543) (libp2p/go-libp2p#2543)
- dashboards: improve naming for black hole panel (#2539) (libp2p/go-libp2p#2539)
- reuseport: use DialContext instead of Dial to fail quickly (#2541) (libp2p/go-libp2p#2541)
- swarm: track dial cancellation reason (#2532) (libp2p/go-libp2p#2532)
- p2p/http: cache json wellknown mappings in the .well-known handler (#2537) (libp2p/go-libp2p#2537)
- feat: Implement HTTP spec (#2438) (libp2p/go-libp2p#2438)
- move libp2p/go-libp2p-gostream to p2p/net/gostream (libp2p/go-libp2p#2535)
- host: disable black hole detection on autonat dialer (#2529) (libp2p/go-libp2p#2529)
- identify: disable racy test when running with race detector (#2526) (libp2p/go-libp2p#2526)
- swarm: return a more meaningful error when dialing QUIC draft-29 (#2524) (libp2p/go-libp2p#2524)
- swarm: fix Unwrap for DialError, implement Unwrap for TransportError (#2437) (libp2p/go-libp2p#2437)
- swarm: return errors on filtered addresses when dialing (#2461) (libp2p/go-libp2p#2461)
- core: add ErrPeerIDMismatch error type to replace ad-hoc errors (#2451) (libp2p/go-libp2p#2451)
- update quic-go to v0.38.1 (#2506) (libp2p/go-libp2p#2506)
- quic: don't claim to be able to dial draft-29 in CanDial (#2520) (libp2p/go-libp2p#2520)
- examples: update go-libp2p to v0.30.0 (#2507) (libp2p/go-libp2p#2507)
- metrics: update dashboard names from libp2p to go-libp2p (#2512) (libp2p/go-libp2p#2512)
- chore: be more descriptive about where public dashboards come from (#2508) (libp2p/go-libp2p#2508)
- release v0.30.0 (#2505) (libp2p/go-libp2p#2505)
- transport tests: add deadline tests (#2286) (libp2p/go-libp2p#2286)
- chore: remove unused and outdated package-list.json (#2499) (libp2p/go-libp2p#2499)
- muxer: remove support for mplex (#2498) (libp2p/go-libp2p#2498)
- transport tests: refactor workers in TestMoreStreamsThanOurLimits (#2472) (libp2p/go-libp2p#2472)
- use standard library sha256 implementation for Go 1.21 (#2309) (libp2p/go-libp2p#2309)
- quic: update quic-go to v0.37.5 (#2497) (libp2p/go-libp2p#2497)
- cleanup: add continue in case of failure in the (*BasicHost).Addrs certhash loop (#2492) (libp2p/go-libp2p#2492)
- tests: add a CertHashes testcase in TestInferWebtransportAddrsFromQuic (#2495) (libp2p/go-libp2p#2495)
- basichost: use byte representation of WebTransport multiaddr as map key (#2494) (libp2p/go-libp2p#2494)
- webtransport: check for UDP multiaddr component in address matcher (#2491) (libp2p/go-libp2p#2491)
- swarm: remove unnecessary reqno for pending request tracking (#2460) (libp2p/go-libp2p#2460)
- quic: drop support for QUIC draft-29 (#2487) (libp2p/go-libp2p#2487)
- metrics: add links to public dashboards (#2486) (libp2p/go-libp2p#2486)
- swarm: remove leftover TODO (#2474) (libp2p/go-libp2p#2474)
- peerstore: deprecate the database-backed peerstore (#2475) (libp2p/go-libp2p#2475)
- identify: fix sorting of observed addresses (#2476) (libp2p/go-libp2p#2476)
- update go-multiaddr to v0.11.0 (#2467) (libp2p/go-libp2p#2467)
- chore: update golang-lru to v2.0.4, fixing semver violation (#2448) (libp2p/go-libp2p#2448)
- swarm: don't open new streams over transient connections (#2450) (libp2p/go-libp2p#2450)
- core/crypto: restrict RSA keys to <= 8192 bits (#2454) (libp2p/go-libp2p#2454)
- chore: add notable project requirement (#2453) (libp2p/go-libp2p#2453)
- examples: update go-libp2p to v0.29.0 (#2432) (libp2p/go-libp2p#2432)
- examples: fix description of command line flags for pubsub (#2400) (libp2p/go-libp2p#2400)
- basichost: remove invalid comment (#2435) (libp2p/go-libp2p#2435)
- github.com/libp2p/go-libp2p-kad-dht (v0.24.2 -> v0.24.4):
- Make v0.24.4 (libp2p/go-libp2p-kad-dht#931)
- github.com/libp2p/go-libp2p-routing-helpers (v0.7.1 -> v0.7.3):
- chore: release v0.7.3
- nit: invert if
- fix: for getValueOrErrorParallel do not return values if they come with errors
- test: add test to make sure we return not found when we get errors back with values
- chore: release v0.7.2
- tracing: do not leak goroutines when the context is canceled
- tracing: allow for reuse of the tracing
- tracing: add tracing to compose parallel's worker
- tests: add more tests
- tests: mark all tests Parallel
- tracing: add highlevel APIs records on the composable routers
- github.com/libp2p/go-reuseport (v0.3.0 -> v0.4.0):
- release v0.4.0 (#111) (libp2p/go-reuseport#111)
- use SO_REUSEPORT_LB on FreeBSD (#106) (libp2p/go-reuseport#106)
- github.com/multiformats/go-multiaddr (v0.10.1 -> v0.11.0):
- release v0.11.0 (#214) (multiformats/go-multiaddr#214)
- update golang.org/x/exp slice comparison to match standard library version (#210) (multiformats/go-multiaddr#210)
- github.com/warpfork/go-testmark (v0.11.0 -> v0.12.1):
- suite: allow disabling file parallelism.
- Suite feature (warpfork/go-testmark#16)
- fix unchecked error in a test
- accept a simplification suggestion from linters
- Trailing whitespace error (warpfork/go-testmark#15)
- FS implementation (#11) (warpfork/go-testmark#11)
- Add a readme for the testexec extension and its conventions. (warpfork/go-testmark#14)
- Strict mode for testexec structure (warpfork/go-testmark#12)
π¨βπ©βπ§βπ¦ Contributors
Contributor | Commits | Lines Β± | Files Changed |
---|---|---|---|
Rod Vagg | 48 | +3578/-1789 | 110 |
Henrique Dias | 24 | +3173/-1128 | 104 |
Jorropo | 51 | +1721/-1297 | 252 |
Marco Munizaga | 6 | +1989/-505 | 39 |
Kay | 3 | +487/-474 | 163 |
hannahhoward | 8 | +626/-136 | 23 |
Calvin Behling | 6 | +496/-259 | 20 |
Eric Myhre | 9 | +610/-121 | 16 |
Adin Schmahmann | 17 | +659/-45 | 35 |
Marten Seemann | 17 | +218/-477 | 119 |
Sukun | 11 | +481/-174 | 29 |
CJB | 1 | +639/-2 | 5 |
Hector Sanjuan | 10 | +450/-127 | 21 |
Wondertan | 2 | +203/-127 | 8 |
Marcin Rataj | 11 | +148/-86 | 18 |
Andrew Gillis | 2 | +163/-14 | 5 |
P. Reis | 3 | +120/-4 | 4 |
Will Scott | 4 | +107/-12 | 6 |
Amir Mohammad Fakhimi | 1 | +97/-2 | 5 |
Ed Schouten | 1 | +55/-7 | 2 |
Icarus9913 | 1 | +30/-30 | 18 |
Dirk McCormick | 1 | +3/-42 | 1 |
RaΓΊl Kripalani | 1 | +20/-18 | 4 |
Michael MurΓ© | 1 | +26/-7 | 5 |
Prem Chaitanya Prathi | 1 | +28/-1 | 2 |
ShengTao | 1 | +13/-14 | 4 |
Prithvi Shahi | 3 | +14/-13 | 3 |
web3-bot | 5 | +12/-10 | 9 |
Alejandro Criado-PΓ©rez | 1 | +11/-11 | 6 |
Steven Allen | 2 | +6/-10 | 2 |
Andrej Manduch | 1 | +5/-5 | 3 |
Russell Dempsey | 2 | +4/-2 | 2 |
Johannes Maria Frank | 1 | +4/-1 | 1 |
downIoads | 1 | +2/-2 | 1 |
Will | 2 | +2/-2 | 2 |
Marin Kirkov | 1 | +2/-2 | 2 |
Gus Eggert | 1 | +2/-2 | 1 |
Bernhard M. Wiedemann | 1 | +4/-0 | 1 |
Dennis Trautwein | 1 | +1/-2 | 1 |
βGheisMohammadiβ | 1 | +1/-1 | 1 |
cce | 1 | +1/-1 | 1 |
Joao Andrade | 1 | +1/-1 | 1 |
guillaumemichel | 1 | +1/-0 | 1 |
Santiago Botto | 1 | +0/-1 | 1 |