github ipfs/kubo v0.4.19
Release 0.4.19
on GitHub

latest releases: v0.32.0, v0.32.0-rc2, v0.32.0-rc1...
5 years ago

We're happy to announce go 0.4.19. This release contains a bunch of important fixes and a slew of new and improved features. Get pumped and upgrade ASAP to benefit from all the new goodies! 🎁

Features

🔌 Initializing With Random Ports

Go-ipfs can now be configured to listen on a random but stable port (across restarts) using the new randomports configuration profile. This should be helpful when testing and/or running multiple go-ipfs instances on a single machine.

To initialize a go-ipfs instance with a randomly chosen port, run: 

> ipfs init --profile=randomports

👂 Gateway Directory Listing

IPNS (and/or DNSLink) directory listings on the gateway, e.g. https://ipfs.io/ipns/dist.ipfs.io/go-ipfs/, will now display the ipfs hash of the current directory. This way users can more easily create permanent links to otherwise mutable data.

📡 AutoRelay and AutoNAT

This release introduces two new experimental features (courtesy of libp2p): AutoRelay and AutoNAT.

AutoRelay is a new service that automatically chooses a public relay when it detects that the go-ipfs node is behind a NAT. While relaying connections through a third-party node isn't the most efficient way to route around NATs, it's a reliable fallback.

To enable AutoRelay, set the Swarm.EnableAutoRelay option in the config.

AutoNAT is the service AutoRelay uses to detect if the node is behind a NAT. You don't have to set any special config flags to enable it.

In this same config section, you may also notice options like EnableRelayHop, EnableAutoNATService, etc. You do not need to enable these:

  • EnableRelayHop -- Allow other nodes to use your node as a relay (disabled by default).
  • EnableAutoNATService -- Help other nodes detect if they're behind a NAT (disabled by default).

📵 Offline Operation

There are two new "offline" features in this release: a global --offline flag and an option to configure the gateway to not fetch files.

Most go-ipfs commands now support the --offline flag. This causes IPFS to avoid network operations when performing the requested operation. If you've ever used the --local flag, the --offline flag is the (almost) universally supported replacement.

For example:

  • If the daemon is started with ipfs daemon --offline, it won't even connect to the network. (note: this feature isn't new, just an example).
  • ipfs add --offline some_file won't send out provider records.
  • ipfs cat --offline Qm... won't fetch any blocks from the network.
  • ipfs block stat --offline Qm... is a great way to tell if a block is locally available.

Note: It doesn't yet work with the refs, urlstore, or tar commands (#6002).

On to the gateway, there's a new Gateway.NoFetch option to configure the gateway to only serve locally present files. This makes it possible to run an IPFS node as a gateway to serve content of your choosing without acting like a public proxy. 🤫

📍 Adding And Pinning Content

There's a new --pin flag for both ipfs block put and ipfs urlstore add to match the --pin flag in ipfs add. This allows one to atomically add and pin content with these APIs.

NOTE 1: For ipfs urlstore add, --pin has been enabled by default to match the behavior in ipfs add. However, ipfs block put does not pin by default to match the current behavior.

NOTE 2: If you had previously used the urlstore and weren't explicitly pinning content after adding it, it isn't pinned and running the garbage collector will delete it. While technically documented in the ipfs urlstore add helptext, this behavior was non-obvious and bears mentioning.

🗂 File Listing

The ipfs ls command has two significant changes this release: it reports file sizes instead of dag sizes and has gained a new --stream flag.

First up, ipfs ls now reports file sizes instead of dag sizes. Previously, for historical reasons, ipfs ls would report the size of a file/directory as seen by IPFS including all the filesystem datastructures and metadata. However, this meant that ls -l and ipfs ls would print different sizes: 

> ipfs ls /ipfs/QmS4ustL54uo8FzR9455qaxZwuMiUhyvMcX9Ba8nUH4uVv

QmZTR5bcpQD7cFgTorqxZDYaew1Wqgfbd2ud9QqGPAkK2V 1688 about
QmYCvbfNbCwFR45HiNP45rwJgvatpiW38D961L5qAhUM5Y 200  contact
QmY5heUM5qgRubMDD1og9fhCPA6QdkMp3QCwd4s7gJsyE7 322  help
QmejvEPop4D7YUadeGqYWmZxHhLc4JBUCzJJHWMzdcMe2y 12   ping
QmXgqKTbzdh83pQtKFb19SpMCpDDcKR2ujqk3pKph9aCNF 1692 quick-start
QmPZ9gcCEpqKTo6aq61g2nXGUhM4iCL3ewB6LDXZCtioEB 1102 readme
QmQ5vhrL7uv6tuoN9KeVBwd4PwfQkXdVVmDLUZuTNxqgvm 1173 security-notes

> ipfs get /ipfs/QmS4ustL54uo8FzR9455qaxZwuMiUhyvMcX9Ba8nUH4uVv
Saving file(s) to QmS4ustL54uo8FzR9455qaxZwuMiUhyvMcX9Ba8nUH4uVv
 6.39 KiB / 6.39 KiB [================================] 100.00% 0s

> ls -l QmS4ustL54uo8FzR9455qaxZwuMiUhyvMcX9Ba8nUH4uVv
total 28
-rw------- 1 user group 1677 Feb 14 17:03 about
-rw------- 1 user group  189 Feb 14 17:03 contact
-rw------- 1 user group  311 Feb 14 17:03 help
-rw------- 1 user group    4 Feb 14 17:03 ping
-rw------- 1 user group 1681 Feb 14 17:03 quick-start
-rw------- 1 user group 1091 Feb 14 17:03 readme
-rw------- 1 user group 1162 Feb 14 17:03 security-notes

This is now no longer the case. ipfs ls and ls -l now return the same sizes. 🙌

Second up, ipfs ls now has a new --stream flag. In IPFS, very large directories (e.g., Wikipedia) are split up into multiple chunks (shards) as there are too many entries to fit in a single block. Unfortunately, ipfs ls buffers the entire file list in memory and then sorts it. This means that ipfs ls /ipfs/QmXoypizjW3WknFiJnKLwHCnL72vedxjQkDDP1mXWo6uco/wiki (wikipedia) will take a very long time to return anything (it'll also use quite a bit of memory).

However, the new --stream flag makes it possible to stream a directory listing as new chunks are fetched from the network. To test this, you can run ipfs ls --stream --size=false --resolve-type=false /ipfs/QmXoypizjW3WknFiJnKLwHCnL72vedxjQkDDP1mXWo6uco/wiki. You probably won't want to wait for that command to finish, Wikipedia has a lot of entries. 😉

🔁 HTTP Proxy

This release sees a new (experimental) feature contributed by our friends at Peergos: HTTP proxy over libp2p. When enabled, the local gateway can act as an HTTP proxy and forward HTTP requests to libp2p peers. When combined with the ipfs p2p command, users can use this to expose HTTP services to other go-ipfs nodes via their gateways. For details, check out the documentation.

Performance And Reliability

This release introduces quite a few performance/reliability improvements and, as usual, fixes several memory leaks. Below is a non-exhaustive list of noticeable changes.

📞 DHT

This release includes an important DHT fix that should significantly:

  1. Reduce dialing.
  2. Speed up DHT queries.
  3. Improve performance of the gateways.

Basically, in the worst case, a DHT query would turn into a random walk of the entire IPFS network. Yikes!

Relevant PR: libp2p/go-libp2p-kad-dht#237

🕸 Bitswap

Bitswap sessions have improved and are now used for all requests. Sessions allow us to group related content and ask peers most likely to have the content instead of broadcasting the request to all connected peers. This gives us two significant benefits:

  1. Less wasted upload bandwidth. Instead of broadcasting which blocks we want to everyone, we can ask fewer peers thus reducing the number of requests we send out.
  2. Less wasted download bandwidth. Because we know which peers likely have content, we can ask an individual peer for a block and expect to get an answer. In the past, we'd ask every peer at the same time to optimize for latency at the expense of bandwidth (getting the same block from multiple peers). We had to do this because we had to assume that most peers didn't have the requested block.

‼️ Pubsub

This release includes some significant reliability improvements in pubsub subscription handling. If you've previously had issues with connected pubsub peers not seeing each-other's messages, please upgrade ASAP.

♻️ Reuseport

In this release, we've rewritten our previously error-prone go-reuseport library to not duplicate a significant portion of Go's low-level networking code. This was made possible by Go's new Control net.Dialer option.

In the past, our first suggestion to anyone experiencing weird resource or connectivity issues was to disable REUSEPORT (set IPFS_REUSEPORT to false). This should no longer be necessary.

🐺 Badger Datastore

Badger has reached 1.0. This release brings an audit and numerous reliability fixes. We are now reasonably confident that badger will become the default datastore in a future release. 👍

This release also adds a new Truncate configuration option for the badger datastore (enabled by default for new IPFS nodes). When enabled, badger will delete any un-synced data on start instead of simply refusing to start. This should be safe on all filesystems where the sync operation is safe and removes the need for manual intervention when restarting an IPFS node after a crash.

Assuming you initialized your badger repo with ipfs init --profile=badgerds, you can enable truncate on an existing repo by running: ipfs config --json "Datastore.Spec.child.truncate" true.

Refactors and Endeavors

🕹 Commands Library

The legacy commands library shim has now been completely removed. This won't mean much for many users but the go-ipfs team is happy to have this behind them.

🌐 Base32 CIDs

This release can now encode CIDs in responses in bases other than base58. This is primarily useful for web-browser integration as it allows us to (a) encode CIDs in a lower-case base (e.g., base32) and then use them in the origin part of URLs. The take away is: this release brings us a step closer to better browser integration.

Specifically, this release adds two flags:

  1. --cid-base: When specified, the IPFS CLI will encode all CIDv1 CIDs using the requested base.
  2. --upgrade-cidv0-in-output: When specified, the IPFS CLI will upgrade CIDv0 CIDs to CIDv1 CIDs when returning them to the user. This upgrade is necessary because CIDv0 doesn't support multibase however, it's off by default as it changes the binary representation of the CIDs (which could have unintended consequences).

🎛 CoreAPI

The work on the CoreAPI refactor (ipfs/go-ipfs#4498) has progressed leaps and bounds this release. The CoreAPI is a comprehensive programmatic interface designed to allow go-ipfs be used as a daemon or a library interchangeably.

As of this release, go-ipfs now has:

  • External interface definitions in ipfs/interface-go-ipfs-core.
  • A work-in-progress implementation (ipfs/go-ipfs-http-client) of these interfaces that uses the IPFS HTTP API. This will replace the (ipfs/go-ipfs-api) library.
  • A new plugin type "Daemon". Daemon plugins are started and stopped along with the go-ipfs daemon and are instantiated with a copy of the CoreAPI. This allows them to control and extend the go-ipfs daemon from within the daemon itself.

The next steps are:

  1. Finishing the remaining API surface area. At the moment, the two key missing parts are:
  2. Config manipulation.
  3. The ipfs files API.
  4. Finalizing the ipfs/go-ipfs-http-client implementation.
  5. Creating a simple way to construct and initialize a go-ipfs node when using go-ipfs as a library.

Changelogs

Check out latest releases or
releases around ipfs/kubo v0.4.19

Don't miss a new kubo release

NewReleases is sending notifications on new releases.

Get notifications