cculianu/Fulcrum v1.6.0

Fulcrum 1.6.0

on GitHub

What's new:

This is an upgrade & enhancement release of Fulcrum. It provides a couple of new features, as well as some performance improvements and minor bugfixes. The database format remains unchanged and is both forward and backward compatible with previous versions of Fulcrum.

Summary of changes:

• Faster sync time - (Experimental) Added a new CLI arg (--fast-sync, or conf option fast-sync, default disabled). With this option enabled, Fulcrum will use a large UTXO Cache as it syncs, which should save on DB back-and-forth. The option takes a numeric argument which specifies the maximum RAM to give to the UTXO Cache, in MB. Larger values generally yield better results, but try not to exceed the system's available RAM. PR: #98.
  • In my experiments, a --fast-sync of 2000 on BCH testnet3 yields ~1.7x speed improvement over no --fast-sync. Using 8000 for testnet3 yields a ~2x speedup.
  • Even modest values of 512 for --fast-sync yield benefits on the order of 1.5x or more speedup.
  • Similar results were found for both BCH and BTC mainnet.
  • I haven't tried it with an HDD setup yet, but in theory this option should be particularly beneficial for users using hard disks, since the savings on disk ops is very real with the cache and should be particularly impactful for such setups.
  • Given that the option is still experimental (but perfectly safe to use!), I am making it opt-in for now, and the default behavior of Fulcrum is to sync as it did before, without any sort of UTXO Cache.
  • This option has no effect outside of initial sync, and if the Fulcrum instance is given this option but it is already synched, the option does nothing. This is because a custom UTXO Cache is not really needed outside of initial sync since the regular rocksdb memory table caches give excellent performance results for normal, post-sync patterns of usage.
• Added JSON-RPC batching support. Fulcrum now understands and serves JSON-RPC batches as described in the JSON-RPC 2.0 specification. PR: #97 .
  • This option is controlled with new conf option max_batch, default 345, which controls the maximum size of batches that are accepted. Set to 0 to disable batching (pre-1.6.0 behavior of Fulcrum).
  • Additionally, batches are limited to max_buffer bytes (default: 8000000), in terms of request + response size.
  • Special thanks to: @craigraw for suggesting I add this and for helping me to debug this and for catching at least 1 bug in the initial implementation. Thank you!
• Increased the defaults on some of the resource-related conf options to more generous settings. The previous defaults were a bit on the conservative side, perhaps, and after 2 years of running my servers with higher-than-default settings, I have not experienced any problems, so I decided to make my own server's settings also be the factory defaults.
  • db_max_open_files default is now 40 (was 20)
  • db_mem default is now 512 (was 420)
  • max_buffer default is now 8000000 (was 4000000)
  • workqueue default is now 15000 (was 10000)
  • max_subs_per_ip default is now 75000 (was 50000)
• For Fulcrum nodes running on the BTC blockchain, the block prefetch has been increased to 250 blocks rather than 100 (on account of BTC blocks being bounded in size to at most ~1.7 MB of data).
• Minor performance enhancements in JSON-RPC processing.
• Minor corner-case bug fixes/correctness handling improvments for things like server.banner, Json serialization, and other miscellaneous odds & ends.
• Update of the embedded robin_hood hash table library to v3.11.5,

A Note about OpenSSL 1.1.1

Note to Linux server admins - There is an extant OpenSSL vulnerability in versions of OpenSSL 1.1.1-1.1.1j. If you are using Qt 5.15 with OpenSSL 1.1.1, it is recommended you update your system OpenSSL to 1.1.1k (or later). The Linux static build does not bundle OpenSSL, but instead loads the system OpenSSL at run-time. So please update your system to use OpenSSL 1.1.1k or later!

Windows static builds do embed OpenSSL, however, and they have been updated below with an embedded OpenSSL 1.1.1k.

Linux Ubuntu 16 (Xenial) static builds are compiled to look for the system OpenSSL 1.0.x series, which does not suffer from the above-linked vulnerability.

If unsure which OpenSSL Fulcrum is using, run ./Fulcrum --version and it should tell you which libssl version it found and is using at runtime.

Pre-compiled binaries for Linux

I have provided two pre-built binaries for Linux:

• Fulcrum-1.6.0-x86_64-linux.tar.gz, which is compiled on an Ubuntu 18.04 system using Docker.
• Fulcrum-1.6.0-arm64-linux.tar.gz, which is compiled on an Ubuntu 18.04 system (for ARM64 bit) using Docker.
• Fulcrum-1.6.0-x86_64-linux-ub16.tar.gz, which is compiled on a stock Ubuntu 16.04 system (using Docker) but with g++ 7.3.0 installed from this ppa source: ppa:jonathonf/gcc-7.3 & ppa:jonathonf/gcc

All 3 of the above binaries contain jemalloc, libzmq, Qt5Core and Qt5Network from Qt 5.15.2 statically linked. They still require libz2, libstd++, and the right libc version as dynamic libs on your system (but those are usually present if you are on a recent system).

If the first regular -linux binary fails, try the second one (-linux-ub16), which should work on older systems, hopefully.

Pre-compiled binaries for Windows

• Fulcrum-1.6.0-win64.zip - Pre-built, statically linked Windows version.
  • It should "just work" on any Windows 7 or above 64-bit system.
  • Includes is a statically-linked Fulcrum.exe, built with jemalloc, libzmq, Qt 5.15.2, OpenSSL 1.1.1k, and gcc 7.5.0.
  • Additionally, FulcrumAdmin.exe is included which is the python script, but made into a onefile .exe using PyInstaller.

See the .asc files at the bottom for signatures; my gpg public key can be obtained here: https://github.com/Electron-Cash/keys-n-hashes/blob/master/pubkeys/calinkey.txt

Binary builds for macOS coming soon -- Until then you can always build from source!