lightvector/KataGo v1.18.1

Better Benchmark/Genconfig, Minor bugfixes

on GitHub

This is a bugfix and minor improvement release on top of v1.18.0, including making benchmark and genconfig help test and recommend additional performance optimizations that might not be obvious for casual users, beyond the major ones that v1.18.0 added.

For the moment, the newest and strongest transformer neural nets can be downloaded from the v1.17.1 release page, but within a few days of this release KataGo's main training run at https://katagotraining.org/ will switch to transformers and those nets will be available there as well like all the others.
Also, for 9x9 boards or for boards larger than 19x19, see https://katagotraining.org/extra_networks/ for networks specially trained for those sizes!

KataGo is continuing to improve at https://katagotraining.org/ and if you'd like to donate your spare GPU cycles and support it, it could use your help there!

Getting Started / Choosing a Backend

If you're a new user, this section has tips for getting started and basic usage! For choosing a backend:

NVIDIA GPU

Use CUDA+CUDNN or TensorRT. Both are decent, either may be better performance. For transformers, CUDA is typically a bit faster on good GPUs (new in 1.18.x - recent optimizations have let it overtake TensorRT).

• You'll also have to install CUDA and one of CUDNN or TensorRT from nvidia depending on your choice.
• CUDA+CUDNN
  • Faster startup times than TensorRT, good performance, and v1.18.x makes it much faster for transformer models on many GPUs.
  • Use CUDNN >= 9.8.0. The CUDNN 8.9.7 builds will be a LOT slower when running transformers - consider upgrading if you're still on old CUDNN. We're offering CUDNN 8.9.7 builds only for continuity with prior releases.
• TensorRT
  • Longer startup time. May be faster on older convolutional nets still, but tends to be slower than CUDA+CUDNN on transformers on recent versions and strong GPUs.
  • TensorRT versions older than 10 are not supported.

AMD GPU

New in v1.18.x - try the new ROCm backend, which should be much faster than OpenCL.

• The Linux build requires installing ROCm on your own, builds are offered for ROCm 7.2.4 and ROCm 7.14.0.
• The Windows build does NOT require installing ROCm (you need a reasonably recent AMD (Adrenalin) driver installed), and bundles everything it needs from ROCm 7.13. Download the package for your GPU family:
  • gfx103X - Radeon RX 6000 series (RDNA2)
  • gfx110X - Radeon RX 7000 series (RDNA3), including RDNA3 APUs such as the Radeon 780M
  • gfx1151 - Ryzen AI Max ("Strix Halo") APUs
  • gfx120X - Radeon RX 9000 series (RDNA4)
• The windows builds for ROCm are a bit experimental and were cross-compiled from a machine that doesn't actually have an AMD GPU for testing it - please let me know if there are any issues with them!
• The DLLs provided and the rocblas and hipblaslt folders must stay next to katago.exe.
  • The exe is actually the same across these packages and is compiled for more GPUs than these, only the bundled AMD libraries differ. If your GPU is not in the list, for example an RX 5000 series (RDNA1) GPU or a Ryzen AI 300 series APU (gfx1150/gfx1152), it may still work: download the ROCm 7.13 Windows tarball for your GPU family from https://repo.amd.com/rocm/tarball-multi-arch/ (a large download), then copy the DLLs from its bin folder over KataGo's and replace KataGo's rocblas and hipblaslt folders with the ones from there. Copy them into KataGo's own directory rather than relying on PATH, since AMD driver installation can itself leave conflicting copies of some of these DLLs on the system.
  • The first run with a given neural net and board size can take 45 seconds or more before anything seems to happen, while MIOpen tunes its convolution kernels for your GPU - KataGo is not hung. The results are cached under %USERPROFILE%\.miopen\, so later startups are fast.

Intel GPU / NPU

New in v1.18.x - try the new ONNX Runtime backend with the OpenVINO execution provider (the onnx-openvino package).

• You must set onnxProvider = openvino in your config.
• Both the Windows and Linux builds are self-contained (ONNX Runtime and the OpenVINO runtime are bundled, no OpenVINO install needed), so beyond downloading them you only need the Intel drivers for your hardware:
  • Intel GPU: on Windows, the normal Intel graphics driver. On Linux, the GPU compute driver (e.g. the intel-opencl-icd package on Ubuntu).
  • Intel NPU: on Windows, the Intel NPU driver, which also comes via Windows Update (Windows 11 only). On Linux, the Intel NPU driver. Also set onnxOpenVINODeviceType = NPU in your config, since the default device is GPU.
• OpenVINO recompiles the model at every startup. To cache the compiled model between runs, set onnxOpenVINOCacheDir to a directory in your config.
• Windows also has a separate onnx-directml package, which runs on any DirectX 12 GPU (Intel, AMD, or NVIDIA) if you set onnxProvider = directml. It is self-contained too, and mostly worth trying if the native backends and OpenCL do not work for you.
• See Compiling.md for the status of the other execution providers it supports.
• ONNX is not currently enabled for contribute on this release, due to some concern about the large surface area of possibly flaky execution providers and newness of backend, but it should work fine for all other KataGo usage.

Other / Old GPUs

If you have some other GPU, or older GPUs on which the above doesn't work, or don't want to install extra stuff, try OpenCL. OpenCL will often work in cases others don't support, but be much slower, particularly for transformer models.

MacOS

For MacOS, use the Metal backend, which you can generally get by installing KataGo from homebrew, which usually updates not too long after KataGo's own release.

CPU

If you need a pure-CPU version of KataGo, use Eigen AVX2. It will be quite slow compared to GPU. If somehow you're on an ancient CPU as well and Eigen AVX2 doesn't work, you can try Eigen, which will be even slower.

Other notes

• +bs50 - these are just for fun, and don't support distributed training but DO support board sizes up to 50x50. They may also be slightly slower and will use much more memory, even when only playing on 19x19, so use them only when you really want to try large boards.

• Linux executables were compiled on a 22.04 Ubuntu machine using AppImage. You will still need to install e.g. correct versions of Cuda/TensorRT or have drivers for OpenCL, etc. on your own. Compiling from source is also not so hard on Linux, see the "TLDR" instructions for Linux here.

Changes in v1.18.1

• The benchmark command now also tests a couple of other settings and recommends the config changes for them if they are faster:
  • Testing a max batch size of half the number of search threads, which splits batches more evenly and be better on many backends or GPUs.
  • Testing 2 NN server threads per GPU (CUDA and ROCm only, the main backends where this is known to often be a speedup). As mentioned in the v1.18.0 release notes, this is now often the best-performing setup on these backends, and now the benchmark tries it for you and if it's good will indicate how to configure it.
  • New command line flags -no-server-thread-test and -no-half-batch-size-test skip these extra tests.
  • The genconfig command runs the same extra tests and automatically writes any faster settings into the config it generates for you.
• Fixed a bug on the CUDA backend where every NN server thread would initialize and allocate some memory on GPU 0, even when KataGo was configured to use only other GPUs.
• If an NN server thread dies with an unrecoverable error, such as running out of GPU memory, KataGo now logs and prints the error before exiting, rather than possibly aborting with no message at all on some platforms.
• Fixed a possible invalid memory access on the CUDA and ROCm backends when shutting down after a GPU error partway through a neural net evaluation.
• Added slightly better error checking and tests for threaded queries to GPUs.