juanfont/headscale v0.27.0-beta.1 on GitHub

Minimum supported Tailscale client version: v1.64.0

Database integrity improvements

This release includes a significant database migration that addresses
longstanding issues with the database schema and data integrity that has
accumulated over the years. The migration introduces a schema.sql file as the
source of truth for the expected database schema to ensure new migrations that
will cause divergence does not occur again.

These issues arose from a combination of factors discovered over time: SQLite
foreign keys not being enforced for many early versions, all migrations being
run in one large function until version 0.23.0, and inconsistent use of GORM's
AutoMigrate feature. Moving forward, all new migrations will be explicit SQL
operations rather than relying on GORM AutoMigrate, and foreign keys will be
enforced throughout the migration process.

We are only improving SQLite databases with this change - PostgreSQL databases
are not affected.

Please read the PR description for more technical details about the issues and solutions.

SQLite Database Backup Example:

# Stop headscale
systemctl stop headscale

# Backup sqlite database
cp /var/lib/headscale/db.sqlite /var/lib/headscale/db.sqlite.backup

# Backup sqlite WAL/SHM files (if they exist)
cp /var/lib/headscale/db.sqlite-wal /var/lib/headscale/db.sqlite-wal.backup
cp /var/lib/headscale/db.sqlite-shm /var/lib/headscale/db.sqlite-shm.backup

# Start headscale (migration will run automatically)
systemctl start headscale

DERPMap update frequency

The default DERPMap update frequency has been changed from 24 hours to 3 hours.
If you set the derp.update_frequency configuration option, it is recommended
to change it to 3h to ensure that the headscale instance gets the latest
DERPMap updates when upstream is changed.

Autogroups

This release adds support for the three missing autogroups: self
(experimental), member, and tagged. Please refer to the
documentation for a detailed
explanation.

autogroup:self is marked as experimental and should be used with caution, but
we need help testing it. Experimental here means two things; first, generating
the packet filter from policies that use autogroup:self is very expensive, and
it might perform, or straight up not work on Headscale installations with a
large number of nodes. Second, the implementation might have bugs or edge cases
we are not aware of, meaning that nodes or users might gain more access than
expected. Please report bugs.

Node store (in memory database)

Under the hood, we have added a new datastructure to store nodes in memory. This
datastructure is called NodeStore and aims to reduce the reading and writing
of nodes to the database layer. We have not benchmarked it, but expect it to
improve performance for read heavy workloads. We think of it as, "worst case" we
have moved the bottle neck somewhere else, and "best case" we should see a good
improvement in compute resource usage at the expense of memory usage. We are
quite excited for this change and think it will make it easier for us to improve
the code base over time and make it more correct and efficient.

BREAKING

Remove support for 32-bit binaries
#2692
Policy: Zero or empty destination port is no longer allowed
#2606

Changes

Database schema migration improvements for SQLite
#2617
- IMPORTANT: Backup your SQLite database before upgrading
- Introduces safer table renaming migration strategy
- Addresses longstanding database integrity issues
Add flag to directly manipulate the policy in the database
#2765
DERPmap update frequency default changed from 24h to 3h
#2741
DERPmap update mechanism has been improved with retry, and is now failing
conservatively, preserving the old map upon failure.
#2741
Add support for autogroup:member, autogroup:tagged
#2572
Fix bug where return routes were being removed by policy
#2767
Remove policy v1 code #2600
Refactor Debian/Ubuntu packaging and drop support for Ubuntu 20.04.
#2614
Remove redundant check regarding noise config
#2658
Refactor OpenID Connect documentation
#2625
Don't crash if config file is missing
#2656
Adds /robots.txt endpoint to avoid crawlers
#2643
OIDC: Use group claim from UserInfo
#2663
OIDC: Update user with claims from UserInfo before comparing with allowed
groups, email and domain
#2663
Policy will now reject invalid fields, making it easier to spot spelling
errors #2764
Add FAQ entry on how to recover from an invalid policy in the database
#2776
EXPERIMENTAL: Add support for autogroup:self
#2789
Add healthcheck command #2659

Changelog

0512f7c .github/ISSUE_TEMPLATE: add node number to environment
05996a5 .github/workflow: only run a few selected postgres tests
f6c4b33 .github/workflows: add generate check
5ba7120 .github/workflows: prettier
4a8d2d9 .github/workflows: reduce integration retry to 3
7f8b14f .github/workflows: remove integration retry
e949859 Add DERP docs
bd35fcf Add FAQ entry about policy migration in the database
30d12da Add FAQ entry about the recommended upgrade path
bcd80ee Add debugging and troubleshooting guide
76ca7a2 Add headscale-console
98fc056 Bump version in docs
33e9e7a CLAUDE: split into agents
3f72ee9 Clarify SIGHUP log message (#2661)
51c6367 Correctly document the default for dns.override_local_dns
2f3c365 Describe how to remove a DERP region
49b3468 Do not ignore config-example.yml
c15aa54 Document HEADSCALE_CONFIG
b50e10a Document breaking change for dns.override_local_dns
30cec3a Document ports in use
c04e17d Document valid log levels
cd70457 Drop support for Ubuntu 20.04
43c9c50 Drop syslog.target and systemd-managed /var/run
be337c6 Enable derp.server.verify_clients by default
e73b2a9 Ensure that a username starts with a letter (#2635)
fa619ea Fix CHANGELOG for autogroup:member and autogroup:tagged (#2733)
086fcad Fix Internal server error on /verify (#2735)
bad7833 Fix /machine/map endpoint vulnerability (#2642)
46c59a3 Fix command in bug report template
a8f2eeb Fix config param name in TLS doc
e7fe645 Fix invocation of golangci-lint (#2703)
3123d52 Fix typos
4e6d42d Keycloak's group format is configurable
30a1f7e Log registrationID to simplify interactive node registration
2d680b5 Misc typos and spelling
5d8a2c2 OIDC: Query userinfo endpoint before verifying user
4a941a2 Refactor Debian/Ubuntu package
d461db3 Refactor OpenID Connect documentation
a2a6d20 Refactor to use reflect.TypeFor
8ff5baa Refresh OIDC docs
b8044c2 Replace magic-nix-cache-action (#2575)
a98d9bd The preauthkeys commands expect a user id instead of a username
881a6b9 The sequential prefix allocation uses a best-effort approach
3fbde7a Update official.md
860a8a5 Update tools.md
4d61da3 Use an IPv4 address range suitable for documentation
c6427aa Use group id instead of group name for Entra ID
c07cc49 add health command (#2659)
7fce506 all: remove 32 bit support (#2692)
73023c2 all: use immutable node view in read path
d41fb4d app: fix sigint hanging
8e25f7f bunch of qol (#2748)
4668e5d changelog: add entry for db
e7a28a1 changelog: prepare for 0.27.0 (#2797)
d29feae chore(derp): allow nil regions in DERPMaps
630bfd2 chore(derp): prioritize loading DERP maps from URLs
022098f chore: make reg cache expiry tunable
081af26 ci: fix golangci-lint flag for v2 compatibility (#2654)
3950f8f cli: use gobuild version handling (#2770)
ea7376f cmd/hi: add integration test runner CLI tool (#2648)
afc11e1 cmd/hi: fixes and qol (#2649)
3326c5b cmd/hi: lint and format
684239e cmd/mapresponses: add mini tool to inspect mapresp state from integration
2b30a15 cmd: add option to get and set policy directly from database (#2765)
c6736dd db: add sqlite "source of truth" schema
50ed248 debug: add json and improve
38be30b derp: allow override to ip for debug
7056fbb derp: fix flaky shuffle test (#2772)
b875676 derp: increase update frequency and harden on failures (#2741)
3e3c72e docs(acls): Add example for allow/deny all acl policy
ded049b don't crash if config file is missing (#2656)
df69840 feat(tools): Add Go client implementation
6750414 feat: add autogroup:member, autogroup:tagged (#2572)
c2a58a3 feat: add autogroup:self (#2789)
d778743 feat: add robots.txt
d325211 feat: add verify client config for embedded DERP (#2260)
1605e2a fix typo in TailSQL's log
efc6974 fix typo in parseCapabilityVersion, and removed unused error (#2644) (#2644)
43f90d2 fix: allow all traffic if acls field is omited from the policy
3f6657a fix: documentation
4927e9d fix: improve mapresponses and profiles extraction in hi tool
c4a8c03 fix: return valid AuthUrl in followup request on expired reg id
3bad5d5 flake.lock: Update (#2585)
6220e64 flake.lock: Update (#2669)
1a7a2f4 flake.lock: Update (#2699)
40b3d54 flake.lock: Update (#2755)
d311d2e flake: dont override gopls
4de56c4 flake: goreleaser doesnt follow go nix convention (#2779)
3944318 gen: new proto version
22e6094 golangci: disable varnamelen
30525ce goreleaser: always do draft (#2595)
a975b6a hscontrol: remove go-grpc-middleware v1 dependency (#2653)
9b96295 integration: Eventually, debug output, lint and format
044193b integration: Use Eventually around external calls (#2685)
c874711 integration: eventually fixups (#2799)
4893cda integration: make timestamp const
c6d7b51 integration: replace time.Sleep with assert.EventuallyWithT (#2680)
3b16b75 integration: rework retry for waiting for node sync
9779adc integration: run headscale with delve and debug symbols (#2689)
306d8e1 integration: validate expected online status in ping
233dffc lint and leftover
b6d5788 mapper: produce map before poll
a058bf3 mapper: produce map before poll (#2628)
ed3a9c8 mapper: send change instead of full update (#2775)
ccd79ed mcp: add some standard mcp server
bd6ed80 policy/v2: error on missing or zero port (#2606)
ee0ef39 policy: fix ssh usermap, fixing autogroup:nonroot (#2768)
2938d03 policy: reject unsupported fields (#2764)
a52f1df policy: remove v1 code (#2600)
01c1f6f policy: validate error message for asterix in ssh (#2766)
c91b9fc poll: add missing godoc (#2763)
b904276 poll: use nodeview everywhere
0303b76 postgres uses more memory
855c48a remove unneeded check (#2658)
fddc711 stability and race conditions in auth and node store (#2781)
9d23657 state/nodestore: in memory representation of nodes
476f30a state: ensure netinfo is preserved and not removed
1553f0a state: introduce state
b4f7782 support force flag for nodes backfillips
4912769 update dependencies (#2798)
81b3e8f util: harden parsing of traceroute
d2879b2 web: change node registration parameter order (#2607)
1b1c989 {policy, node}: allow return paths in route reduction (#2767)