Announcing a new release of Atuin! v16 is out now.
This has been an interesting one. Two pretty core parts of the Atuin sync have
remained the same for the past couple of years, and performed better than I
ever anticipated. I'd always intended to replace the sync, but initially just
released something that worked "fine".
We've had issues though. In some rare circumstances, sync just won't complete.
In others, it requires atuin sync -f
, which attempts to upload/download all
history again in case some has been missed. As we grow, it becomes increasingly
important that we revisit sync and ensure it is as reliable as possible - with
lots of users, "rare" circumstances happen more often.
In this release we're introducing the "key value store". This is a new feature,
built on top of new sync and new cryptography.
The reasoning behind this is two-fold - we'd like to extend Atuin's sync and
storage capabilities, but also build a sync protocol we can truly rely upon.
Conrad did an excellent job explaining the
crypto we are using for this feature,
and I'll be following up with a similar post about the new sync protocol soon!
The goal here is to first prove that this crypto + sync are stable and
scalable, and then switch history sync over. Once done, we should be syncing
history much faster, with fewer errors, and with less data transferred. Your
data will also be kept safe with newer crypto algorithms, and we will have the
ability to upgrade and replace them as needed.
Community
kv store
So first! Atuin v16 adds a kv store. This can be used for storing small
snippets of text and syncing them encrypted across machines. Env, small shell
scripts, whatever you can fit into 100KiB. After a sync, all kv pairs you set
will be available across machines.
Presently this should be considered "beta". Data loss is unlikely, and it has
been tested pretty thoroughly. However we don't make any promises for the
longer-term life or scalability of this, and you probably don't want to go
putting production secrets in there. There are many improvements we can make
to performance and user experience here.
We will also be using it to generally enhance Atuin (eg, synced config, host
aliases, etc). With v17, we hope to polish this off and release it fully.
Please do give it a try and let us know what you think!
# set foo = bar
atuin kv set -k foo bar
# kv supports namespaces
# set foo = bar in the "nsp" namespace
atuin kv set -n nsp -k foo baz
atuin kv get foo => prints "bar"
atuin kv get -n nsp foo => prints "baz"
# run sync to ensure you are up to date
atuin sync
Workspaces
Atuin now has workspaces! This is a pseudo filter-mode, which automatically
enables when you are in a git repository.
To enable workspaces, try
workspaces = true
in your Atuin config. With workspace filtering enabled, Atuin will filter for
commands executed in any directory within a git repo tree! Filtering modes can
still be cycled with ctrl-r.
Self hosted
This release runs database migrations on both the client and the server, though
nothing touching the history tables. No dependency versions have changed.
GitHub org
Atuin now lives in an org! The project has grown a lot since I first released
it back in 2021, and it's reached the point where an org makes more sense. Please ensure
you update any references to https://github.com/atuinsh/atuin!
ARM Builds
For a long time, Atuin has lacked aarch64 builds, which has been in demand
(especially for M1 mac). Cross compilation was never very reliable for us, but
GitHub actions does not offer runners for ARM.
For this release I bought an older M1 mac, which is now setup as a runner on
the main
branch. All future releases will have ARM binaries available.
Growth stats
Atuin sync continues to grow! api.atuin.sh, at time of writing:
- serves 2300 registered users
- stores 27m (or 35GiB) of shell history
- averages around 100,000 new lines of history per day
- sees 40 signups per week
- gets 1200 MAU, 1000 WAU, and 800 DAU
This data is gathered only from what we can infer from usage patterns, which
are tracked in order to prevent abuse. Your data is end-to-end encrypted and
can only be read by you.
Full changelog
- Add graceful shutdown on SIGTERM by @InCogNiTo124 in #1014
- Update Arch Linux links in README by @eclairevoyant in #1016
- fix: Adjust broken link to supported shells by @ap-1 in #1013
- feat: do not allow empty passwords durring account creation by @YummyOreo in #1029
- Bump lukemathwalker/cargo-chef from latest-rust-1.68.0 to latest-rust-1.70.0 by @dependabot in #1026
- Fix movement keys in inverted mode by @majutsushi in #1035
- Make Ctrl-d behaviour match other tools by @majutsushi in #1040
- refactor server to allow pluggable db and tracing by @conradludgate in #1036
- Add support to override hostname and username via env vars by @Lugoues in #1041
- Bump uuid from 1.2.1 to 1.3.4 by @dependabot in #1046
- Fix
--delete-it-all
and--delete
commands by @edeustua in #913 - Make requirement of PostgreSQL 14 explicit by @wzzrd in #1048
- Bug/fix gh action aarch64 tarball by @m2nx in #872
- Bump debian from bullseye-20230502-slim to bullseye-20230612-slim by @dependabot in #1047
- Key values by @ellie in #1038
- Add namespaces to kv store by @ellie in #1052
- Builder interface for History objects by @utterstep in #933
- clean apt cache in Dockerfile by @utterstep in #932
- remove rmp-serde by @conradludgate in #1057
- fix: fixes unix specific impl of shutdown_signal by @YummyOreo in #1061
- Add RecordIndex data structure by @ellie in #1059
- remove decryption from api-client by @conradludgate in #1063
- Revamp getting started guide by @ellie in #1065
- Redirect old docs by @ellie in #1066
- Move key binding docs so they can be more easily found by @ellie in #1067
- record encryption by @conradludgate in #1058
- Fix
nu
section inkey-bindings.md
-set-env
->let-env
by @eopb in #1074 - Update Nushell support to use support* remove decryption from api-client by @conradludgate in #1063
- Revamp getting started guide by @ellie in #1065
- Redirect old docs by @ellie in #1066
- Move key binding docs so they can be more easily found by @ellie in #1067
- record encryption by @conradludgate in #1058
- Fix
nu
section inkey-bindings.md
-set-env
->let-env
by @eopb in #1074 - Update Nushell support to use supported $env update syntax by @jntrnr in #1080
- fix key encodings again by @conradludgate in #1089
- Update config.toml path default comments by @simon-b in #1092
- fix for zsh no-unset environments by @ds-cbo in #921
- atuin crypto blog post by @conradludgate in #1079
- Add OG image by @ellie in #1097
- write some key tests by @conradludgate in #1095
- Bump regex from 1.7.2 to 1.9.1 by @dependabot in #1094
- Bump semver from 5.7.1 to 5.7.2 in /docs by @dependabot in #1100
- Add new sync by @ellie in #1093
- Add total history count to the index API by @ellie in #1102
- Delete the count cache row when a user is deleted by @ellie in #1103
- Add workspace mode, enable if in git repo by @ellie in #1053
- Unvendor ratatui by @ellie in #1101
- skim: fix filtering aggregates by @conradludgate in #1114
- Serve docusaurus on /docs/ by @ellie in #1117
- Try adjusting docs root by @ellie in #1118
- Remove vercel config by @ellie in #1119
- Tidy up docs by @ellie in #1120
- Bump colored from 2.0.0 to 2.0.4 by @dependabot in #1110
- Bump lukemathwalker/cargo-chef from latest-rust-1.70.0 to latest-rust-1.71.0 by @dependabot in #1109
- Bump debian from bullseye-20230612-slim to bullseye-20230703-slim by @dependabot in #1084
- Bump serde_json from 1.0.86 to 1.0.99 by @dependabot in #1075
- Bump indicatif from 0.17.3 to 0.17.5 by @dependabot in #1050
- Bump h2 from 0.3.14 to 0.3.17 by @dependabot in #873
- Bump config from 0.13.2 to 0.13.3 by @dependabot in #646
- use Ctrl-n instead of Alt-n on macOS by @zygous in #1106
- some simple server tests by @conradludgate in #1096
- Disable server tests in the nix build by @patricksjackson in #1123
- Drop default sync frequency to 10m by @ellie in #1130
- Move all references to the old repo by @ellie in #1132
- Add self hosted runner by @ellie in #1133
- Split integration and unit tests, use runner for unit by @ellie in #1134
- fix(server): Teapot is a cup of coffee by @LeoniePhiline in #1137
- rebase: fix: nushell empty hooks by @LeoniePhiline in #1138
- Add support for
max_preview_height
setting by @RichardDRJ in #1088 - fix(docs): List all presently documented commands by @LeoniePhiline in #1140
- Bump futures-util from 0.3.24 to 0.3.28 by @dependabot in #1129
- encode paseto payloads as json by @conradludgate in #1146
- fix(docs): Correct command overview paths by @LeoniePhiline in #1145
- Prepare release v16.0.0 by @ellie in https://github.com/atuinsh/atuin/pull/1143ed $env update syntax by @jntrnr in #1080
- fix key encodings again by @conradludgate in #1089
- Update config.toml path default comments by @simon-b in #1092
- fix for zsh no-unset environments by @ds-cbo in #921
- atuin crypto blog post by @conradludgate in #1079
- Add OG image by @ellie in #1097
- write some key tests by @conradludgate in #1095
- Bump regex from 1.7.2 to 1.9.1 by @dependabot in #1094
- Bump semver from 5.7.1 to 5.7.2 in /docs by @dependabot in #1100
- Add new sync by @ellie in #1093
- Add total history count to the index API by @ellie in #1102
- Delete the count cache row when a user is deleted by @ellie in #1103
- Add workspace mode, enable if in git repo by @ellie in #1053
- Unvendor ratatui by @ellie in #1101
- skim: fix filtering aggregates by @conradludgate in #1114
- Serve docusaurus on /docs/ by @ellie in #1117
- Try adjusting docs root by @ellie in #1118
- Remove vercel config by @ellie in #1119
- Tidy up docs by @ellie in #1120
- Bump colored from 2.0.0 to 2.0.4 by @dependabot in #1110
- Bump lukemathwalker/cargo-chef from latest-rust-1.70.0 to latest-rust-1.71.0 by @dependabot in #1109
- Bump debian from bullseye-20230612-slim to bullseye-20230703-slim by @dependabot in #1084
- Bump serde_json from 1.0.86 to 1.0.99 by @dependabot in #1075
- Bump indicatif from 0.17.3 to 0.17.5 by @dependabot in #1050
- Bump h2 from 0.3.14 to 0.3.17 by @dependabot in #873
- Bump config from 0.13.2 to 0.13.3 by @dependabot in #646
- use Ctrl-n instead of Alt-n on macOS by @zygous in #1106
- some simple server tests by @conradludgate in #1096
- Disable server tests in the nix build by @patricksjackson in #1123
- Drop default sync frequency to 10m by @ellie in #1130
- Move all references to the old repo by @ellie in #1132
- Add self hosted runner by @ellie in #1133
- Split integration and unit tests, use runner for unit by @ellie in #1134
- fix(server): Teapot is a cup of coffee by @LeoniePhiline in #1137
- rebase: fix: nushell empty hooks by @LeoniePhiline in #1138
- Add support for
max_preview_height
setting by @RichardDRJ in #1088 - fix(docs): List all presently documented commands by @LeoniePhiline in #1140
- Bump futures-util from 0.3.24 to 0.3.28 by @dependabot in #1129
- encode paseto payloads as json by @conradludgate in #1146
- fix(docs): Correct command overview paths by @LeoniePhiline in #1145
- Prepare release v16.0.0 by @ellie in #1143
New Contributors
- @eclairevoyant made their first contribution in #1016
- @ap-1 made their first contribution in #1013
- @Lugoues made their first contribution in #1041
- @edeustua made their first contribution in #913
- @wzzrd made their first contribution in #1048
- @m2nx made their first contribution in #872
- @eopb made their first contribution in #1074
- @jntrnr made their first contribution in #1080
- @simon-b made their first contribution in #1092
- @ds-cbo made their first contribution in #921
- @zygous made their first contribution in #1106
- @LeoniePhiline made their first contribution in #1137
- @RichardDRJ made their first contribution in #1088
Full Changelog: v15.0.0...v16.0.0
Potential issues
If at any point after running v15 you ran main
, there is a chance you will see the following error
Error: migration 20230619235421 was previously applied but is missing in the resolved migrations
Location:
/atuin-client/src/record/sqlite_store.rs:53:9
This can be resolved by removing the records.db
file inside the Atuin data directory.
On most systems
rm ~/.local/share/atuin/records.db*
During the development of v16, we updated a migration. This will only occur if
you ran a build from source, from main
, after v15.
If you'd like to avoid issues like this in the future, I'd suggest avoiding
running main
unless you are OK with the occasional breakage. Drop by our
Discord with any problems <3