Fangyuan025/Chaty v2.1.7

on GitHub

A long run that stops starting over

A prompt is resumed from the engine's cache up to the first token
that differs from last time. On the architectures the Qwen3.5
family uses that is all-or-nothing: three quarters of their layers
keep a recurrent state that cannot be rewound to a midpoint, so
anything the cache cannot account for throws all of it away and the
conversation is read from the beginning. Six things in code mode
were doing exactly that.

The clock sat in the system prompt, to the minute, so a turn that
began in a new minute matched about twenty tokens and re-read the
rest. The answer that ended a turn was never written down, so every
continuation started cold — and the model could not see what it had
last told you. Tool results arrived as if you had typed them, which
a Qwen3.5 template counts as a new request: the model would
announce that you had sent one halfway through the work it was
already doing, and repeat your request back at you. The closing tag
of a tool call went missing, and one round threw away 5006 cached
tokens over the four it could not account for. Two paths recorded a
turn their own way. And a repeated call — usually progress,
re-running the tests after an edit — turned reasoning off for one
step, which re-renders the whole history, and then back again.

After: a median of 99% of each prompt resumed from cache on both
engines, across turn boundaries as well as within a turn. A first
turn still costs a full read, so does a conversation crossing the
point where its history must be compacted, and so does the first
turn after the app restarts — the cache lives in memory and does
not outlive the process. The turn after that resumes 99% again.

Speculative decoding, on models that bring their own head

Some checkpoints carry a small block trained to guess the tokens
the model is about to produce. The model then checks a whole run of
guesses in one pass instead of one pass per token, and every token
that survives is one its own sampler drew — the reply is the reply
it would have given either way. Both engines can drive it now.

It ships off, and marked experimental. Measured, it is a large win
on text whose continuation is obvious and roughly a wash on
ordinary prose, and nothing readable before a reply starts says
which one it will be; a default that is sometimes slower is not a
default. The switch is in Settings → Model, offered only on models
that actually carry a head — read from the file, not assumed.

Three things that reached v2.1.6 as a rebuild are in here for
anyone who installed that version on release day: a chat reply now
survives the interface reloading under it, the composer stops
losing a line to a scrollbar at 110% and 120% UI scale, and the
error log can be emptied from Settings.

Install

⚠️ macOS first launch — "Apple could not verify…" / "damaged"

Chaty is ad-hoc signed but not notarized (no paid Apple Developer
account), so Gatekeeper flags it on first launch. The app is safe —
everything runs locally. Clear the download quarantine once, in Terminal:

xattr -dr com.apple.quarantine /Applications/Chaty.app

Then open Chaty normally. (Alternatively: try to open it once, then go to
System Settings → Privacy & Security → Open Anyway.)

Full Changelog: v2.1.6...v2.1.7