github turboderp-org/exllamav3 v1.1.0
1.1.0

latest releases: v1.6.4, v1.6.3, v1.6.2...
3 months ago
  • Support HYV3ForCausalLM
  • Reduced CPU overhead for TP mode and improved prefill performance in native backend
  • Adjust top-P=1 sampling to match implicit truncating behavior of torch.multinomial
  • Banned strings now supported for recurrent models
  • Bugfixes and QoL improvements

Full Changelog: v1.0.0...v1.1.0

Don't miss a new exllamav3 release

NewReleases is sending notifications on new releases.