Major Changes
- Align beam search with
hf_model.generate. - Stablelize AsyncLLMEngine with a background engine loop.
- Add support for CodeLLaMA.
- Add many model correctness tests.
- Many other correctness fixes.
What's Changed
- Add support for CodeLlama by @Yard1 in #854
- [Fix] Fix a condition for ignored sequences by @zhuohan123 in #867
- use flash-attn via xformers by @tmm1 in #877
- Enable request body OpenAPI spec for OpenAI endpoints by @Peilun-Li in #865
- Accelerate LLaMA model loading by @JF-D in #234
- Improve _prune_hidden_states micro-benchmark by @tmm1 in #707
- fix: bug fix when penalties are negative by @pfldy2850 in #913
- [Docs] Minor fixes in supported models by @WoosukKwon in #920
- Fix README.md Link by @zhuohan123 in #927
- Add tests for models by @WoosukKwon in #922
- Avoid compiling kernels for double data type by @WoosukKwon in #933
- [BugFix] Fix NaN errors in paged attention kernel by @WoosukKwon in #936
- Refactor AsyncLLMEngine by @Yard1 in #880
- Only emit warning about internal tokenizer if it isn't being used by @nelson-liu in #939
- Align vLLM's beam search implementation with HF generate by @zhuohan123 in #857
- Initialize AsyncLLMEngine bg loop correctly by @Yard1 in #943
- FIx vLLM cannot launch by @HermitSun in #948
- Clean up kernel unit tests by @WoosukKwon in #938
- Use queue for finished requests by @Yard1 in #957
- [BugFix] Implement RoPE for GPT-J by @WoosukKwon in #941
- Set torch default dtype in a context manager by @Yard1 in #971
- Bump up transformers version in requirements.txt by @WoosukKwon in #976
- Make
AsyncLLMEnginemore robust & fix batched abort by @Yard1 in #969 - Enable safetensors loading for all models by @zhuohan123 in #974
- [FIX] Fix Alibi implementation in PagedAttention kernel by @zhuohan123 in #945
- Bump up the version to v0.1.5 by @WoosukKwon in #944
New Contributors
- @tmm1 made their first contribution in #877
- @Peilun-Li made their first contribution in #865
- @JF-D made their first contribution in #234
- @pfldy2850 made their first contribution in #913
- @nelson-liu made their first contribution in #939
Full Changelog: v0.1.4...v0.1.5