Using the incorrect version for your system will cause AllTalk to have start up issues, so you may wish to check AllTalk is starting ok before installing DeepSpeed.
Back to the DeepSpeed install instructions
THESE ARE COMPILED FOR PyTorch 2.1.x and not the new 2.2.x
Python 3.11.x
DeepSpeed v11.2 for CUDA 11.8 and Python 3.11.x
DeepSpeed v11.2 for CUDA 12.1 and Python 3.11.x
Python 3.10.x (for a custom Python environment)
DeepSpeed v11.2 for CUDA 11.8 and Python 3.10.x
DeepSpeed v11.2 for CUDA 12.1 and Python 3.10.x
You can check the version you are using by running cmd_windows.bat in the Text-generation-webui folder and then:
python --version to get the Python version
pip show torch to get the CUDA version.
NOTE: You DO NOT need to set Text-generation-webUI's --deepspeed setting for AllTalk to be able to use DeepSpeed. These are two completely separate things and incorrectly setting that on Text-generation-webUI may cause other complications.
