Skip to content

zigzag1001/LLM-to-TTS

Repository files navigation

Basic interactive LLM to voice pipeline project


Currently changing lots of things so README is probably outdated in some parts

Create an issue and i might update it & help

Demo (turn on the sound)

2024-03-07.03-34-04.mp4

Requirements

Install

  • clone repo
  • create venv and pip install requirements.txt into it
  • find devices (TTS input, output) (Im currently using VoiceMeeter Input, VoiceMeeter Out)
  • python get_audio_devices.py
  • copy the names of the devices into appropriate place in main.py and bot.py
  • main.py gets the input, bot.py gets the output
  • create .env and set TOKEN="your discord bot token"
  • download llama.cpp compatible models into ./models
  • edit config.json

Usage

  • launch both main and bot
  • press enter in main to start
  • join vc and ping bot join
  • @yourbot join
  • it probably wont work because i forgot to mention something

Notes

  • Currently using/tested mistral dolphin 2.1 and nous hermes 2
  • For tts using Xtts_v2 Vits (theres probably better options)
  • Often came across torch not detecting CUDA
  • To fix torch, install using this

About

Live voice chat with LLM through discord

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages