hadr0n/scraibe repository overview

⁠ScrAIbe: Streamlined Conversation Recording with Automated Intelligence Based Environment

ScrAIbe is a state-of-the-art, PyTorch⁠ based multilingual speech-to-text framework to generate fully automated transcriptions.

Beyond transcription, ScrAIbe supports advanced functions, such as speaker diarization and speaker recognition.

Designed as a comprehensive AI toolkit, it uses multiple AI models:

• whisper⁠: A general-purpose speech recognition model.
• payannote-audio⁠: An open-source toolkit for speaker diarization.

The framework utilizes a PyanNet-inspired pipeline, with the Pyannote library for speaker diarization and VoxCeleb for speaker embedding.

During post-diarization, each audio segment is processed by the OpenAI Whisper model, in a transformer encoder-decoder structure. Initially, a CNN mitigates noise and enhances speech. Before transcription, VoxLingua identifies the language segment, facilitating Whisper's role in both transcription and text translation.

⁠Docker Usage

sudo docker run  -p 7860:7860  --name scraibe hadr0n/scraibe/scraibe0.1.0medium  --start-server --server-name 0.0.0.0