Sign inSign up

lewangdev/faster-whisper

By lewangdev

•Updated almost 3 years ago

An OpenAI whisper API replacement.

Image
1

2.0K

lewangdev/faster-whisper repository overview

⁠An OpenAI whisper API replacement

An OpenAI whisper compatible API implements with https://github.com/SYSTRAN/faster-whisper⁠

Support:

  • large-v3
  • large-v2
  • VAD

⁠Requirements

Install CUDA Toolkit and NVIDIA Container Toolkit on your machine:

  1. CUDA Toolkit 12.3 Update 1 Downloads https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&Distribution=Debian&target_version=12&target_type=deb_network⁠
  2. NVIDIA CUDA Installation Guide for Linux https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#common-installation-instructions-for-debian⁠
  3. Installing the NVIDIA Container Toolkit https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html⁠

⁠How To

Default model size is tiny and shipped with docker image

sudo docker run --runtime=nvidia --gpus all -d -p 8800:80 lewangdev/faster-whisper

Set container to use large-v3, the docker container will download large-v3 from huggingface, please wait a moment to download

sudo docker run --runtime=nvidia --gpus all -d -p 8800:80 -e MODEL_SIZE=large-v3 lewangdev/faster-whisper

Check the download progress like this

sudo docker logs -f {your_container_name}
lewang@devrtx:~$ sudo docker logs -f blissful_easley

==========
== CUDA ==
==========

CUDA Version 11.8.0

Container image Copyright (c) 2016-2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved.

This container image and its contents are governed by the NVIDIA Deep Learning Container License.
By pulling and using the container, you accept the terms and conditions of this license:
https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license

A copy of this license is made available in this container at /NGC-DL-CONTAINER-LICENSE for your convenience.

preprocessor_config.json: 100%|██████████| 340/340 [00:00<00:00, 939kB/s]
config.json: 100%|██████████| 2.39k/2.39k [00:00<00:00, 7.49MB/s]
vocabulary.json: 100%|██████████| 1.07M/1.07M [00:00<00:00, 1.38MB/s]
tokenizer.json: 100%|██████████| 2.48M/2.48M [00:00<00:00, 2.64MB/s]
model.bin: 100%|██████████| 3.09G/3.09G [04:22<00:00, 11.7MB/s]MB/s]

⁠Try

Open your browser and visit: http://{your_machine_ip}:8800/docs

Tag summary

Content type

Image

Digest

sha256:1071a6607…

Size

2.6 GB

Last updated

almost 3 years ago

docker pull lewangdev/faster-whisper