6zar/alpaca repository overview

To run locally:

docker run --gpus=all --shm-size 64g -p 7860:7860 -v ${HOME}/.cache:/root/.cache --rm 6zar/alpaca generate.py     --load_8bit     --base_model 'decapoda-research/llama-7b-hf'     --lora_weights 'tloen/alpaca-lora-7b'

If you want to use with GPU, from my tests you need at least 20GB of VRAM at the GPU.

For this test you can use runpod.io⁠ to allocate container with GPU of your choice for a good price.

⁠Running on Runpod.io

Create a template and set:

• container image: 6zar/alpaca
• docker command: generate.py --load_8bit --base_model 'decapoda-research/llama-7b-hf' --lora_weights 'tloen/alpaca-lora-7b'
• container disk: 30GB
• expose HTTP: 7860,