internnicomatic/llama-cpp-q8 repository overview

Uses llama-cpp python wrapper to infer GGUF models of llama 2: 7b. The intent is just causal text generation. no RAG