amithkoujalgi/pdf-bot repository overview

⁠PDF Bot with Ollama

A bot that accepts PDF docs and lets you ask questions on it.

The LLMs are downloaded and served via Ollama⁠.

⁠Table of Contents

• Requirements⁠
• How to run⁠
• Demo⁠
• Improvements⁠
• Contributing⁠
• Credits⁠

⁠Requirements

⁠How to run

⁠CPU version

make start

⁠GPU version

make start-gpu

When the server is up and running, access the app at: http://localhost:8501⁠

Switch to a different model by changing the MODEL env variable in the docker-compose.yaml⁠. Check out the available models from here⁠.

Note:

• It takes a while to start up since it downloads the specified model for the first time.
• If your hardware does not have a GPU and you choose to run only on CPU, expect high response time from the bot.
• Only Nvidia is supported as mentioned in Ollama's documentation. Others such as AMD isn't supported yet. Read how to use GPU on Ollama container⁠ and docker-compose⁠.
• Make sure to have Nvidia drivers setup on your execution environment for the best results.

Image on DockerHub: https://hub.docker.com/r/amithkoujalgi/pdf-bot⁠

⁠Demo⁠

https://github.com/amithkoujalgi/ollama-pdf-bot/assets/1876165/40dc70e6-9d35-4171-9ae6-d82247dbaa17⁠

⁠Sample PDFs

Hl-L2351DW v0522.pdf⁠

HL-B2080DW v0522.pdf⁠

⁠Improvements

• Expose model params such as temperature, top_k, top_p as configurable env vars

⁠Contributing

Contributions are most welcome! Whether it's reporting a bug, proposing an enhancement, or helping with code - any sort of contribution is much appreciated.

⁠Requirements

⁠Setup Ollama server for development

docker run -it -v ~/ollama:/root/.ollama -p 11434:11434 ollama/ollama

⁠Install the libs

pip install -r requirements.txt

⁠Start the app

streamlit run pdf_bot/app.py

⁠Credits

Thanks to the incredible Ollama⁠, Langchain⁠ and Streamlit⁠ projects.

⁠Appreciate my work?