mahdisabetkish/vision2words-app repository overview

⁠vision2words-app

Gradio demo for Vision2Words, an image captioning project trained on Flickr8k. Upload a photo and compare captions from three decoders: an LSTM with Bahdanau attention, a Transformer, and a Transformer that also reads the captions of the most similar training photos. The LSTM output includes a word-by-word attention map. A Retrieval tab lists the retrieved captions and can ask a small local language model (Qwen2.5-0.5B-Instruct) to write a caption from them.

⁠Run

docker run --rm -p 7860:7860 mahdisabetkish/vision2words-app

Then open http://localhost:7860⁠. On first use the trained models and the retrieval index download from Hugging Face (mahdisabetkish/vision2words-decoders), so allow a minute. The language model (about 1 GB) downloads the first time its caption is requested.

⁠Tags

latest and release versions, e.g. 0.2.0.

⁠Source

https://github.com/mahdisabetkish/Vision2Words⁠