Application to determine if a claim is worthy of fact-checking.
343
Docker application of Claimbuster- Spotter - https://github.com/utaresearch/claimbuster-spotter/tree/master
Application live demo - https://idir.uta.edu/claimbuster/api/
Note - There are two versions, V2 and V1. Version 1 has not pre-installed GPU-related libraries (such as NVIDIA packages). It is lighter, but users who opt for this version must undergo a process to make their system's GPU recognizable to the model. Version 2 includes the Nvidia drivers, making it heavier (larger) than Version 1, but it spares users from having to perform additional GPU configurations independently.
The following instructions are for V2
To utilize GPUs for running the model, install GPU support for docker make sure cuda and cudnn are installed properly on host machine.
nvidia-smi
make sure GPU devices are visible.
To run the container for inference-
docker run -p 5001:5001 --gpus all idirlab/claimbuster-spotter
For inferencing, use POST method - For single-sentence inference-
import requests
# Define the URL of your Flask application
url = 'http://0.0.0.0:5001/score/text/'
#or try
#url = 'http://localhost:5001/score/text/'
# Define the input text you want to send
text_to_score = {"input_text": "Average income in the USA increased by 5%"}
headers = {"Content-Type": "application/json"}
response = requests.post(url, json=text_to_score, headers=headers)
For batch inference -
# Define the URL of your Flask application
url = 'http://0.0.0.0:5001/score/batches/'
#or try
#url = 'http://localhost:5001/score/batches/'
# Define the input text you want to send
text_to_score = {"paragraphs":["Average income in the USA increased by 5%", "But during the same period inflation increased by 7%"]}
headers = {"Content-Type": "application/json"}
response = requests.post(url, json=text_to_score, headers=headers)
To train the model, you can mount your data and output folders (if you wish to store weights locally). Note- Once the container is stopped , all the data will be wiped off unless stored on host machine.
docker run --gpus all -it --entrypoint /bin/bash idirlab/claimbuster-spotter
conda activate cb_env
python -m ..
Once drives are mounted, invoke train.py with the necessary flags for training.
Additional Details -
Dockerfile -
FROM condaforge/mambaforge
# Copy your environment.yml file into the container
COPY environment.yml .
# Create the environment using Mamba
RUN conda env create -f environment.yml
# Clean conda of unnecessary files
RUN conda clean --yes --all && conda clean -afy
# Set the working directory
WORKDIR /app
# Copy your Python script(s) into the container
COPY . /app
# Cache stored
#ENV TRANSFORMERS_CACHE=/app/cache/
ENV HF_HOME=/app/cache/
ENV NLTK_DATA=/app/cache/nltk_data
# Expose port 5001 to the outside world
EXPOSE 5001
# Make RUN commands use the new environment:
SHELL ["conda", "run", "-n", "cb_env", "/bin/bash", "-c"]
# Demonstrate the environment is activated:
RUN echo "Make sure tensorflow is installed:"
RUN python -c "import tensorflow"
# The code to run when container is started:
ENTRYPOINT ["conda", "run", "--no-capture-output", "-n", "cb_env", "python", "app.py"]
environment.yml
name: cb_env
channels:
- conda-forge
- defaults
dependencies:
- cudatoolkit=11.2.2
- cudnn=8.1.0.77
- dataclasses=0.8
- nltk=3.7
- pandas=1.3.5
- pip=22.3.1
- python=3.7.12
- requests=2.31.0
- scikit-learn=1.0.2
- scipy=1.7.3
- setuptools=59.8.0
- tqdm=4.66.2
- pip:
- absl-py==0.15.0
- cachetools==5.3.2
- emoji==2.10.1
- google-auth==2.28.0
- markdown==3.4.4
- numpy==1.19.2
- spacy==2.3.9
- tensorflow-gpu==2.5.0
- textblob==0.17.1
- tokenizers==0.10.3
- transformers==4.4.2
- typing-extensions==3.7.4
- packaging==21.3
- sanic==21.3.2
- sanic-routing==0.4.1
- params-flow==0.7.4
- sentencepiece==0.1.91
- tensorflow-hub==0.7.0
Content type
Image
Digest
sha256:dd8a5448b…
Size
5.8 GB
Last updated
over 2 years ago
docker pull idirlab/claimbuster-spotter:v2