convmind/soco-core-trainer repository overview

⁠soco-core-trainer

A task-queue based server used to train model for QA, MRC, DISTILL. It is designed to be distributed for horizontal scalability.

⁠Install

pip install -r requirements.txt

⁠Run

docker-compose.yml -f docker/docker-compose.yml up --scale worker=3

⁠Configuration

• ENV REGION US
• ENV USE_GPU on
• ENV REDIS_URL redis://redis:6379/0
• ENV REDIS_QUEUES trainer

⁠Run Test:

pytest --cov=./
codecov --token=a5313626-6ef1-418e-9718-ee4ff3297aea

⁠Build an trainer image

docker build -t convmind/soco-core-trainer:0.1-mrc -f trainer/mrc/Dockerfile .