Sign inSign up

hltcoe/patapsco

By hltcoe

•Updated almost 5 years ago

Patapsco CLIR pipeline

Image
0

3.6K

hltcoe/patapsco repository overview

Place all required files in a directory on the host. Then run Patapsco:

docker run -v /home/user/my_data:/data --rm -i hltcoe/patapsco /data/eng/eng_basic.yml

It is best if the paths in the YAML configuration file be absolute paths. They should use the container path of the mounted volume.

Example:

run:
  name: English basic run
  path: /data/runs/eng-basic-run

text:
  normalize:
    lowercase: true
  tokenize: whitespace
  stem: porter

documents:
  input:
    format: json
    lang: eng
    encoding: utf8
    path: /data/eng/eng_mini_docs.jsonl
  process:
    inherit: text

database:
  name: sqlite

index:
  name: lucene

topics:
  input:
    format: json
    lang: eng
    encoding: utf8
    path: /data/eng/eng_mini_topics.jsonl
  fields: title

queries:
  process:
    inherit: text

retrieve:
  name: bm25
  number: 5

score:
  input:
    path: /data/eng/eng_mini_qrels

This image has the spacy models for English and the multi-language model used by moses for sentence segmentation. If other languages are desired, it is best to inherit from this image and install those models in the new image.

Tag summary

Content type

Image

Digest

Size

7.2 GB

Last updated

almost 5 years ago

docker pull hltcoe/patapsco