Public Repository

Last pushed: 11 hours ago
Short Description
Short description is empty for this repo.
Full Description

Evaluation framework

<p align="center">
<a href=""><img alt="pipeline status" src="" /></a>

This directory contains the evaluation framework. The scorer program is the
same as the one used in Word Sense Disambiguation: A Unified Evaluation
Framework and Empirical Comparison.
. First you need to obtain it:


Running the baselines

./ /path/to/eval.xml /path/to/eval.key

Running the UKB experiments

First set the environment variable UKB_PATH to where your compiled copy of UKB
is located.

cd ukb-eval && ./ && cd ..
pipenv run python --en-synset-ids > wndict.en.txt
pipenv run python run_all /path/to/eval/corpus.xml ukb-eval/wn30/wn30g.bin wndict.txt /path/to/eval/corpus.key


This project is licensed under the Apache v2 license. The code in ukb-eval is
vendorized from UKB, and therefore licensed under the GPL. The scorer in
support/scorer is under an unknown license, possibly public domain.

See also

  • STIFF: Automatically created sense
    tagged corpus of Finnish and corpus wrangling tools.
  • STIFF-explore: Some exploratory
    coding related to STIFF.
  • finn-man-ann: Small, Finnish
    language, manually annotated word sense corpus.
  • FinnTK: Simple, high-level toolkit for
    Finnish NLP, mainly providing convenience methods for, and gluing together,
    other tools.
  • extjwnl_fiwn: Java code to make
    extjwnl interoperate with FinnWordNet.
  • FinnLink: Link between FinnWordNet
    and Finnish Propbank created by joining with PredicateMatrix.
  • finn-sense-clust: Sense
    clusterings of FinnWordNet.


  • ItMakeseSense: ItMakesSense fork to
    support FiWN for use by finn-wsd-eval
  • AutoExtend: AutoExtend fork to
    support FiWN and ConceptNet Numberbatch
  • babelnet-lookup:
    babelnet-lookup fork to obtain BABEL2WN_MAP.
  • FinnWordNet: Temporary fixes to
    FinnWordNet 2.0.
Docker Pull Command