wisesight/fasttext repository overview

⁠Sentiment and Text Classification API

Sentiment and text classification API.

จำแนกประเภทและหาความรู้สึกของข้อความ

⁠Implementation Notes

• Current implementation supports only Thai language
• More language supports is relatively easy, providing we have enough training data (say, more than 10,000 messages for each label) and a reliable tokenizer.
• Dependencies
  • Python 3.6+
  • uWSGI - as a web application framework
  • fastText⁠ - for text classification (training and prediction)
  • Wiseling⁠ - for word segmentation (the tokenizer is dictionary-based and destructive, previously known as "tzwordcut") -- Has to be separately installed, as it's not in the requirements.txt
• For a version with web interface, see https://github.com/ThothMedia/zocial-metric-sentiment⁠

⁠Quick Demo

sentiment_cli.py is a small program with command line interface to test how actual classification logic and model is working, without the need to start the entire API server.

Start a prompt waiting for a text to be classified (hit Ctrl-C to quit):

python3 sentiment_cli.py

Classify a text "This is a test":

python3 sentiment_cli.py "This is a test"

Test with predefined sample texts:

python3 sentiment_cli.py -sample

Benchmark the max speed (without network overhead) with 200 messages:

python3 sentiment_cli.py -benchmark 200

Note: Classification models (files with .ftz extensions) should be already in models/ directory. If it's not, run sh build.sh test train first.

⁠API Endpoints

Make POST request to these endpoints:

• /hello - test if serivce is alive
• /get-sentiment - sentiment analysis (negative, neutral, positive)
• /is-question - if text is a question (true, false)
• /is-rejected - if text should be excluded from an analysis (true, false)
• /is-salespam -if text is a sale spam (true, false)

⁠API Authorization

• Access token is required for every APIs. Request's header must include this:

"Authorization": "JWT SOMETOKENS0M3T0K3N"

• Don't forget to add "JWT", then a space, before the token.
• See how to authenticate and get access token.⁠

⁠Sentiment Analysis

/get-sentiment

Input:

{
    "text" : "มนุษย์ทั้งหลายเกิดมาอิสระเสรีและเท่าเทียมกันทั้งศักดิ์ศรีและสิทธิ
             ทุกคนได้รับการประสิทธิประสาทเหตุผลและมโนธรรม 
             และควรปฏิบัติต่อกันอย่างฉันพี่น้อง"
}

Output:

{
    "confidence": 0.6536582112312317,
    "has_royal_word": false,
    "neg": false,
    "neg_confidence": 0.5727977752685547,
    "pos": false,
    "pos_confidence": 0.7345186471939087,
    "value": "neutral"
}

value is a string with three possible values:

• negative
• neutral
• positive

⁠Is Question?

/is-question

Input:

{ "text" : "ราคาเท่าไหร่" }

Output:

{
    "confidence": 0.9991546869277954,
    "value": true
}

⁠Is Rejected?

/is-rejected

If a text should be rejected as irrelevant and get excluded from the analysis. -- Used internally by Analytic team.

Input:

{ "text" : "ราคาเท่าไหร่" }

Output:

{
    "confidence": 0.9223958253860474,
    "value": false
}

⁠Is Sale Spam?

/is-salespam

Note that the definition of "sale spam" is vary.

Input:

{ "text" : "ราคาเท่าไหร่" }

Output:

{
    "confidence": 0.9899575114250183,
    "value": false
}

⁠Notebooks

Once the server is started, you can try access it from interactive Python shell or Jupyter Notebook. Notebooks are inside notebooks/ directory.

This notebook shows how to process texts in Excel file and get sentiment of each text: https://github.com/ThothMedia/sentiment-api/blob/master/notebooks/sentiment_excel.ipynb⁠

⁠Model update

• Models for prediction should be inside models/ directory
• Data to train and test models are in model_data/
  • To update model data, including information on tokenization, see data preparation steps⁠.
  • model_data/MODEL_VERSIONS.txt is used to keep track (manually) of model versions.
    • The checksum of model_data/MODEL_VERSIONS.txt will also be used by CircleCI to determine if it can use previously trained (and cached) models or it has to train models again.
• To train all models, run:

sh build.sh test train

• To adjust training parameters, see "TRAINING PARAMETERS" section in train.sh
  • fastText classification parameters explained⁠
  • fastText options⁠
  • Explanation on quantization, see https://arxiv.org/abs/1612.03651⁠

⁠Build

The app will be run in a Docker container.

To understand the build steps, consult these files:

Build scripts:

• build.sh
• test_init_env.sh <-- use this to build all the necessary packages required for the server to run
• train.sh

Docker configs:

• docker/*/Dockerfile
• docker-compose.*.yml

CircleCI config:

• .circleci/config.yml

⁠Production Notes

⁠Start the service

Sentiment API is running in Docker container. To start the service run

sh build.sh production

⁠Network mode

Sentiment API running docker container in network_mode: "host" so the port of the machine will be reserved.

⁠Nginx configuration

Sentiment API in production should be use with Nginx with proper configuration.

⁠Testing Notes

Sentiment API will run as a server.

You can make a request using command-line tools like curl and wget. For convenience, we recommend Postman⁠ (GUI), as you can save different requests and run them as a collection.

⁠Install & Run - Using Docker

Sentiment API can be run from a Docker container. To start the service, run:

sh build.sh <test|staging>

To bring down the running docker, run:

docker-compose --file docker-compose.<test|staging>.yml down

If you haven't train the classification model yet, or want to retrain it, run:

sh build.sh <test|staging> train

To start the service in no daemon mode, run:

sh build.sh <test|staging> nodaemon

⁠Install & Run - Not using Docker

⁠Install required libraries

virtualenv -p python3 env
. env/bin/activate
pip install -r requirements.txt

You may also need to install fastText Python binding⁠ and datrie⁠ manually.

⁠Run Sentiment API server

Run this command to start uWSGI

uwsgi sentiment-api.uwsgi.ini

⁠Load testing

Use vegeta⁠ for load testing.

• For macOS, install by:

brew update && brew install vegeta

• For other operating systems, you can compile it from source.

To run the test, go inside tests/ directory and run:

vegeta attack -duration=10s -rate=500 -targets targets.txt | tee results.bin | vegeta report

tests/targets.txt will describe the requests, while the actual input will be in tests/data/ directory.

See the result:

vegeta report results.bin

Plot graph and save to an HTML file:

vegeta plot results.bin > graph.html

Monitor uWSGI stats:

pip install uwsgitop
uwsgitop :3032

⁠Troubleshoots

⁠fasttext: command not found

• Check your PATH
• train.sh and other scripts will use fasttext executable as accessible from PATH e.g. /usr/local/bin, /usr/bin

⁠Model is missing

• Check if models are in models/ directory
• Model extension should be .ftz (a quantized model)
• To re-train or update a model, see model update⁠ section

⁠JWT Authentication Error

Get access token by following these steps.

• Send a POST request to this URL: {hostname}:3031/auth
• The body of the request should have content-type as application/json and contains this data:

{
    "username": "tz",
    "password": "=*394A6$Q+N)665%"
}

• It will return an access token in JSON:

{
  "access_token": "SOMETOKENS0M3T0K3N"
}

• Use this access token for JWT authorization