padho/python-classifier repository overview

⁠Satos text classifier

This is the repository of the text classifier used by Satos.

⁠Setup

(1) Make sure you are using Python > 3.4

(2) Install dependencies

pip install beautifulsoup4
pip install -U nltk
pip install fire
pip install spacy
pip install -U selenium

# Install PhantomJS
# http://phantomjs.org/

(3) That's is it. You can now run the app using the following commandline interface:

python Classifier.py classify_url 'https://www.unitymedia.de/privatkunden/agb/'

(4) The result will either be agb or random

⁠Algorithm

Currently, the classification is only done using the raw input url. Therefore, we extract certain features from the url and train a Naive Bayes classifier.