louisdorard/gmsc repository overview

⁠Give Me Some Credit

⁠Contents of this repo

⁠App

src/⁠ contains all required code files for the application serving predictions. In particular, it contains featurization, scoring and thresholding functions (in featurizer.py⁠, scorer.py⁠ and thresholder.py⁠).

A scikit-learn model is also required, but it is stored outside of the code repository. Make sure to create a .env file at the root of this repo, specifying a value for a DATA_PATH key where data files and models can be found (see what mine looks like in sample.env⁠).

⁠Serving

Serving can be done with one of the following ways:

• locally via Flask: see source code at local_app.py⁠, and command lines for serving the app in Local-Serving.ipynb⁠ (Bash notebook)
• via the Algorithmia⁠ deployment platform: see source code at gmsc.py⁠ and live deployment⁠

Both Python files follow the same structure:

• loading the model
• defining the function to be called upon a prediction request
• passing the model and the request's raw input to the front function in front.py⁠

⁠Model builder

model_builder/⁠ contains scripts for building the scikit-learn model used by the app:

• 01-Split.ipynb⁠
• 02-Featurize.ipynb⁠ (which reuses featurizer.py⁠ from the src/ directory)
• 03-Train-model.ipynb⁠
• 04-Optimize-threshold.ipynb⁠

A training set is required. Use 00-Get-data.ipynb⁠ to get raw data from Kaggle⁠.

⁠Requirements

• requirements.txt⁠ lists the requirements for running the app "in production", on the Algorithmia platform. It can be generated from the Python environment you used to build the model by running requirements.py⁠.
• requirements-dev.txt⁠ is where you would customize the list of requirements for this project. It is used by DeepNote upon starting up the project, for initializing the environment.

⁠Others

• algorithmia.conf⁠: this was automatically generated by Algorithmia
• Git.ipynb⁠: git commands cheatsheet
• Dvc.ipynb⁠: dvc commands cheatsheet
• dvc files generated by dvc

⁠Building a model

Make sure you've set up .env file (which contains the path to the data/ folder) and your Kaggle secrets (to download the raw data).

dvc repro

⁠Bringing changes to the code or data

Changes to the training set, to the scripts in model_builder/⁠, or to the functions in src/⁠ result in a different model. After making such changes, you should:

• update __version__ in __init.py__⁠ (I recommend setting this to the date of the change, in YYYYMMDD format)
• re-run the model building scripts, which will produce a gmsc_YYYYMMDD.pkl file in the data path specified in .env, where YYYYMMDD is the date of the change
• re-run requirements.py⁠
• update the code hosted on Algorithmia (including __init.py__⁠, requirements.txt⁠, and the contents of src/⁠)
• upload gmsc_YYYYMMDD.pkl to Algorithmia
• publish a new version of the "algorithm" on Algorithmia

⁠Copyright

Louis Dorard © 2020-2021 All Rights Reserved