fcalice/knowledge-base repository overview

⁠Research Knowledge Base

This repo is connected to CI, all the python files added to the subdirectory ./knowledge_base/ will be converted to and executed as .ipynb unless the python file name ends with _slow.py. Since this run can take some time, it is better to make your change on a separate branch and then merge it into master on PR.

Currently the knowledge repo website is a pool of knowledge posts that users can access and browse by folder, tag and author. No backend database is connected to allow features like login/like/subscribe/comment etc to work. A future development is possible if some of these featurs are needed eventually.

⁠Access

Two ways to access the knowledge repo

1. Remote: Connect to fcexplore server with ssh and forward the port 7000 to local, you can just add the line LocalForward 7000 127.0.0.1:7000 to your .ssh/config.

2. Local: To open the knowledge base website without internet. In a python environment, install the python package knowledge-repo with pip. Then type the following commands

$ cd ~/forever/knowledge-base
$ knowledge_repo --repo ./web runserver

The knowledge base website is at http://127.0.0.1:7000/⁠ for both cases.

For information about the knowledge_repo command, you can check knowledge_repo⁠ documentation.

⁠Code Structure

• ./knowledge_base saves knowledge notebooks and the corresponding python files added by user. The python files are then used to generate knowledge posts for the web application.
• ./web stores the knowledge posts for the web application. It is updated automatically on the server.
• ./helper is a simple python package that keeps several tools for python file to post conversion or deployment. It can be tested later in CI if needed.

⁠Useful commands

⁠Execute notebooks on batch

ls notebooks/*.ipynb | xargs -L1 jupyter nbconvert --to notebook --execute --inplace --allow-errors

• nbconvert Executing Notebooks⁠

⁠Conversion .py <-> .ipynb

ls *.py | xargs -L1 python -m dataio.pynb

⁠Others

1. To manage the conversion among python file, executed ipython notebook and knowledge post, you can run the following command in a python virtual environment with access to rrm and findata

python helper/repo_helper.py -h

or

python -m helper.repo_helper -h