shamannetworks/fetchpatterns repository overview

⁠FetchPatterns

FetchPatterns reads documents, audio & video from indexed directories recursively, extracts text and displays a Word Cloud.

This is a preview release and we intend to develop the software further to generate and display additional patterns from the available text.

⁠Preparation before Use

FetchPatterns indexes documents, audio & video files from selected directories on the host file system. Docker volumes do not reflect any updates to the directories after the container is started. And hence the use of Bind Mounts. Bind Mounts need to be configured when the container is started.

We developed an application that will help you configure Bind mounts easily. Add the paths of directories you want indexed by FetchPatterns and the application will generate a docker compose file for you.

So to start:

• Identify the directories on your machine that you want indexed by FetchPatterns
• Make a list of the complete paths to each of these directories

⁠Installation & Use

1. Install Docker Desktop (if you haven't already)
2. Start a browser and navigate to 'https://fpc-two.vercel.app/⁠'
3. In the text field with placeholder 'Enter path to a root folder', type or paste one of the paths to be indexed. Click the 'Add' button. You should see the path being displayed below the text field. Repeat this step for each path you want FetchPatterns to index.
4. Once you have added all the directories, Click the 'Finished adding directories' button. You should now see a lot of text displayed in the textarea.
5. Click the 'Download' button. This will download the generated content as 'docker-compose.yml' file.
6. Open a terminal, navigate to the directory where the docker-compose.yml file was downloaded.
7. Enter the following command: docker compose up -d
8. This will start the process of downloading all the images required by FetchPatterns (Elasticsearch, Kibana, FetchPatterns Server, WhisperWS)
9. Once the applications are up (check docker the 'containers' tab for status), start a browser and navigate to 'http://localhost:8080⁠' or click 8080 in the 'Ports' column
10. You should now see the FetchPatterns UI listing out all the directories that were configured
11. Click the 'Sync all' button to start indexing documents, audio & video files
12. The progress dialog should keep you updated
13. Once the sync is complete, you can browse through the folders. Selecting documents, audio or video files should show you all the significant words from the file. You may switch between the text based view or the Word Cloud to get an insight into what the file contains.

⁠More Information on Use

For more information on using the Software please visit Shaman Networks Website⁠