anindyadeep/yoga_pose_gnn repository overview

⁠YogaPosGNN Version 2.0

GIF credits⁠. This is another kinda same project done and made on the top of SignLangGNN⁠. But this project is much more robust and accurate and can perform real time Yoga position classification using Graph Neural Networks. The best part of this project is that the CPU utlization. As its just using some pixel co-ordinate changes in the video. Also Graph neural networks are emerging more and more in several aspects of computer vision. So this problem is framed as a graph classification problem. I used a simple two Graph Attention layers and a softmax classifier as the network architecture. In just 20 epochs it gives an accuracy of 0.91 and 0.89 of train and test accuracy respectively. Better than the previous versions.

⁠Contents

1. Running the project locally ⁠
2. Results ⁠
3. Improvements after version 1.0 release ⁠
4. Future Works ⁠

⁠Running the project Locally

⁠Using Docker

The docker image is been released in DockerHub 🥳

$ docker run --privileged --device=/dev/video0:/dev/video0 anindyadeep/yoga_pose_gnn:master-3e72318

⁠Cloning the repo and building the image

You can clone the project using:

$ git clone https://github.com/Anindyadeep/YogaPosGNN.git

And after that if you want to run the build the docker image locally and then run the following.

$ docker build -t <image_name> .
$ docker run --privileged --device=/dev/video0:/dev/video0 <image_name>

Just replace the <image name> with the any arbitary name for e.g. test_image. providing OS and camera privilages are very much important, so we need to use the additional commands, otherwise, you might not use video based results.

⁠Results

Intial results of the model is pretty good based on the fact it has relatively less training data as its just 250 images (per class). After training the model is giving a train accuracy of 0.89 and a test accuracy of 0.91. This could be improved by improving the model and adding edge features also. This model is also using limited nodes, as I did't add the face and the palm nodes, in order to make it more realistic.

⁠Improvements done after release of version 1.0

Software improvements

1. Improved the docker file and solved the issue of streamlit port problem
2. Set up GitHub actions. Implemented CI/CD pipeline for docker. So every merge to main branch will result in improvement in the docker image directly in the DockerHub.

ML Model Improvements

1. Minor changes in model, tweaked some hyper parameters and implemented BatchNorm.
2. GraphAttention Conv increased it's test accuracy by 2 %/

⁠Future Works

Currently there are some issues both interms of the Software infrastructure and the AI model. Both of them has a great scope of developement. Some of the works that are currently taking place and can be done in future are listed below.

Improvements based on software developement

1. Implementing better CI/CD pipeline that will directly deploy the docker image to DockerHub and required changes to Heroku.

2. Solving the problem of providing OS privilages like camera acess to the user while running on heroku.

Model based future works and potential improvements

1. Finding a proper video based dataset providing different actions of Yoga poses.
2. Implemeting Attention based TGN models for better and much robust performance.
3. Implementing a mechanism, that will help the user through guiding the switch between one pose to another pose.
4. An AI based yoga assitant that will asses how much better the user is doing all the poses and what can be done. This can be potentially done by fusing NLP, XAI and Existing Approaches of Computer Vision and Graph ML.