frankierr/skelshop repository overview

⁠ SkelShop

⁠ ⁠ ⁠

SkelShop is a toolkit usable either as a suite of command line tools or a Python library aimed at offline analysis of the ``talking heads'' genre of videos, typically created in a television studio. This genre includes news and current affairs programmes, variety shows and some documentaries. The main attributes of this type of material is:

• Multiple fixed place cameras in a studio with shot changes to different cameras and non-studio material.
• Mainly upper bodies are visible.
• Frontal shots are the most frequent type.
• Faces are usually visible and most often frontal.
• Occlusion is as often by an OSD (On Screen Display) than by some other object in the scene.

⁠Getting started

See the documentation.⁠

⁠Features

• Dump OpenPose skeletons to a fast-to-read HDF5 file format
• A processing pipeline starting with shot segmentation
• Apply black box person tracking on OpenPose skeletons
• Draw skeletons over video and...
  • View the result in real time
  • Output the result to another video
• Convert from some existing JSON based dump formats
• Embed faces using dlib
  • Using OpenPose's face detection and keypoint estimation
  • Or using dlib's own face detection/keypoint estimation
  • Select best frames for best quality embeddings
• Dump across heterogeneous HPC environments including GPU and CPU nodes.
• Identify faces based on embeddings
  • Identify by direct comparison with references
  • Or cluster faces with no reference (also useful re-identitication)
  • Identify faces based on clusters, either with a reference or interactively
  • Quickly build up libraries of references using data from WikiData/WikiMedia commons

Here's a screenshot of the playsticks command:

Find out more about what SkelShop can do in the documentation.⁠

⁠Contributions & Questions

Contributions are welcome! Feel free to use GitHub discussions to ask any questions. See also the contributing section of the documentation.⁠

⁠Acknowledgments

Thanks to the authors of all the useful libraries I have used.

Some of the black box tracking code is based on this repository⁠.

Some code to do with clustering and usage of FAISS is based on this repository⁠.

Icon by Adrien Coquet, FR from the Noun Project used under CC-BY.⁠