phamsonn/dog-bark repository overview

⁠Deploy Voice Recognition by Deep Learning to Recognize Dog's Bark

⁠Research:

Some links for recognizing dog bark. In short, it is possible with the simple detection of sound amplitude or more complicated with deep learning. There are guidelines and models alreay available, which looks possible to pursue:

• This link is promising as codes are shared and training data is available. I tried this and successfully build up the Deep Learning model to recognize dog bark: https://devopstar.com/2020/04/13/dog-bark-detector-machine-learning-model⁠
• Sound classification using deep learning, this is the good explantion for the above link as it is using the same data set and model: https://mikesmales.medium.com/sound-classification-using-deep-learning-8bc2aa1990b7⁠
• Video: Bark Detection in PYthon by Nicholas Kridler in 2017; however, no code or data is shared: https://channel9.msdn.com/Events/PyData/Seattle2017/BRK08⁠
• There is an available Python package named BarkTracker to detect pet's noise by setting up a certain level of decibels. This is simple but has a high risk of catching wrong sound/noice or not catching dog bark: https://pypi.org/project/BarkTracker/⁠
• Dog mood detector with Google clould and arm-based raspberry pi. This is more comprehensive with big data from Google Research and 527 data labels; however, it is more complicated than the scope of this project. We will check this direction in the future if having time: https://www.codeproject.com/Articles/5254477/Dog-Mood-Detector-with-Google-Cloud-and-Arm-based⁠

⁠Expected Outcome:

• Input sound in WAV file, the model needs to predict whether it is the sound of dog bark.
• Model needs to be precise, and we only need to train it once then are able to re-use the training result for prediction.
• To re-use the trained model, there are some approaches including:
  • Transfer learning pioneered by Andrew Ng: This allow to train the trained model with new data on top of it.
  • Use pickle package to reuse Python models without retraining them. We use this pickle package in our project: https://towardsdatascience.com/how-to-reuse-your-python-models-without-retraining-them-39cd685659a5⁠

⁠Methodology

Ref:

https://devopstar.com/2020/04/13/dog-bark-detector-machine-learning-model⁠

https://mikesmales.medium.com/sound-classification-using-deep-learning-8bc2aa1990b7⁠

⁠Issues:

• Work with big training data (6GB) takes much more time than usual. Strategy to overcom:
  • Extract sample data in a smaller volumne from the main data. After the codes run smoothly without error, we then switch to big dataset
  • When coding, divide codes into smaller modules and work with each small module first before moving on to the next one.
• Conflictions of library package version:
  • Machine learning and deep learning involve quickly. The project use tensorflow developed by Google which is updated regularly and some latest is not compatible with previousone. Also the code is working with Python 3.6 but not working with Python 3.8 and some IDE, ie. Jupyter Notebook, are not working with the code due to this issue of version confliction.
  • Solution: Setup the environment with proper python and package versions
    • Use Python virtual environment.
    • Enforce the installation of Python 3.6, tensorflow 1.14

⁠How-to

⁠Other route for running Tensorflow in Raspberry Pi 3: Install docker

https://docs.docker.com/engine/install/debian/⁠

⁠Old way, not working in the current Raspberry Pi 3

In raspberry pi 3b, install miniconda by follow this link: https://gist.github.com/simoncos/a7ce35babeaf73f512be24135c0fbafb⁠

The above miniconda is old and with Python 3.4. Since our project needs Python 3.6, the below command can be run to upgrade Python to 3.6

conda config --add channels rpi
conda install python=3.6

Update miniconda

conda update conda
conda update --all

Setg up the environment and activate it.
```bash
conda create -y -n dog-bark
conda activate dog-bark

piwheels might be the solution

Install required libraries

pip3 install --upgrade pip
pip install tensorflow==1.14 #if the installation stop in the middle, try pip install tensorflow==1.14 --no-cache-dir
pip3 install keras==2.3.1
pip3 install matplotlib
pip3 install numpy
pip3 install librosa
pip3 install pandas
pip3 install IPython

⁠Deep Learning Training:

• Python file: DogBarkTraining.py
• Dataset: UrbanSound8K released in year 2014 with 8732 labed sound from 10 classes. https://urbansounddataset.weebly.com/urbansound8k.html⁠
• After downloading, extract the dataset.
• In Python file, set the paths for the audio and metadata files.

# Set the path to the full UrbanSound dataset 
fullDataSetPath = 'd:/temp/UrbanSound8K/audio/'
metaDataPath = 'd:/temp/UrbanSound8K/metadata/UrbanSound8K.csv'

• The UrbanSound8K has 8732 files. In our Python file, we can set the dataCount variable to load all data, or just some of them.

# Set amount data to load for training and validating
dataCount = 10000

• Training result
  • Accuracy to recognize 10 classes. Our predicting code will utilize that result and predict dog bark positively if the dog bark percentage is higher than 50%.:
    • Training accuracy: 93%
    • Testing accuracy: 88%
  • Output files in the working folder:
    • dog-bark.pickle: It is the trained model, which can be used for future prediction with other wav files.
      • Training once, use forever!
      • Compressed 6GB of data into 1MB trained model!
    • list.csv: It includes 10 classes of the Urbansound dataset

⁠Deep Learning Predicting

• Python file: DogBarkPredicting.py
  • Files dog-bark.pickle and list.csv are used for prediction, thus those files need to be in the same folder with DogBarkPredicting.py
  • The file can be run directly with python, or can be imported as a library.
    • If running directly, the file is testing the file './model/bark.wav' as the default. We can go to the Python file to change the wav file to test.

⁠Opportunity to improve:

Accuracy of the current model with the current dataset is 80-85%. Such accuracy would be improved further.

• Deep learning model: Due to the scope of this project, we do not intend to re-invent the wheel, thus employ the existing model. More reasearch of models and optimization of hyper-parameters of the models would improve the accuracy further.
• Dataset:
  • Dataset plays a vital role in the accuracy. Current model is trained with dataset UrbanSound8K with 8732 labeled sound excerpts divided into 10 cateogories. The data is not specifically for Australia urban, and some categories, such as gunshot, are not very relevant.
    • https://urbansounddataset.weebly.com/urbansound8k.html⁠
    • https://urbansounddataset.weebly.com/taxonomy.html⁠
  • Bigger dataset would improve the accuracy. The bigger dataset can be from the existing dataset in internet or from data collection when the device are used and more sounds are recorded. To use the recorded data for further training the model, data needs to be QAQC and labeled.
• Keep updating the model with more data when devices are used. This can be done by either:
  • Compile new data with previous data and train the model again.
  • Or use Transferred Learning, which is to train the new data with the trained model
  • Or use Federated Learning, which is to train the model with the small dataset in the device, then send the trained result back to server on the cloud to compile and train again.
• Adversarials input:
  • It is the knowned issue of deep learning, which the machine cannot recognize the accuracy issue while it is very obvious with human.
  • This is the active research area, whose result might help to improve the accuracy of this model.