varikmp/profiler repository overview

⁠Authors

• Wes Lloyd & Huazeng Deng & Ling-hong Hung
• Raymond Schooley & David Perez & Varik Hoang

⁠General Information

The Container Profiler can be used as a tool to profile an application or workflow by taking interval snapshots of a collection of linux resource utilization metrics throughout the course of the job. These snapshots are then stored as JSON data which can then be used to see how the metrics changed once the job is finished.

To profile a sleep function in ubuntu

docker run --rm -it -v ${PWD}:/data -e PROFILER_OUTPUT_DIR=/data varikmp/profiler "sleep 10"

You can also profile a multiple commands in just one profiling run

docker run --rm -it -v ${PWD}:/data -e PROFILER_OUTPUT_DIR=/data varikmp/profiler "sleep 10; cat /proc/cpuinfo"

To use another docker command inside the profiling tool, make sure you map another volume of the docker daemon in the host machine

docker run --rm -it -v ${PWD}:/data -e PROFILER_OUTPUT_DIR=/data -v /var/run/docker.sock:/var/run/docker.sock varikmp/profiler "docker run --rm -v ${PWD}:/data varikmp/bwa mem -M -t 1 /data/hg19bwaidx/hg19bwaidx /data/TUMOR.bam.fq"

After the above command, you should see two different JSON files generated. We now calculate the deltas of those metrics listed in the JSON files

docker run --rm -it -v ${PWD}:/data varikmp/delta /data/2021_05_20_00_04_45.json /data/2021_05_20_00_04_35.json

The result is generated from the above command will not be stored on the disk. There is an alternative way to save the result

docker run --rm -it -v ${PWD}:/data varikmp/delta /data/2021_05_20_00_04_45.json /data/2021_05_20_00_04_35.json > delta.json