mreif/jcg repository overview

⁠The Java Call Graph Project (JCG)

This repository accommodates the Java Call Graph (JCG) project, a collection of annotated test cases that are relevant for call-graph construction in Java. The test cases specifically target the call graph's soundness and, therefore, tests the correct support of Java language features, core APIs, and runtime (JVM) callbacks. Not supporting those features/APIs will render the call graph unsound.

Please find the repository here: https://bitbucket.org/delors/jcg/src/master/⁠

⁠How to use the Docker container

The Docker container provides a setup environment to evaluate the pipeline proposed in the paper.

Please make sure to provide Docker at least 20GB of memory which is still not enough for all call graphs. (60GB is recommended but still not enough for everything)

⁠Evaluation

⁠Fingerprints

Usage: Java Call Graph Tests [options]

  -i, --inputDir <value>   Defines the directory with the configuration files for the input projects.
  -o, --outputDir <value>  Defines the output directory; all files will be placed here.
  --project-prefix prefix  Defines a prefix-based filter for the input project's name. If applied only projects starting with the *prefix* will be processed.
  --algorithm-prefix prefix
                           Defines a prefix-based filter for the adapters call-graph algorithms names. (e.g. filter only for RTAs)
  -a, --adapter adapter    Run the pipeline for a selecton of adapters. (e.g., "-a opal" to run OPAL's algorithms)

To create the call graphs fingerprints execute:

Generate all fingerprints:

sbt "project jcg_evaluation" "runMain FingerprintExtractor -i testcaseJars -o fingerprints"

Hint: raw data for table 3.

Generate a fingerprint for only one adapter:

sbt "project jcg_evaluation" "runMain FingerprintExtractor -i testcaseJars -o fingerprints --adapter Soot"

Generate a fingerprint for only one adapter and a particular test case category (e.g. Virtual Calls (VC)):

sbt "project jcg_evaluation" "runMain FingerprintExtractor -i testcaseJars -o fingerprints --adapter Soot --project-prefix VC"

Generate a fingerprint for one adapter, one category, and a particular algorithm (e.g. (Class Hierarchy analysis) CHA):

sbt "project jcg_evaluation" "runMain FingerprintExtractor -i testcaseJars -o fingerprints --adapter Soot --project-prefix VC --algorithm-prefix CHA"

(to generate the entire fingerprints takes several hours)

⁠Hermes results

Rerun the evaluation an extract Hermes features for all projects, we've provided an evaluation script.

run /artefactEvaluation/evaluation.sh (better run this over night...)

the output will be written to /artefactEvaluation/results/*corpus*/*program*/*tool*/*algorithm*/....

Hint: Raw data for table 2.

Perform a project-specific evaluation:

1. Make sure that you generated the fingerprints for the frameworks/algorithms you want to test

2. execute e.g. sbt "jcg_evaluation/runMain Evaluation --project-specific --hermes --fingerprint-dir fingerprints --input /corpora/files/project-specific/xalan --adapter WALA --output evaluationWithoutJDK --exclude-jdk"

Hint: Raw data for table 5 for WALA in the above command.

⁠Performance Evaluation

execute sbt "jcg_evaluation/runMain Evaluation --input /corpora/files/project-specific/xcorpus-performance --adapter OPAL --output performanceEval”

Hint: Raw data for table 4

⁠Evaluation corpora

You can find the eval projects under /corpora/files/*concrete corpus*.

⁠Known issues:

• some result might not be recomputable on all on all experiments since some call graphs require a large amount of memory (> 64GB).

• Unfortunately, we were not able to set up DOOP in the container. Please note that this is not an issue of our pipeline but results from a bug across DOOP and Souffle within the provided environment. However, DOOP is installed and can be run but throws an exception when it tries to compile the Souffle analysis from the extracted facts.