A C++ Template Library for Distributed Data Structures with Support
for Hierarchical Locality for HPC and Data-Driven Science.
Exascale systems are scheduled to become available in 2018-2020 and will be
characterized by extreme scale and a multilevel hierarchical organization.
Efficient and productive programming of these systems will be a challenge,
especially in the context of data-intensive applications. Adopting the
promising notion of Partitioned Global Address Space (PGAS) programming the
DASH project develops a data-structure oriented C++ template library that
provides hierarchical PGAS-like abstractions for important data containers
(multidimensional arrays, lists, hash tables, etc.) and allows a developer
to control (and explicitly take advantage of) the hierarchical data layout
of global data structures.
In contrast to other PGAS approaches such as UPC, DASH does not propose a
new language or require compiler support to realize global address space
semantics. Instead, operator overloading and other advanced C++ features
are used to provide the semantics of data residing in a global and
hierarchically partitioned address space based on a runtime system with
one-sided messaging primitives provided by MPI or GASNet.
As such, DASH can co-exist with parallel programming models already in
widespread use (like MPI) and developers can take advantage of DASH by
incrementally replacing existing data structures with the implementation
provided by DASH. Efficient I/O directly to and from the hierarchical
structures and DASH-optimized algorithms such as map-reduce are also part
of the project. Two applications from molecular dynamics and geoscience
are driving the project and are adapted to use DASH in the course of the
DASH is funded by the German Research Foundation (DFG) under the priority
programme "Software for Exascale Computing - SPPEXA" (2013-2018).
- Slack channel: https://dash-project.slack.com
- Support: firstname.lastname@example.org
- Developer mailing list: email@example.com
See guidelines in CONTRIBUTING.md.
DASH installations are available as Docker containers or build from source
For pre-build Docker container images, see the
DASH project on Docker Hub.
Building from Source
DASH is build using CMake.
Build scripts are provided for typical DASH configurations and can serve
as starting points for custom builds:
|Script file name||Description|
||Standard release build|
||Development / debug build|
||Release build for Intel MIC (Xeon Phi)|
- CMake version 2.8.5 or greater (3.0.0 or greater recommended)
- C compiler supporting the C99 standard
- C++ compiler supporting the C++11 standard
Optional third-party libraries directly supported by DASH:
- BLAS implementation like Intel MKL, ATLAS
Building DASH from Source
To build the DASH project using CMake with default build settings, run:
(dash/)$ cmake --build .
Or, specify a new directory used for the build:
(dash/)$ mkdir build && cd ./build
For a list of available CMake parameters:
(build/)$ cmake .. -L
Parameters can be set using
-D flags. As an example, these parameters
will configure the build process to use icc as C compiler instead of the
(build/)$ cmake -DCMAKE_C_COMPILER=icc ..
To configure build parameters using ccmake:
(build/)$ ccmake ..
1. Choosing a DASH runtime (DART)
DASH provides the following variants:
- MPI: The Message Passing Interface, requiring a MPI 3.0 compliant
- CUDA: nNvidia's Compute Unified Device Architecture (contributor
- SHMEM: Symmetric Hierarchical Memory access (contributor distribution
The build process creates the following libraries:
By default, DASH is configured to build all variants of the runtime.
You can specify which implementations of DART to build using the cmake
(build/)$ cmake -DDART_IMPLEMENTATIONS=mpi,shmem ...
Programs using DASH select a runtime implementation by linking against the
Specifying the MPI implementation for the DART-MPI runtime
The most reliable method to build DART-MPI with a specific MPI installation
is to specify the CMake options
(build/) $ cmake -DMPI_C_COMPILER=/path/to/mpi/bin/mpicc \ -DMPI_CXX_COMPILER=/path/to/mpi/bin/mpiCC \ ...
3. Examples and Unit Tests
Source code of usage examples of DASH are located in
Examples each consist of a single executable and are built by default.
Binaries from examples and unit tests are deployed to the build direcory
but will not be installed.
To disable building of examples, specify the cmake option
(build/)$ cmake -DBUILD_EXAMPLES=OFF ...
To disable building of unit tests, specify the cmake option
(build/)$ cmake -DBUILD_TESTS=OFF ...
The example applications are located in the bin/ folder in the build
The default installation path is
$HOME/opt as users on HPC systems
typically have install permissions in their home directory only.
To specify a different installation path, use
(build/)$ cmake -DINSTALL_PREFIX=/your/install/path ../
-DINSTALL_PREFIX=<DASH install path> can also be given in Step 1.
The installation process copies the 'bin', 'lib', and 'include' directories
in the build directory to the specified installation path.
(dash/)$ cmake --build . --target install
Or manually using make:
(build/)$ cmake <build options> ../ (build/)$ make (build/)$ make install
Running DASH Applications
Building a DASH application
DASH provides wrapper scripts that ensure correct include
paths as well as linking for all required DASH, DART, and
The wrappers are named as
dash-<variant>c++, depending on the
DASH variant activated at build time, e.g., for the MPI variant the
wrapper will be called
dash-mpic++ (and its aliases
To build a DASH (MPI) application, replace the call to the MPI C++ compiler
with a call to
dash-mpicxx (or its alias
$ CXX=dash-mpicxx make
The compiler wrapper currently provides two options:
--dash:verbosewill cause all invocations of the underlying compiler to
be printed to the console.
--dash:nocppflagsdisables passing DASH-related precompiler flags to
the underlying compiler, including flags that control DASH verbosity (if enabled during DASH build) and assertions.
All other parameters will be passed to the underlying compiler, allowing you to control
optimization flags and pass precompiler options.
Note that the compiler wrappers also set the language standard to C++11.
Running a DASH application
With the MPI variant, applications are spawn by MPI:
$ mpirun <MPI args> <app>-mpi
For CUDA and SHMEM, use
$ dartrun-cuda <dartrun-args> <app>-cuda
$ dartrun-shmem <dartrun-args> <app>-shmem
Launch the DASH unit test suite using
(dash/shmem/bin/)$ dartrun-shmem <dartrun args> dash-test-shmem <gtest args>
(dash/mpi/bin/)$ mpirun <MPI args> dash-test-mpi <gtest args>
For example, you would all unit tests of matrix data structures on 4 units
using the MPI runtime with:
(dash/mpi/bin/)$ mpirun -n 4 dash-test-mpi --gtest_filter="MatrixTest*"
or all tests except for the Array test suite:
(dash/mpi/bin/)$ mpirun -n 4 dash-test-mpi --gtest_filter="-ArrayTest*"
Profiling DASH Applications using IPM
LD_PRELOAD to run a DASH application built with the DART-MPI backend:
$ LD_PRELOAD=/$IPM_HOME/lib/libipm.so mpirun -n <nproc> <DASH executable>
Available options for IPM are documented in the
IPM user guide.
The DASH project homepage: http://www.dash-project.org
The Munich Network Management homepage: http://www.mnm-team.org