acaicia/minkowski4cpp repository overview

⁠Minkowski4CPP

CUDA-Accelerated Sparse Tensor Engine for Native C++ with Adaptive Voxelization and Gaussian Support

Minkowski4CPP is a native C++ port and extension of NVIDIA's MinkowskiEngine, designed for high-performance sparse convolutional neural networks operating on 3D and 4D spatial data.

This Docker image provides a fully configured, GPU-accelerated development environment with CUDA, LibTorch, and a precompiled Minkowski4CPP static library.

Unlike the original MinkowskiEngine, Minkowski4CPP eliminates the Python runtime dependency and extends traditional sparse tensor processing with adaptive mixed-resolution voxelization and Gaussian-based spatial representations.

These capabilities enable efficient processing of complex 3D environments, particularly where spatial detail varies significantly across a scene.

Minkowski4CPP is designed for applications including autonomous driving, robotics, 3D scene reconstruction, object detection, spatial AI, and real-time sensor fusion.

⁠Key Features

⁠1. Native C++ Sparse Tensor Engine

Minkowski4CPP provides a native C++ implementation of sparse convolutional neural network operations.

Core capabilities include:

• SparseTensor abstraction
• CoordinateManager and kernel map generation
• Sparse convolution and transposed convolution
• Sparse pooling and interpolation
• Sparse tensor pruning
• GPU acceleration through CUDA
• LibTorch tensor integration
• CPU execution and OpenMP support
• No Python runtime dependency (MINK_NO_PYBIND)

The engine is designed for integration into native C++ applications without requiring Python-based inference infrastructure.

⁠2. Adaptive Mixed-Resolution Voxelization

Traditional sparse voxel engines use a fixed voxel resolution throughout the entire spatial domain. Although sparse tensors avoid storing empty space, occupied regions must still be represented at the selected resolution.

Minkowski4CPP extends this model with adaptive, mixed-resolution voxelization, allowing voxels of different sizes to coexist within a single sparse tensor.

This enables the engine to allocate spatial detail where it is needed while representing less important regions at lower resolution.

Capabilities include:

• Mixed-resolution voxels within a single sparse tensor
• Octree-style hierarchical spatial refinement
• Configurable refinement depth
• Region-of-interest-based refinement
• Fine-resolution representation of complex geometry
• Coarse-resolution representation of distant or low-detail regions
• Convolution across resolution boundaries
• Mixed-resolution pooling and interpolation
• Adaptive resampling and spatial transformations
• Differentiable operations for neural network training
• CPU and CUDA execution
• Opt-in functionality that preserves the standard uniform SparseTensor API

Memory Efficiency

An included benchmark demonstrates the potential reduction in spatial representation size:

This represents approximately 846 times fewer voxels for the benchmarked spatial representation.

Actual memory savings and computational performance depend on the scene, refinement strategy, and workload.

Example Applications

In autonomous driving, adaptive voxelization allows a perception system to represent nearby vehicles, pedestrians, and road obstacles at high resolution while retaining lower-resolution representations of distant buildings, terrain, and background geometry.

In robotics, it enables detailed manipulation-region modeling without requiring the entire surrounding environment to use the same voxel density.

⁠3. Gaussian Spatial Representation Support

Minkowski4CPP extends its sparse spatial processing capabilities with Gaussian-based representations.

Unlike conventional voxel representations, which associate features with discrete spatial cells, Gaussian representations describe spatial information using continuous distributions.

A Gaussian primitive can represent a spatial location, its extent, and its directional uncertainty or shape using a mean vector and covariance matrix.

For a 3D Gaussian:

• Mean (μ): Spatial center
• Covariance (Σ): Spatial extent, orientation, and anisotropy
• Features: Associated learned attributes or other spatial information

This provides a flexible foundation for representing geometry, spatial uncertainty, and smoothly distributed features.

Gaussian support is intended to enable:

• Continuous spatial representations alongside sparse voxel data
• Gaussian-based geometric modeling
• Spatially distributed feature representations
• Anisotropic spatial modeling
• Gaussian-to-voxel integration
• Sparse neural network processing of Gaussian-derived features
• Hybrid Gaussian and voxel-based machine learning pipelines

Why Gaussian Representations Matter

Voxel representations are well suited to sparse convolution and spatial occupancy reasoning, but their discrete structure introduces resolution-dependent approximations.

Gaussian representations provide a complementary way to model continuous spatial information.

Combining the two approaches can support pipelines in which Gaussian primitives describe spatial structure while sparse convolutional networks learn features, classifications, or occupancy estimates.

Potential applications include:

• 3D reconstruction from multiple cameras
• Autonomous vehicle perception
• Object and scene representation
• Sensor fusion and uncertainty modeling
• Neural scene understanding
• Hybrid Gaussian-voxel reconstruction pipelines

Gaussian representation support does not inherently imply a complete differentiable Gaussian splatting renderer. Rendering, rasterization, and image synthesis require the corresponding pipeline components.

⁠4. GPU Acceleration

Minkowski4CPP uses CUDA to accelerate computationally intensive sparse tensor operations.

The Docker image includes the required development environment for compiling and linking GPU-accelerated C++ applications.

GPU capabilities include:

• CUDA sparse convolution kernels
• GPU tensor operations through LibTorch
• cuBLAS integration
• cuSPARSE integration
• CUDA runtime integration
• GPU memory management
• Support for NVIDIA Container Toolkit

The engine is designed to support high-throughput sparse spatial processing without requiring a Python execution layer.

⁠What This Docker Image Includes

The image contains a preconfigured native C++ development environment.

• CUDA Toolkit
• LibTorch (C++ PyTorch distribution)
• Precompiled libminkowski_cpp.a
• CMake build system
• Ninja build tool
• cuBLAS, cuSPARSE, and cuDART linkage
• OpenMP support
• CUDA GPU runtime compatibility
• Minkowski4CPP headers and installed library

The static Minkowski4CPP library is already compiled and installed.

Downstream applications can link directly against the installed library without rebuilding the engine.

⁠Architecture Overview

Minkowski4CPP combines sparse tensor processing, adaptive spatial representations, and GPU acceleration within a native C++ architecture.

Core Sparse Tensor Layer

• SparseTensor
• CoordinateManager
• Kernel generators
• Sparse convolution
• Pooling and interpolation
• Pruning and resampling

Adaptive Spatial Layer

• Mixed-resolution voxel representation
• Hierarchical spatial refinement
• Cross-resolution operations
• Adaptive convolution and pooling
• Differentiable mixed-resolution processing

Gaussian Representation Layer

• Gaussian spatial primitives
• Mean and covariance representations
• Continuous spatial feature modeling
• Gaussian and voxel integration

Compute Backend

• Native C++
• LibTorch
• CUDA
• cuBLAS and cuSPARSE
• OpenMP

The architecture supports conventional sparse CNN workflows while enabling more advanced spatial representations for research and production applications.

⁠Running the Container

Ensure Docker and NVIDIA Container Toolkit are installed and configured.

Pull the image:

docker pull acaicia/minkowski4cpp

Run an interactive GPU-enabled container:

docker run --gpus all --rm -it \
    -v $(pwd):/workspace \
    acaicia/minkowski4cpp \
    bash

This command:

• Exposes available NVIDIA GPUs to the container
• Mounts the current directory at /workspace
• Starts an interactive Bash shell
• Removes the container when it exits

⁠Building Your Own C++ Project

Applications can link directly against the precompiled Minkowski4CPP static library.

⁠Example CMake Configuration

cmake_minimum_required(VERSION 3.18)
project(MinkowskiApplication LANGUAGES CXX CUDA)

set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)

find_package(Torch REQUIRED)
find_package(CUDAToolkit REQUIRED)
find_package(OpenMP)

find_library(MINK_INSTALLED_LIB
    NAMES minkowski_cpp
    HINTS /usr/local/lib
    REQUIRED
)

add_library(minkowski_cpp STATIC IMPORTED)

set_target_properties(minkowski_cpp PROPERTIES
    IMPORTED_LOCATION "${MINK_INSTALLED_LIB}"
)

add_executable(your_target main.cpp)

target_link_libraries(your_target PRIVATE
    minkowski_cpp
    ${TORCH_LIBRARIES}
    CUDA::cudart
    CUDA::cublas
    CUDA::cusparse
)

if(OpenMP_CXX_FOUND)
    target_link_libraries(your_target PRIVATE
        OpenMP::OpenMP_CXX
    )
endif()

Additional include directories and dependencies may be required depending on the APIs used by the application.

⁠Compile the Application

mkdir -p build
cd build

cmake .. -G Ninja \
    -DCMAKE_BUILD_TYPE=Release \
    -DCMAKE_PREFIX_PATH=/opt/libtorch

ninja

The application links against the installed Minkowski4CPP library and its required CUDA and LibTorch dependencies.

⁠Applications

⁠Autonomous Driving and Robotics

Minkowski4CPP can serve as a sparse spatial processing backend for perception systems that combine camera, LiDAR, and other sensor inputs.

Adaptive voxelization enables detailed modeling of nearby objects while reducing the representation cost of distant or less important regions.

Gaussian representations provide an additional mechanism for modeling continuous geometry and spatial uncertainty.

⁠3D Scene Reconstruction

The combination of sparse voxel operations and Gaussian representations supports research into hybrid 3D reconstruction pipelines.

Potential workflows include converting multi-view observations into spatial representations, extracting sparse features, and learning geometric or semantic properties using sparse CNNs.

⁠3D Object Classification and Detection

Sparse convolutional networks can learn features from voxelized 3D objects without performing dense convolution over empty regions.

Adaptive resolution provides an additional mechanism for concentrating model capacity on geometrically complex regions.

⁠Spatial AI and Scene Understanding

Minkowski4CPP can be used as a computational foundation for models that reason about 3D occupancy, object geometry, spatial relationships, and temporal changes.

Its support for 4D sparse data also makes it suitable for spatiotemporal neural network architectures.

⁠Source Code and Docker Image

GitHub Repository

https://github.com/QuiteLiterallyConnor/Minkowski4CPP⁠

Docker Hub

https://hub.docker.com/r/acaicia/minkowski4cpp⁠

The source repository provides the engine implementation, build configuration, and examples.

The Docker image provides a preconfigured GPU development environment for applications built on Minkowski4CPP.

⁠Citation

Minkowski4CPP is based on NVIDIA's MinkowskiEngine. If you use the Minkowski convolutional neural network architecture in your research, please cite the original work:

4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks, CVPR 2019.

@inproceedings{choy20194d,
  title={4D Spatio-Temporal ConvNets:
         Minkowski Convolutional Neural Networks},
  author={Choy, Christopher and
          Gwak, JunYoung and
          Savarese, Silvio},
  booktitle={Proceedings of the IEEE
             Conference on Computer Vision
             and Pattern Recognition},
  pages={3075--3084},
  year={2019}
}

Minkowski4CPP brings native C++ sparse convolution, adaptive mixed-resolution voxelization, and Gaussian spatial representations together in a CUDA-accelerated engine designed for advanced 3D and 4D machine learning workloads.