paularg/visualise_3d repository overview

3D Visualisation tool based on VTK and PyVista for assessing 3D pose prediction. Github: https://github.com/PaulaRamirezGilliland/3D_Visualisation_SVORT⁠

⁠Interactive Display

An interactive display as depicted below is generated. To navigate thorugh it, keys "j" - up one slice, "k" - down one slice, right click on any 2D slice to highlight it. Zoom in and out by scrolling on mouse/hold right click and zoom. The different windows showcase:

1. The Ground Truth 2D echocardiography image (pre-aligned, no transformation applied)
2. The predicted transformations applied to the 2D echocardiography image (transforms_gt + inv(transforms_pred))
3. The overlap of these two, in different colourmaps
4. The corresponding closest axial slice in the STIC image (3D acquisition)
5. The corresponding closest axial slice in the STIC image (3D acquisition) with slight opacity for all other slices, to showcase the full volume

⁠Loading and applying saved transforms to Slicer

If convert_transform is True, two files (transformation and image) for each slice are saved within path_save directory. These can be loaded into 3D Slicer.

To apply the transformation to each slice in 3D Slicer, go to "Transforms" module, select relevant volume and transformation and click "Apply". The resultant transformed volume can then be plotted in 3D using the "Volume Rendering" module, or setting visibility for the transformed slice.

⁠Running the container

1. Pull the docker image

   docker pull paularg/visualise_3d

2. Prepare input data:

   Place all required data within a folder (it can be in subfolders). There should be a 2D pre-aligned echocardiography image (3D volume), 3D STIC, model output for transforms (transforms.npy and transforms_gt.npy).

   Additionally, a config.yaml file is required, which contains paths to the dataset and transforms (from within the folder containing the config.yaml file). An example of config.yaml is:

   2D_volume_path: /case1/r2-reg-2d-3d-masked-1.nii.gz # Real 2D echo path STIC_path: /case1/reg-IM_0170-ED-iso-1.nii.gz # STIC path mask_path: /case1/Repeat_mask-ED-iso-1.nii.gz # Masks used to generate the transformation predictions
   transforms_path: /case1/ # Folder containing transforms.npy and transforms_gt.npy convert_transform: False # If True, it converts the composed transformation matrices (GT + inverse of Predicted) to a format readable by 3D Slicer (.tfm), and saves it in the specified directory (per slice), as well as volume with corresponding slice (.nii). path_save: ./ #directory to save the converted transformation and slices. Only used if convert_transform is True.

3. Run the container with interactive display

   docker run -it --env DISPLAY=$DISPLAY --volume /tmp/.X11-unix:/tmp/.X11-unix --volume "path_to_your_config_and_dataset:/data" visualise_3d python main.py --config /data/config.yaml