fnndsc/pl-unstack-folders repository overview

⁠Unstack Folders

pl-unstack-folders is a ChRIS⁠ ds plugin which copies deeply nested subfiles to the base level of the specified output directory.

It is especially useful in ChRIS following "copy" (ts and unextpath) operations, such as pl-dircopy⁠, because the ChRIS backend renames paths to be inconveniently long.

⁠Installation

pl-unstack-folders is a ChRIS⁠ plugin, meaning it can run from either within ChRIS or the command-line.

⁠Local Usage

To get started with local command-line usage, use Apptainer⁠ (a.k.a. Singularity) to run pl-unstack-folders as a container:

singularity exec docker://fnndsc/pl-unstack-folders unstack [--args values...] input/ output/

To print its available options, run:

singularity exec docker://fnndsc/pl-unstack-folders unstack --help

⁠Examples

unstack requires two positional arguments: a directory containing input data, and a directory where to create output data. First, create the input directory and move input data into it.

mkdir incoming/ outgoing/
mv some.dat other.dat incoming/
singularity exec docker://fnndsc/pl-unstack-folders:latest unstack [--args] incoming/ outgoing/

⁠Development

Instructions for developers.

⁠Building

Build a local container image:

docker build -t localhost/fnndsc/pl-unstack-folders .

⁠Running

Mount the source code unstack.py into a container to try out changes without rebuild.

docker run --rm -it --userns=host -u $(id -u):$(id -g) \
    -v $PWD/unstack.py:/usr/local/lib/python3.10/site-packages/unstack.py:ro \
    -v $PWD/in:/incoming:ro -v $PWD/out:/outgoing:rw -w /outgoing \
    localhost/fnndsc/pl-unstack-folders unstack /incoming /outgoing

⁠Testing

Run unit tests using pytest. It's recommended to rebuild the image to ensure that sources are up-to-date. Use the option --build-arg extras_require=dev to install extra dependencies for testing.

docker build -t localhost/fnndsc/pl-unstack-folders:dev --build-arg extras_require=dev .
docker run --rm -it localhost/fnndsc/pl-unstack-folders:dev pytest

⁠Release

Steps for release can be automated by Github Actions⁠. This section is about how to do those steps manually.

⁠Increase Version Number

Increase the version number in setup.py and commit this file.

⁠Push Container Image

Build and push an image tagged by the version. For example, for version 1.2.3:

docker build -t docker.io/fnndsc/pl-unstack-folders:1.2.3 .
docker push docker.io/fnndsc/pl-unstack-folders:1.2.3

⁠Get JSON Representation

Run chris_plugin_info⁠ to produce a JSON description of this plugin, which can be uploaded to a ChRIS Store.

docker run --rm localhost/fnndsc/pl-unstack-folders:dev chris_plugin_info > chris_plugin_info.json