bhagesh/cycondor repository overview

⁠Short description

cyCONDOR 0.3.1 in a pinned R 4.4 image, with a compensate-and-QC-gate wrapper for whole blood.

⁠Full description

⁠cycondor

cyCONDOR⁠ 0.3.1 in a reproducible container, wrapped in a three-stage pipeline that solves the two things a vanilla prep_fcd() call does not handle on ungated whole-blood acquisitions: compensation and a leukocyte gate.

Unsupervised flow-cytometry analysis: load and transform → PCA → UMAP → FlowSOM clustering → figures, tables and per-cluster frequency statistics.

• cyCONDOR upstream: https://github.com/lorenzobonaguro/cyCONDOR⁠
• Licence: cyCONDOR is third-party; see its repository for terms. The wrapper scripts in this image are covered by the repository they come from.

⁠What this image adds

Three things, and they are the reason it exists rather than a bare R image.

1. Compensation. prep_fcd() reads with flowCore and goes straight to the transform; it has no compensation step. Acquisitions that carry a $SPILL keyword are therefore not compensated by cyCONDOR, and spectral bleed-through drives the embedding and the clustering. The staging step applies each file's own spillover matrix, then drops the keyword so nothing can compensate twice. Files whose matrix is missing, non-square, mismatched or singular are staged uncompensated and flagged in a manifest rather than silently mis-corrected.

2. A QC gate. cyCONDOR embeds every event it is handed. On ungated whole blood most events are not leukocytes, and without a gate one CD45-negative cluster of red cells and debris dominates and flattens everything else. The gate here works on clusters, not events: cluster coarsely, then judge each cluster by its median marker profile. The debris/leukocyte and live/dead splits come from 2-means on those medians, so no cutoff is hard-coded. A cluster that is top-percentile in most lineage markers at once is an aggregate, not a cell type, and goes too. Typical retention on whole blood is 10–18%.

3. Flattening and subsampling. prep_fcd(data_path=) reads one directory. Acquisitions usually live in several, with spaces and parentheses in the names. Staging materialises them flat, subsampled and seeded per file, so adding one file does not reshuffle the others.

It also sets truncate_max_range = TRUE on the raw read, which is load-bearing. With $DATATYPE F the acquisition software may write floats above the $PnR it declared, and auto-logicle takes its scale from t = max(channel) over the pooled data — two dozen out-of-range events in four million are enough to stall the logicle root-finder outright.

⁠Quick start

The image expects two mounts: /data read-only, and /results for everything it produces.

docker pull bhagesh/cycondor:0.3.1

# every flag each stage accepts
docker run --rm bhagesh/cycondor:0.3.1 help

# all three stages in order
docker run --rm \
  -v "$PWD/data:/data:ro" \
  -v "$PWD/results:/results" \
  bhagesh/cycondor:0.3.1 all

Or one stage at a time — stage 2 is idempotent, so re-running the analysis at a different depth costs only stage 3:

... bhagesh/cycondor:0.3.1 metadata   # build the annotation table
... bhagesh/cycondor:0.3.1 stage      # flatten + compensate
... bhagesh/cycondor:0.3.1 analysis   # prep_fcd -> PCA -> UMAP -> FlowSOM

Common variations:

# a fast end-to-end check
... bhagesh/cycondor:0.3.1 all --max-cell 1000

# deeper, more threads for the UMAP
... bhagesh/cycondor:0.3.1 analysis --max-cell 40000 --threads 8

# re-stage from scratch
... bhagesh/cycondor:0.3.1 stage --force

# a shell, or an R session with cyCONDOR loaded
... bhagesh/cycondor:0.3.1 bash
... bhagesh/cycondor:0.3.1 R

On Windows PowerShell use ${PWD} for $PWD; on Git Bash prefix the command with MSYS_NO_PATHCONV=1, or the shell rewrites /data into a Windows path.

Run one container at a time. The stages are already multi-threaded, so nothing is gained by running two, and a run killed by the out-of-memory killer writes nothing at all.

⁠Configuring it

Everything the pipeline decides lives in one commented YAML — which files are read, how they are compensated, which markers drive the embedding, how many clusters. Nothing is set in the R scripts.

# the baked-in config, as a starting point
docker run --rm --entrypoint cat bhagesh/cycondor:0.3.1 \
  /opt/cycondor/pipeline/config.yaml > config.yaml

# edit, then point the run at it
docker run --rm -v "$PWD/data:/data:ro" -v "$PWD/results:/results" \
  -v "$PWD/config.yaml:/cfg.yaml:ro" \
  bhagesh/cycondor:0.3.1 all --config /cfg.yaml

The values worth knowing: staging.mode (compensate / copy / link), staging.truncate_max_range, prep.transformation (auto_logi is the recommendation for conventional flow; arcsinh compresses the bulk against zero and merges clusters), qc.enabled, markers.lineage, and clustering.flowsom.n_clusters.

The whole pipeline can also be run from a mounted working tree without rebuilding, which is how to iterate on the scripts:

docker run --rm -v "$PWD/data:/data:ro" -v "$PWD/results:/results" \
  -v "$PWD/testcycondor:/pipeline:ro" -e CYCONDOR_PIPELINE_DIR=/pipeline \
  bhagesh/cycondor:0.3.1 analysis

⁠What comes out

results/
├── metadata/       the annotation table prep_fcd() expects
├── staged_fcs/     flattened, compensated + staging_manifest.csv
└── analysis/
    ├── condor.rds        the full object, reload with readRDS()
    ├── run_record.yaml   every parameter, seed and version this run used
    ├── figures/          UMAPs by cluster / group / sample, marker UMAPs,
    │                     per-marker UMAPs faceted by group, marker heatmap,
    │                     frequency barplots and boxplots
    └── tables/           qc_clusters.csv, cells_umap.csv, per-sample counts
                          and frequencies, Kruskal-Wallis + Dunn post-hoc

Read tables/qc_clusters.csv first. It records every cluster, its size, its marker medians and why it was kept or dropped. It is the one table that would show the gate going wrong, and a run whose gate is wrong produces internally consistent numbers that are unusable. The pipeline refuses to continue if the gate would keep less than qc.min_retained_fraction, which would mean the markers are mis-named rather than the data being bad.

Then figures/marker_heatmap_FlowSOM.png, which is what tells you what the numbered clusters are — FlowSOM returns 1…N with no names.

⚠️ Cluster numbers are not stable between runs. FlowSOM renumbers, so the same population can be cluster 17 at one sampling depth and 19 at another. Identify a cluster by its marker profile, never by its number.

⁠Reproducibility

The CRAN pin matters more than it looks: the same seed on a different uwot gives a visually similar but numerically different embedding.

Two GitHub-only dependencies (Rphenograph, Rphenoannoy) install from their default branches, as cyCONDOR's own DESCRIPTION specifies. They are the one part of the environment that is not version-pinned.

Rphenograph is present but off by default — it builds a k-nearest-neighbour graph over every cell, which is slow and memory-hungry at these cell counts. Enable it deliberately.

⁠A note on interpretation

This is an unsupervised tool: it finds structure without being told what to look for, and it returns numbered clusters you then have to identify. That is its strength — it can surface a population nobody thought to declare — and its limit: the number of clusters is a setting, and the number of significant results moves with how many cells you draw. Report the sampling depth alongside any finding.

Nothing here measures whether a batch effect is confounded with your study groups. If your groups were acquired on separate days, check that before believing any between-group result.