align once and query forever

What is Gravlax?

Gravlax’s recommended workflow builds a compact, molecule-resolved evidence index (.aie) from raw 10x reads and a genome without consulting a gene annotation during alignment or evidence extraction. ingest-archive can also consume other compatible tagged alignments, and root-bound provenance records whether the caller declared an alignment annotation. A compatible GTF supplied at query time drives Gene, GeneFull, and Velocyto-style replay from fixed genome alignments and barcode correction fixed at ingest. Archive-sourced and BAM-sourced Gravlax replay are byte-identical; against a fresh annotation-aware STARsolo run, the deployed archive differs by 0.22–0.45% of UMI mass across four datasets. It occupies 11–18 bits per input read and opens in ~10 ms. Against a function-matched CRAM holding the same post-correction molecule placements and UMI graph, it is 2.10–2.55× smaller and 6.64–13.69× faster to quantify.

The count matrix is one reading of the data under a transient editorial artifact — the gene annotation — and every annotation release silently invalidates it. Gravlax stores the evidence rather than the interpretation, so reanalysis stops being a pipeline and becomes a query.

Fast replay

Quantify compatible annotations, past or future, in seconds — no FASTQ, no realignment. End-to-end deviation from fresh STARsolo is 0.22–0.45% of UMI mass across four evaluated datasets.

Indexed queries

Region, junction, splice-event, molecular-path, and 3′-site queries at cell or group resolution, straight from the index.

Atlas-wide event discovery

Find recurrent molecule-supported junctions, splice patterns, and terminal-tail events across donors and cell groups without supplying coordinates or remapping reads.

Compare annotations

Replay two releases independently on fixed evidence and report exact signed count changes, changed-class states, non-exclusive causes, and bounded witnesses within the retained archive quotient.

Projects and plans

Name resources once, check assembly and biological intent before a run, preserve a content-addressed resolved plan, and resume only after exact identities are verified.

Discover → replay

Find unannotated loci from the index alone, emit them as a GTF, and re-quantify them with the full replay machinery. Bounded residual sites raise complete-denominator recall from 44.7% to 72.5%.

EM multimapper recovery

Packed, cell-sharded cross-cell EM over the stored paralog evidence recovers the multi-gene molecules per-cell quantifiers discard.

Federation

One query across N indexes — the atlas access pattern at molecule resolution, with per-sample, per-cell answers.

Small and fast

11–18 bits per input read. The current files are 32–49× smaller than FASTQ and 9–13× smaller than information-richer, tag-preserving CRAM 3.1. They remain 2.10–2.55× smaller than function-matched post-correction CRAM. Open cost is ~10 ms.

The workflow at a glance

# Check the installation, then align annotation-free
# (STAR two-pass, no GTF, secondaries kept) → BAM:
aie doctor
aie ingest recipe --chemistry 10x-3p-v3
aie ingest check align.bam --whitelist 3M-february-2018.txt --chemistry 10x-3p-v3

# Build and compile once:
aie ingest-archive align.bam --whitelist 3M-february-2018.txt --out sample.aie
aie compile-annotation gencode.v49.gtf --out gencode.v49.aic

# Query and reinterpret forever:
aie replay-rows sample.aie --gtf gencode.v49.aic --barcodes barcodes.tsv --out-dir counts/
aie resolve gencode.v49.aic TP53 --assembly GRCh38.p14 --annotation "GENCODE 49"
aie query sample.aie region chr1:1000000-2000000 --format json
aie query sample.aie junction chr2:1234567-1250000 --format json
aie compare-annotations sample.aie --annotation-a gencode.v44.gtf \
  --annotation-b gencode.v49.aic --assembly GRCh38.p14 \
  --annotation-a-label "GENCODE 44" --annotation-b-label "GENCODE 49"
aie query sample.aie junctions chr2:1200000-1300000 --with-cells --format tsv
aie federate a.aie b.aie c.aie chr1:155234452-155235327 --format json
aie dev em sample.aie --gtf gencode.v49.gtf

See the quick start for the full walkthrough, and the capabilities guide for what the index can do that a count matrix cannot. For repeatable analyses, the workflow and interfaces guide connects projects, plans, safe identifier resolution, Explorer, typed Python results, and the uniform-output contract.

Citation

Manuscript in preparation. The GitHub repository is the reference implementation.