Workflow and interfaces

Direct commands cover the full supported command tree and are convenient for one-off work. For the operation subset represented by source-plan v1 step kinds, projects add stable resource names, checked plans, exact provenance, and identity-verified resume. The Explorer and Python client use the same resolver, plan, and result contracts; they do not reimplement archive semantics. A project plan does not imply a second entry point for every direct CLI command.

Choose an entry point

Need	Start here
Try one query against paths you already know	A direct aie query or aie replay-rows command
Diagnose an installation or input	aie doctor
Prepare a correct annotation-free ingest	aie ingest recipe and aie ingest check
Maintain a repeatable multi-step analysis	Projects and plans
Resolve a gene, transcript, or exon safely	aie resolve
Build a query visually without running it	aie explore
Run plans or load typed results in a notebook	Python and AnnData

Shell completions are generated by the installed binary, so they always match its command tree. Development and evaluation instruments are grouped beneath aie dev; their hidden former top-level spellings exist only as deprecated compatibility aliases.

A portable project

Create a project once, register inputs by role, and refer to their names from plans:

aie project init pbmc-analysis --name pbmc-analysis
cd pbmc-analysis
aie project add sample data/sample.aie --kind archive --assembly GRCh38.p14
aie project add genes references/gencode.v49.aic --kind annotation \
  --assembly GRCh38.p14 --annotation-label "GENCODE 49"
aie project add cells metadata/barcodes.tsv --kind barcodes

Ordinary resources are stored as project-relative paths and remain movable. An intentionally shared, read-only input outside the project must be marked --external; its absolute dependency is then visible in the manifest and resolved plan. Paths cannot escape the project through an unmarked symlink.

Source plans use schema version 1 and may be YAML or JSON. They name a scientific operation and its inputs without introducing a second implementation of that operation:

schema_version: 1
name: replay-gencode-v49
steps:
  - id: quantify
    kind: replay-rows
    archive: sample
    annotation: genes
    barcodes: cells
    out_dir: results/counts-v49

The uniform interface is additive for the source-plan v1 step kinds that expose it. Supported query, cohort, federation, collection, and archive-inspection steps opt in with uniform_output: {format: json, output: results/example.json}. Plan steps with a distinct operation-report surface use the parallel uniform_report object. Omitting these objects preserves the older command-specific format/output behavior and canonical argument vector. Resolved-plan v6 exposes the selected mode, format, publication boundary, destination, and all output schema IDs explicitly.

Collection region, junction and jset plan steps also accept an optional locations: <resource-id>. Register the location manifest as project metadata; the plan binds its content and passes its resolved path to --locations. Entries are paths relative to that manifest, keyed by expected archive or parent-collection roots. Relocation does not rewrite the collection. The location manifest is part of execution provenance, so changing it intentionally changes the resolved plan.

Check the plan before it reads evidence:

aie plan check plans/replay.yaml --explain

The checker resolves every named input and biological identifier, verifies known assembly compatibility, reports exact coordinates and defaults, names the output schemas, and gives only those I/O bounds it can prove. An unavailable bound is reported as unavailable rather than estimated as exact. --json emits the complete resolved-plan document.

Exact dataflow and resume

A later step can consume a typed output from an earlier declaration with step:<id> when there is only one output or step:<id>:<output-name> when the producer has several. The checker rejects forward references, cycles, ambiguous outputs, and role mismatches before execution.

aie plan run writes a content-addressed resolved snapshot beneath .aie/resolved-plans/ before invoking the existing command implementations. Outputs are first written to recorded staging locations and installed without overwriting an existing destination.

Resume is explicit:

aie plan run plans/replay.yaml --resume

A step is skipped only when its resolved-plan and step digests, producer executable, upstream output identities, and freshly observed final output identities all match its versioned completion record. A merely present file is not treated as a completed step. Missing, partial, modified, or unrecorded outputs stop resume with an error. Keep the project’s .aie/ directory: it contains the resolved snapshots and completion evidence, not molecule archives.

Biological intent and annotation identity

The shared identifier resolver accepts exact gene symbols and stable gene, transcript, or exon IDs. Ambiguity never selects the first match; qualify a request as gene:, transcript:, or exon: when necessary. Every successful resolution carries its zero-based, half-open locus, strand, assembly, annotation label, and observed annotation digest.

Compiled annotation format v2 retains the exact transcript and source exon identifiers needed for these queries while keeping replay’s merged assignment exons separate. Legacy AIC v1 files remain readable for replay and gene resolution, but transcript or exon resolution fails with an explicit request to recompile the source GTF. Gravlax does not invent identifiers absent from an old artifact.

In direct path-based commands, an --assembly value is a caller assertion recorded in provenance. A checked project can additionally compare it with registered annotation, archive, collection, and genome metadata. If the necessary coordinate identity is missing, compatibility is reported as unverified rather than inferred from chromosome names.

Explorer: inspect and export, never execute

Run aie explore inside a project to open the loopback-only interface. It can browse registered resources, source plans, exact resolved snapshots, and results. Its scientific builder currently covers annotation comparison, region, splice-event, exact-junction, junction-set, and terminal-boundary questions. It exports synchronized plan YAML, plan JSON, CLI arguments, and a Python snippet.

Explorer is deliberately read-only. It accepts only local GET and HEAD requests, cannot save or execute a plan, rejects paths and symlinks outside its known project locations, and binds only to 127.0.0.1. Use an SSH loopback tunnel when the project lives on a remote machine; do not expose its port.

Typed results and the current boundary

The shared outer JSON contract is gravlax.result-envelope.v1. It carries a versioned result schema, producer identity, provenance, warnings, and typed tables. The Python client validates these fields and can convert compatible tables to pandas, PyArrow, or row-oriented AnnData. Shared MEX readers validate the completion marker, matrix dimensions, labels, coordinates, and provenance before constructing SciPy or AnnData objects.

Uniform output is currently deployed at the direct CLI boundary for:

• aie resolve;
• aie compare-annotations;
• aie query … transcript-ecs;
• the other single-archive query surfaces, aie federate, and cohort event, splice-graph, transcript-end, and PolyASite analyses;
• every collection build, inspection, and query surface;
• archive lifecycle reports for ingest-archive, replay-rows, stamp-genome, and seal-archive through --report-format (the primary archive or MEX path remains separate);
• inspect-archive through --format; and
• operation reports for compile-annotation, export-molecule-bam, and extend.

aie dev em reports several purpose-built evaluation measures and artifact modes. It does not expose the shared typed-result format.

Historical --json and --tsv modes remain command-specific so existing scripts keep byte compatibility; select --format or, on commands that expose a separate operation report, --report-format for a uniform result. Use Python’s result_raw() for a historical format and run_to_file() to keep any large stdout stream out of memory. The core supports bounded Arrow batches, but the CLI does not yet generally emit Arrow IPC. There is no R client yet. These are explicit boundaries, not implied capabilities.

New scientific workflows

aie compare-annotations independently replays annotations A and B against one fixed archive, then reports signed per-cell/gene count deltas, complete changed-class transitions, non-exclusive causes, and bounded witnesses. Its exactness is relative to the retained archive quotient and fixed alignment/barcode policy, not to a fresh annotation-aware realignment. The current command does not directly aggregate comparison rows by group.

aie query … transcript-ecs derives annotation-conditional compatible-transcript sets for archived UMI classes. It does not estimate transcript abundance, call isoforms, or phase complete transcripts. Its equivalence classes describe retained archive evidence and can differ from classes derived from every original read, especially for fragmented 3′ protocols.

Distribution

Versioned tags use cargo-dist to build native archives and installers for Linux, macOS, and Windows. The same release publishes the Rust packages to crates.io and gravlax-client to PyPI, and attaches checksums, an SPDX software bill of materials, and a vendored source archive to GitHub Releases. Bioconda availability follows review of its release recipe. See releases and distribution for installation and maintainer instructions.

For a small offline exercise of the project, plan, resume, and Explorer paths, use examples/demo-project from the repository checkout.