Sample and dataset formats
Choose the stream by its meaning
| Stream | Width per shot | Contents |
|---|---|---|
| Measurement samples | Circuit measurement count | Readout bits, including paired loss flags where present |
| Detector samples | Detector count | Parities relative to the noiseless reference |
| Observable answers/predictions | Observable count | Private actual flips or decoder predictions |
--append-observables concatenates detector bits then observable bits into one row. Keep observables in a separate --obs-out file for decoder evaluation. A raw measurement row cannot be used as a detector row; model dimensions and stream type must agree.
01 and b8
01 is text: one character per bit and one line per shot. b8 is binary: fixed-width, byte-aligned rows in shot order, least-significant bit first within each byte. Row stride is (bits_per_shot + 7) // 8. Unused high bits in the last byte must be zero. There is no embedded shape header; preserve the circuit or manifest.
For a row with logical bits [1,0,1], b8 contains byte 0x05. Two three-bit rows take two bytes, not one packed six-bit stream. Empty-width data cannot reveal the shot count from byte length; callers must preserve it explicitly. The CLI help lists other formats supported by each command; not every command accepts the same encodings.
QASM semantic documents and result views
rstim.qasm2-program.v1 JSON stores checked operations, declared registers and
source provenance for QASM import/export. It is separate from native Stim text,
QP101 drawing data and shot encodings. Use circuit sample --in-format qasm2-json
to sample a document with an explicit backend.
The default --result-view measurement-record retains every measurement in
execution order. --result-view classical writes final classical-register values
in declaration order, then increasing bit index; overwritten results are replaced
and unwritten bits stay zero. Integer conditions interpret bit 0 as the least
significant bit. The same 01 or b8 encoding can therefore have different row
widths depending on the selected view. Preserve the document and result-view
choice alongside the shots.
The QASM feedback example demonstrates two stored measurements and a single final classical bit. These shots do not carry detector or observable semantics; QASM documents are not direct DEM/decoder input.
Loss records and derived features
For each loss-visible readout, the flag is stored before the value. Preserve both bits, retain their record indices, and deinterleave them into distinct value/mask channels for training. A lost value is a placeholder. Raw detector parities involving it are not reliable syndrome evidence. The training tutorial gives tensor shapes and a checked loader.
Dataset bundles
A public bundle contains manifest.json, circuit.stim and shots.b8. A private bundle contains its manifest and answers.b8, plus masks.b8 for blinded measurements and optional trace.jsonl. Pair manifests by dataset_id, verify declared hashes and shapes, and preserve shot order. rstim dataset import validates imported bundles; native loss decode additionally checks its accepted subset before compiling.
detectors mode publishes detector rows. measurements_blinded publishes measurement rows from a privately randomized logical input; its private answer is the public observable parity XOR the hidden source bit. Score predictions directly against answers.b8; applying the mask again changes the scoring target. The public manifest excludes the seed, private paths, answers and masks.
Trace contracts and loss logs
Simulator traces (rstim.sample_trace.v1) and per-shot dataset error traces (rstim.error-trace.v1) are different schemas. A trace can reveal realized physical errors and private logical input metadata; keep it private for blind evaluation. Loss-log ordinals identify loss-visible readouts, not arbitrary measurement-bit indices. Loaders must validate shot and readout alignment rather than zip unrelated files.
See error-trace v1 and loss-log v1 for fields and validation rules.
Dataset export contract
rstim export_decoder_dataset creates two directory bundles: a public bundle for decoder contestants and a private bundle for scoring.
Detector Mode
rstim export_decoder_dataset \
--circuit memory.stim \
--shots 100000 \
--batch_shots 10000 \
--mode detectors \
--public_out public-data \
--private_out private-truth
The public bundle contains detector-event rows in shots.b8 and the logical-zero circuit.stim. The private bundle contains answers.b8.
Contestants decode each public row to one predicted observable bit. Scoring compares each prediction with the corresponding private answer bit.
Detector mode rejects both --logical_x_qubits and --logical_z_qubits.
Bounded-memory export
The exporter generates and writes shots in bounded batches instead of keeping
the full dataset in memory. --batch_shots controls the maximum number of
shots held by one generation batch and defaults to 10000; lowering it reduces
peak memory at the cost of more sampling calls. The selected value is recorded
in the private manifest so seeded exports remain reproducible.
Output files and their SHA-256 digests are written incrementally. Peak dataset
memory therefore scales with --batch_shots, not the total value of --shots.
Optional per-shot error trace
--error_trace additionally writes trace.jsonl into the private bundle and
a trace_file entry into the private manifest. Each line records the complete
noise realization behind one shot — every Pauli branch that fired and every
LOSS onset — in the versioned rstim.error-trace.v1 schema (see
docs/specs/error-trace-v1.md). Traced export samples shot by shot, so it is
slower than batch sampling and produces a different batch than an untraced
export with the same seed; within one traced export the trace, shots, answers,
and masks always describe the same executions, and a fixed seed remains
byte-for-byte reproducible.
Blinded Measurement Mode
The generated Mid-SWAP memory-Z circuit carries both the insertion point and a default logical-X representative derived from its distance and initial data-qubit layout. For distance 3, its marker is:
TICK[rstim:logical_flip_point] # rstim:logical_input=X:1,8,15
The inline comment declares the operator X and its physical support, not the
random logical input bit. The default --mode auto selects
measurements_blinded when this metadata or an explicit logical-qubit option
is present, and detectors otherwise. For a generated Mid-SWAP circuit, run:
rstim dataset export \
--circuit circuit.stim --shots 64 --seed 7 \
--public-out data/public --private-out data/private
The same default applies to the direct rstim export_decoder_dataset command.
In blinded mode, explicit --logical_x_qubits or --logical_z_qubits options
override the declared default; the chosen operator is still checked noiselessly
against every detector and observable 0. Explicit --mode detectors ignores
the default operator and rejects explicit logical-qubit options as before.
The exported manifests always record the resolved mode, never auto.
For a custom circuit, retain the inline metadata on its canonical marker,
or supply the logical-qubit option explicitly as described below. The
supported metadata spellings are rstim:logical_input=X:qubits and
rstim:logical_input=Z:qubits, with a comma-separated, nonempty list of unique
physical qubit indices. In auto or blinded mode, misplaced, duplicate or malformed
declarations are rejected before export. Comment-stripping tools remove this metadata, so
explicit support is required after such a transformation.
Place the canonical tagged instruction exactly once at top level, after ideal logical-state initialization and before the first positive-probability noise instruction anywhere in the circuit:
R 0 1 2
TICK[rstim:logical_flip_point]
The tag must annotate TICK and must not occur inside a REPEAT block. The
legacy # RSTIM_LOGICAL_FLIP_POINT comment is not recognized. The exporter
rejects any positive-probability Pauli, loss, correlated-error, or measurement
noise before the marker, including noise nested in a completed REPEAT block.
Zero-probability noise instructions are allowed. Noise after the marker,
including physical loss immediately before a loss-visible measurement, remains
part of the sampled circuit.
rstim export_decoder_dataset \
--circuit memory.stim \
--shots 100000 \
--mode measurements_blinded \
--logical_x_qubits 0,2,4 \
--public_out public-data \
--private_out private-truth
Without circuit metadata, blinded mode requires exactly one logical-qubit option. Use --logical_x_qubits for an ideal physical X representative, such as a logical X for a Z-basis memory experiment. Use --logical_z_qubits for an ideal physical Z representative, such as a logical Z for an X-basis memory experiment:
rstim export_decoder_dataset \
--circuit memory-x.stim \
--shots 100000 \
--mode measurements_blinded \
--logical_z_qubits 1,7,13 \
--public_out public-data \
--private_out private-truth
For each shot, the exporter privately chooses a bit b, samples either the public circuit or the circuit with the requested ideal X or Z, publishes the measurement row, and stores answer = O_public(m) XOR b privately. Before exporting, it verifies noiselessly that the injected Pauli preserves every detector value and flips observable 0.
With --error_trace, each private trace line also contains the training label
that selected the producer circuit:
"logical_input":{"bit":1,"applied":true,"pauli":"X","support":[0,2,4]}
bit is exactly the aligned masks.b8 bit; applied is the same value and
states whether the ideal Pauli gate was inserted for that shot. pauli and
support describe the configured representative. This object is omitted in
detector mode, and the intentional X or Z gate is never recorded as a
physical noise event. See doc/examples/load_blinded_training_data.py for a
small loader that aligns shots, answers, masks, and trace rows.
Here O_public(m) is the observable computed from the published measurement row. Contestants must decode the underlying unmasked logical-error bit from each measurement row (or its derived syndrome); they must not submit the directly recomputed public observable O_public(m). The organizer's private scoring key remains answer = O_public(m) XOR b, while masks.b8 retains the private per-shot b values used to produce that key.
Equivalently, O_public(m) combines the intentional source bit b with any
logical flip accumulated from physical errors. XORing out b makes answer
the supervised physical logical-error target rather than the randomized source
state itself.
Files
Public files are exactly manifest.json, circuit.stim, and shots.b8. Private files are exactly manifest.json, answers.b8, and, in blinded measurement mode, masks.b8, plus trace.jsonl when --error_trace is on.
The manifests describe their respective bundle files and associate the public and private bundles with the same dataset. Publish only the public directory; retain the private directory for the scoring authority.