Sampling and training datasets
Export aligned measurements, loss flags and labels for decoding or training.
Three paths, depending on what comes next
Raw measurement shots
Use circuit sample. Start with readable 01; switch to compact b8 for volume.
Detector rows
Use dataset export --mode detectors. Publish detector inputs and keep observable answers private.
Fully labeled simulator data
Blinding randomizes the logical input with a private mask so public measurements do not reveal the scoring label. Add --error-trace only when training or debugging also needs realized physical error events.
1. Generate and sample a loss-visible circuit
Generate a distance-3, two-round Mid-SWAP Z-memory circuit. Mid-SWAP exchanges data/check roles during the checks so readout can reveal loss; each loss-visible measurement stores a flag,value pair.
rstim circuit gen \
--code surface_code \
--task rotated_memory_z_midswap \
--distance 3 --rounds 2 \
--after-clifford-depolarization 0.002 \
--operation-loss-probability 0.002 \
--measurement-loss-probability 0.003 \
--out circuit.stim
rstim circuit sample \
--in circuit.stim --shots 4 --seed 7 \
--out raw-shots.01 --out-format 01raw-shots.01 contains one ASCII bit row per shot. Use these rows for debugging. The dataset export below adds the metadata and labels needed for training.
2. Export public inputs and private labels
Use detectors for ordinary detector-input benchmarks without erased records. For atom loss, prefer measurements_blinded: it retains loss flags and avoids declaring an arbitrary lost-value parity to be valid syndrome.
| Mode | Public row | Private truth | Use when |
|---|---|---|---|
detectors | Detector bits | Observable flips | No referenced measurement is lost, or the result is explicitly a legacy baseline |
measurements_blinded | Measurement records | Unmasked logical-error target + source mask | Training, blind evaluation, and loss-aware decoding |
For the generated d=3 Mid-SWAP example, 1,8,15 is a verified logical-X representative. Do not copy that support to a different circuit: the exporter checks that the requested Pauli preserves detector references and flips observable 0.
mkdir -p data
rstim dataset export \
--circuit circuit.stim --shots 100000 --seed 7 \
--mode measurements_blinded \
--logical-x-qubits 1,8,15 \
--public-out data/public \
--private-out data/private \
--error-tracedata/public/
Decoder input
Shareable circuit, manifest, and measurement rows for the decoder.
manifest.json # shape, hashes, dataset id
circuit.stim # public logical-zero circuit
shots.b8 # one measurement row per shotdata/private/
Scoring and training
Answer labels, masks, seed, and optional error trace for scoring or training.
manifest.json # seed and private hashes
answers.b8 # physical logical-error target
masks.b8 # injected logical input bit
trace.jsonl # realized errors, one shot/lineThe public manifest deliberately contains no seed, answers, masks, private path, or producer-circuit choice. Pair the two bundles by their shared dataset_id, and never publish the private directory for a blind evaluation.
Omit --error-trace for faster, smaller exports when answers.b8, masks.b8, and public loss flags are enough. The checked loader below intentionally verifies trace alignment and therefore requires the flag. Traced sampling is shot-by-shot; enabling it changes RNG consumption, so the same seed produces a different batch than an untraced export.
How b8 stores each row
b8 packs bits least-significant-bit first inside each byte (bit_order: lsb_first), concatenates fixed-width rows in shot order, and requires unused padding bits to be zero.
Row size
bytes_per_shot = ceil(bits_per_shot / 8)
The manifest records both values; use them instead of guessing from the file size.
Verified example
The example circuit has 50 record bits: 7 bytes per shot. Its 100,000-shot export writes 700,000 bytes to shots.b8.
# Decode bit k of shot s
row_bytes = (bits_per_shot + 7) // 8
bit = (raw[s * row_bytes + k // 8] >> (k % 8)) & 1Keep loss separate from the stored value
Loss-visible measurement (ML) and measurement-plus-reset (MRL) instructions store loss_flag,value_bit. Preserve the complete packed row, then deinterleave each pair into value and mask channels. A lost value may be stored as 1, but that is only a placeholder—not a physical eigenvalue and not valid syndrome evidence.
| Tensor | Shape | Source and role |
|---|---|---|
record_bits | [shots, row.bits] | Directly unpacked public shots.b8 |
measurement_bits | [shots, values] | The value_bit from each loss-visible pair |
measurement_loss_mask | [shots, values] | The paired loss_flag; a separate model input channel |
raw_detector_bits | [shots, detectors] | Optional derived feature, never sufficient by itself under loss |
detector_valid | [shots, detectors] | Derived loss-aware mask for detectors untouched by lost records |
answer_targets | [shots, 1] | Private answers.b8: supervised physical logical-error label |
logical_masks | [shots, 1] | Private masks.b8: training metadata, never decoder input in blind evaluation |
Do not feed matching a detector parity computed by replacing every lost measurement with 0 or 1. Invalidate affected checks or combine them into loss-aware superchecks; A supercheck combines affected parity checks to cancel the missing values. Changing a lost-value placeholder must not change the decoder answer.
Put the hidden logical flip before the first noise
R 1 3 5 8 10 12 15 17 19
TICK[rstim:logical_flip_point]
X_ERROR(0.002) 1 3 5 8 10 12 15 17 19The circuit must contain exactly one top-level TICK[rstim:logical_flip_point], after ideal state preparation and before every positive-probability noise instruction, including noise inside completed REPEAT blocks. The old comment marker is intentionally unsupported.
3. Load the training data
The checked standard-library loader validates manifests, dataset identity, row sizes, trace indices, logical masks, and the private answer relation before returning fixed-width Python lists ready for NumPy, PyTorch, or JAX. Because it verifies logical_input metadata, this command requires a dataset exported with --error-trace.
Download load_blinded_training_data.py into your working directory, then run it with Python 3. This optional loader prepares training inputs; sampling and decoding use rstim.
python3 load_blinded_training_data.py \
--public-dir data/public \
--private-dir data/private \
--observable-rec -17 \
--observable-rec -15 \
--observable-rec -13Those three record offsets belong to the generated d=3 Mid-SWAP example. Read OBSERVABLE_INCLUDE(0) from your own circuit when using a different layout.
trace.jsonl is ragged by design. Keep it as an auditable sidecar, or encode events by operation path into fixed-width features only when auxiliary physical-error supervision is useful.