Sampling and training datasets
Export aligned measurements, loss flags and labels for decoding or training.
Three paths, depending on what comes next
Raw measurement shots
Use circuit sample. Start with readable 01; switch to compact b8 for volume.
Detector rows
Use dataset export --mode detectors. Publish detector inputs and keep observable answers private.
Fully labeled simulator data
Blinding randomizes the logical input with a private mask so public measurements do not reveal the scoring label.
1. Generate and sample a loss-visible circuit
Generate a distance-3, two-round Mid-SWAP Z-memory circuit. Mid-SWAP exchanges data/check roles during the checks so readout can reveal loss; each loss-visible measurement stores a flag,value pair.
rstim circuit gen \
--code surface_code \
--task rotated_memory_z_midswap \
--distance 3 --rounds 2 \
--after-clifford-depolarization 0.002 \
--operation-loss-probability 0.002 \
--measurement-loss-probability 0.003 \
--out circuit.stim
rstim circuit sample \
--in circuit.stim --shots 4 --seed 7 \
--out raw-shots.01 --out-format 01
raw-shots.01 saves one row of ASCII measurement bits per shot.
2. Export public inputs and private labels
Use detectors for ordinary detector-input benchmarks without erased records. For atom loss, prefer measurements_blinded: it retains loss flags and avoids declaring an arbitrary lost-value parity to be valid syndrome.
| Mode | Public row | Private truth | Use when |
|---|---|---|---|
detectors | Detector bits | Observable flips | No referenced measurement is lost, or the result is explicitly a legacy baseline |
measurements_blinded | Measurement records | Unmasked logical-error target + source mask | Training, blind evaluation, and loss-aware decoding |
The generated circuit records its logical-input insertion point and logical-X qubits. The exporter reads this metadata and selects blinded measurements automatically.
mkdir -p data
rstim dataset export \
--circuit circuit.stim --shots 100000 --seed 7 \
--public-out data/public \
--private-out data/private \
--error-trace
| Directory | File | Contents |
|---|---|---|
data/public/ | manifest.json | Row shape, hashes and dataset ID |
circuit.stim | Public logical-zero circuit | |
shots.b8 | One packed measurement row per shot | |
data/private/ | manifest.json | Seed and private file hashes |
answers.b8 | Physical logical-error targets | |
masks.b8 | Injected logical input bits | |
trace.jsonl | Realized error events, one shot per line |
The decoder uses data/public/; data/private/ is for scoring and training. The public manifest contains no seed, answers, masks, private path, or producer-circuit choice. Pair the bundles by their shared dataset_id, and keep the private directory private for blind evaluation.
Optional --error-trace records per-shot physical errors for training or debugging and is required by the loader below; it increases export time and size, and traced and untraced exports produce different batches even with the same seed.
How b8 stores each row
b8 packs bits least-significant-bit first inside each byte (bit_order: lsb_first), concatenates fixed-width rows in shot order, and requires unused padding bits to be zero.
Row size
bytes_per_shot = ceil(bits_per_shot / 8)
The manifest records both values; use them instead of guessing from the file size.
Verified example
The example circuit has 50 record bits: 7 bytes per shot. Its 100,000-shot export writes 700,000 bytes to shots.b8.
# Decode bit k of shot s
row_bytes = (bits_per_shot + 7) // 8
bit = (raw[s * row_bytes + k // 8] >> (k % 8)) & 1
Keep loss separate from the stored value
Loss-visible measurement (ML) and measurement-plus-reset (MRL) instructions store loss_flag,value_bit. Preserve the complete packed row, then deinterleave each pair into value and mask channels. A lost value may be stored as 1, but that is only a placeholder—not a physical eigenvalue and not valid syndrome evidence.
| Tensor | Shape | Source and role |
|---|---|---|
record_bits | [shots, row.bits] | Directly unpacked public shots.b8 |
measurement_bits | [shots, values] | The value_bit from each loss-visible pair |
measurement_loss_mask | [shots, values] | The paired loss_flag; a separate model input channel |
raw_detector_bits | [shots, detectors] | Optional derived feature, never sufficient by itself under loss |
detector_valid | [shots, detectors] | Derived loss-aware mask for detectors untouched by lost records |
answer_targets | [shots, 1] | Private answers.b8: supervised physical logical-error label |
logical_masks | [shots, 1] | Private masks.b8: training metadata, never decoder input in blind evaluation |
Lost measurements cannot become syndrome evidence by substituting 0 or 1 for their values. Invalidate affected checks or combine them into loss-aware superchecks that cancel the missing values. Changing a stored placeholder must not change the decoder answer.
Put the hidden logical flip before the first noise
R 1 3 5 8 10 12 15 17 19
TICK[rstim:logical_flip_point] # rstim:logical_input=X:1,8,15
X_ERROR(0.002) 1 3 5 8 10 12 15 17 19
The generator places this marker after ideal state preparation and before noise, with the logical-X support for its layout. For a custom circuit, use exactly one top-level marker before every positive-probability noise instruction, including noise inside completed REPEAT blocks. See the export reference for metadata syntax and explicit overrides.
3. Load the training data
The checked standard-library loader validates manifests, dataset identity, row sizes, trace indices, logical-input metadata, masks, and the private answer relation before returning fixed-width Python lists ready for NumPy, PyTorch, or JAX.
Download load_blinded_training_data.py into your working directory, then run it with Python 3. This optional loader prepares training inputs; sampling and decoding use rstim.
python3 load_blinded_training_data.py \
--public-dir data/public \
--private-dir data/private \
--observable-rec -17 \
--observable-rec -15 \
--observable-rec -13
Those three record offsets belong to the generated d=3 Mid-SWAP example. Read OBSERVABLE_INCLUDE(0) from your own circuit when using a different layout.
trace.jsonl is ragged by design. Keep it as an auditable sidecar, or encode events by operation path into fixed-width features only when auxiliary physical-error supervision is useful.