Sampling and training data

Choose the output

Three paths, depending on what comes next

Inspect

Raw measurement shots

Use circuit sample. Start with readable 01; switch to compact b8 for volume.

Decode

Detector rows

Use dataset export --mode detectors. Publish detector inputs and keep observable answers private.

Train

Fully labeled simulator data

Use blinded measurements; add --error-trace only when training or debugging also needs realized physical error events.

Quick sampling

Generate a loss-visible circuit and inspect four shots

Run from the repository root. The first command creates a d=3 Mid-SWAP Z-memory circuit whose loss-visible measurements are stored as interleaved flag,value records.

Run from a configured repository checkout. These commands are not included in the two-binary native archive. This advanced guide uses the development CLI and a repository Python loader; start with the installed CLI example if you are new to RustQEC.

cargo run --release --locked -p rustqec-cli --bin rustqec -- \
  circuit gen \
  --code surface_code \
  --task rotated_memory_z_midswap \
  --distance 3 --rounds 2 \
  --noise 0.002 \
  --operation-loss-probability 0.002 \
  --measurement-loss-probability 0.003 \
  --out circuit.stim

cargo run --release --locked -p rustqec-cli --bin rustqec -- \
  circuit sample \
  --in circuit.stim --shots 4 --seed 7 \
  --out raw-shots.01 --out-format 01

What gets saved?

raw-shots.01 contains one ASCII bit row per shot. It is easy to inspect, but it has no manifest, answer labels, or separate sidecar. Use it for debugging—not as the long-term training contract.

Recommended training layout

Export aligned public inputs and private truth

Use detectors for ordinary detector-input benchmarks without erased records. For atom loss, prefer measurements_blinded: it retains loss flags and avoids declaring an arbitrary lost-value parity to be valid syndrome.

Mode	Public row	Private truth	Use when
`detectors`	Detector bits	Observable flips	No referenced measurement is lost, or the result is explicitly a legacy baseline
`measurements_blinded`	Measurement records	Unmasked logical-error target + source mask	Training, blind evaluation, and loss-aware decoding

For the generated d=3 Mid-SWAP example, 1,8,15 is a verified logical-X representative. Do not copy that support to a different circuit: the exporter checks that the requested Pauli preserves detector references and flips observable 0.

mkdir -p data

cargo run --release --locked -p rustqec-cli --bin rustqec -- \
  dataset export \
  --circuit circuit.stim --shots 100000 --seed 7 \
  --mode measurements_blinded \
  --logical-x-qubits 1,8,15 \
  --public-out data/public \
  --private-out data/private \
  --error-trace

Safe to publish

`data/public/`

manifest.json  # shape, hashes, dataset id
circuit.stim  # public logical-zero circuit
shots.b8      # one measurement row per shot

Keep private

`data/private/`

manifest.json  # seed and private hashes
answers.b8    # physical logical-error target
masks.b8      # injected logical input bit
trace.jsonl   # realized errors, one shot/line

The public manifest deliberately contains no seed, answers, masks, private path, or producer-circuit choice. Pair the two bundles by their shared dataset_id, and never publish the private directory for a blind evaluation.

Trace is optional

Omit --error-trace for faster, smaller exports when answers.b8, masks.b8, and public loss flags are enough. The checked loader below intentionally verifies trace alignment and therefore requires the flag. Traced sampling is shot-by-shot; enabling it changes RNG consumption, so the same seed produces a different batch than an untraced export.

Binary layout

How `b8` stores each row

b8 packs bits least-significant-bit first inside each byte (bit_order: lsb_first), concatenates fixed-width rows in shot order, and requires unused padding bits to be zero.

Row size

bytes_per_shot = ceil(bits_per_shot / 8)

The manifest records both values; use them instead of guessing from the file size.

Verified example

50 measurement bits become 7 bytes per shot. Eight shots therefore produce a 56-byte shots.b8.

# Decode bit k of shot s
row_bytes = (bits_per_shot + 7) // 8
bit = (raw[s * row_bytes + k // 8] >> (k % 8)) & 1

Neural-network input

Keep loss separate from the stored value

For ML and MRL, records appear as loss_flag,value_bit. Preserve the complete packed row, then deinterleave each pair into value and mask channels. A lost value may be stored as 1, but that is only a placeholder—not a physical eigenvalue and not valid syndrome evidence.

Tensor	Shape	Source and role
`record_bits`	`[shots, row.bits]`	Directly unpacked public `shots.b8`
`measurement_bits`	`[shots, values]`	The `value_bit` from each loss-visible pair
`measurement_loss_mask`	`[shots, values]`	The paired `loss_flag`; a separate model input channel
`raw_detector_bits`	`[shots, detectors]`	Optional derived feature, never sufficient by itself under loss
`detector_valid`	`[shots, detectors]`	Derived loss-aware mask for detectors untouched by lost records
`answer_targets`	`[shots, 1]`	Private `answers.b8`: supervised physical logical-error label
`logical_masks`	`[shots, 1]`	Private `masks.b8`: training metadata, never decoder input in blind evaluation

Decoder rule

Do not feed matching a detector parity computed by replacing every lost measurement with 0 or 1. Invalidate affected checks or combine them into loss-aware superchecks; changing the placeholder under a set loss mask must not change the decoder answer.

Blinding contract

Put the hidden logical flip before the first noise

R 1 3 5 8 10 12 15 17 19
TICK[rstim:logical_flip_point]
X_ERROR(0.002) 1 3 5 8 10 12 15 17 19

The circuit must contain exactly one top-level TICK[rstim:logical_flip_point], after ideal state preparation and before every positive-probability noise instruction, including noise inside completed REPEAT blocks. The old comment marker is intentionally unsupported.

Load and verify

Turn rows into tensors without losing alignment

The checked standard-library loader validates manifests, dataset identity, row sizes, trace indices, logical masks, and the private answer relation before returning fixed-width Python lists ready for NumPy, PyTorch, or JAX. Because it verifies logical_input metadata, this command requires a dataset exported with --error-trace.

python3 rstim/doc/examples/load_blinded_training_data.py \
  --public-dir data/public \
  --private-dir data/private \
  --observable-rec -17 \
  --observable-rec -15 \
  --observable-rec -13

Those three record offsets belong to the generated d=3 Mid-SWAP example. Read OBSERVABLE_INCLUDE(0) from your own circuit when using a different layout.

trace.jsonl is ragged by design. Keep it as an auditable sidecar, or encode events by operation path into fixed-width features only when auxiliary physical-error supervision is useful.

Training-data guide Error-trace v1 Loss-aware detectors v1 Conditioned decoding v1