4d — Ptychography: how the step works

Processing scripts · WASP · 4d User Manual

← Back to the Ptycho step on the WASP page

The WASP step 04_ptychography.py (“Ptycho – patches, reconstruction and merge”) turns a calibrated, drift-corrected 4D-STEM stack into the image the particle analysis reads. It works in three stages: it lays out the regions that are reconstructed (stage A), reconstructs each of them by ptychography (stage B), and merges the amplitude and phase of that reconstruction into one image (stage C). The step does no image processing itself. Each stage is a complete script of its own in scripts/image/WASP/dependents/; the step bakes, runs and chains them. This page follows the data through all three stages, with the default settings unless stated otherwise.

The three stages at a glance

GUI parameters & calibration KV · CSA · CPE · scan_hole · … baked into every stage HDF5 stack fd_stack data/dpatterns positions/shadow · angs mask/patches/ptycho mask/frame_id data/defocus/map data/descan_model data/descan_direction A · ROI layout tile the field with ROIs (3 × 3) label each position with its ROI dose = electrons ÷ scanned area reads writes ptycho_roi.png ROI overlay on the scan pty_dose B · Reconstruction py4DSTEM Torchslice GSharp each ROI: preprocess, solve, fuse on CUDA, Metal or the CPU one set of files per engine reads ptycho_<engine>_amplitude.mrc ptycho_<engine>_phase.mrc …_levels.png · …_summary.png suffix _roiN, one per ROI pty_engine_used C merges this engine read by C C · Amplitude–phase merge filter amplitude and phase z-score, flip the phase, add crop ROIs to a shared size a_p_merge.mrc · .png the merged image
Figure 1. What each stage reads and writes. Stage A labels every scan position in the stack with its ROI and measures the dose. Stage B reads both and writes one amplitude–phase pair per ROI under its engine's name. Stage C reads the pair that stage B has just written. Blue arrows are the hand-overs between stages.

Each stage has its own switch: pty_run_patches, pty_run_reconstruct and pty_run_merge. A stage that is switched off is skipped, and the stages after it use whatever its last run left behind. You can therefore lay the ROIs out and check the overlay without reconstructing, reconstruct again on the stored layout, or rerun the merge in seconds with other filter cutoffs.

How the step runs a stage

Start choose the engine from pty_engine progress bar: A 5 % · B 85 % · C 10 % take the next stage: A, B, then C switched on? pty_run_patches pty_run_reconstruct pty_run_merge no skip it: the later stages use what its last run left behind yes Bake header: baked values + forwarded values body: dependents/<stage>.py → proc/ stage C after B ran: apmerge_source := B's engine Run same Python, in the image directory its progress mapped into the stage's band exit code 0? no stop the step: the later stages would run on stale results yes Forward set-lines it wrote to the results file → baked into the later stages after B, also: pty_engine_used another stage? yes no Program finished correctly.
Figure 2. The loop that runs the stages. Each stage that is switched on is baked, run and checked. Its returned values are then forwarded to the stages after it. A failing stage stops the step; a stage that is switched off is skipped.
  1. Engine and progress bar. pty_engine picks the engine: GSharp if it names GSharp, Torchslice if it names Torchslice (the default), and otherwise py4DSTEM, with a warning for anything unrecognised. The progress bar is split into bands, roughly by run time: 5 % for the ROI layout, 85 % for the reconstruction and 10 % for the merge. When some stages are off, the bands of the stages that run are scaled to fill the bar.
  2. Bake. The stage's source is read from dependents/. Everything below its # From here on follows line is kept, and a header is written above it with one assignment for every value the step was given, exactly as the app does for a script it starts. The values returned by the earlier stages of this run are included. The result is written to proc/ in the image directory.
  3. Run. The baked copy runs with the same Python interpreter, in the image directory, writing to this step's results file. Its output streams into the step's log as it is produced, and its <<@progress>> marks are rescaled into the stage's band. A non-zero exit code stops the whole step, because the stages after it would otherwise work on stale results.
  4. Forward. Run as separate steps, the stages would pass values to each other through the project configuration, which 4d only updates after a step has ended. Inside this one step, the set lines (including set -force) that a stage appends to the results file are read back and baked into the stages after it. This is how pty_dose from stage A reaches stage B in the same run.

Two rules belong to this step alone. When stage B runs, stage C is baked with apmerge_source set to the engine B has just run, whatever the GUI says. Every engine writes files under its own name, and latest or another engine's name would otherwise merge an older result. After stage B, the step itself writes set pty_engine_used for the merge and the particle analysis.

Stage A – ROI layout

A full field of view is too large, and too uneven, to reconstruct as one object. Stage A (dependents/ptycho_patches.py) therefore cuts it into regions of interest (ROIs), each reconstructed on its own.

  1. Positions. The stage uses the most refined positions the stack holds: positions/shadow from the drift correction, else the calibrated positions/angs, else the raw positions/units. The hole found by the scan calibration is carried into that frame through the transform fitted between the two position sets.
  2. Dose. The total number of electrons (the sum of all patterns divided by the counts per electron, CPE) is divided by the area of the convex hull of the scan positions. The result is the mean applied dose in e/Ų, returned as pty_dose.
  3. Layout. An N × N block of tiles is laid over the hole (default 3 × 3 squares), or over the centre of the scanned field. Square tiles sit on a square lattice; round and hexagonal ones sit on a hexagonal lattice. An odd N puts one tile on the centre; an even N puts a corner there. In the automatic size mode the tiles cover the field of view exactly. The 2 + 3 + 2 rosette instead lays 7 tiles over a round field, with less of each tile outside the scan.
  4. Assignment. Every scan position gets the 1-based index of the tile it falls in, or 0 outside all tiles; where tiles overlap, the higher index wins. The labels go to mask/patches/ptycho in the stack, and an overlay is drawn as ptycho_roi.png. The rim transit waypoints (mask/transit, where the beam travels round the scanned area between passes, off the specimen) are left out of every selection.

Stage B – reconstruction

Stage B runs the engine's own script, dependents/ptycho_<engine>.py. Figure 3 follows one ROI through the default engine, Torchslice. The GSharp engine shares everything down to the levels; the py4DSTEM engine has the same overall shape with a different solver and preprocessing (see the three engines).

for each ROI, all of them in turn when pty_ROI_id = 0 pty_ROI_id Cut out the ROI its patterns and positions; rim transit waypoints dropped Defocus mean of the defocus map over the ROI, sign flipped local: a fitted surface gives each exposure its own pty_defocus_mode pty_defocus_local Preprocessing 1 hot pixel, bright-field disc → disc centre and pixel size 2 smooth descan Zernike fit of the disc centres 3 row-direction descan ± disc shift, sign per scan row 4 CBED alignment kept only if the disc edge sharpens 5 aperture fit → soft-edged starting probe 6 descan model chosen stack's model, else the ROI fit 7 scan-coil lag correction (optional) pty_descan_direction pty_cbed_align pty_aperture_fit pty_lag_nm pty_lag_refine Recursive levels scheme: all, two thirds, one third Level 0 passes 1–9 50.7 e/Ų Level 1 passes 1–6 33.8 e/Ų Level 2 passes 1–3 16.9 e/Ų later levels start from the damage-weighted levels before them sharp_level_scheme sharp_lvl_iter pty_torchslice_loss on CUDA, Metal or the CPU register the levels onto level 0 sharp_level_align Fuse in Fourier space Wiener (default): each level up to its own damage cutoff ring: coarse bands from level 0, the finest from level 2 sharp_dmg_mode sharp_fusion_mode write the amplitude and phase MRCs and the figures ptycho_torchslice_* next ROI a failed ROI is reported and skipped
Figure 3. One ROI through the Torchslice engine, with the GUI parameters that steer each step on the right. The level numbers are from a 9-pass stack at 50.7 e/Ų (honeycomb9). Each level reconstructs from the first passes only, so it sees less dose and less damage than the one before.

ROI selection

With pty_ROI_id = 0 every ROI of the mask is reconstructed in turn and every output gains a _roiN suffix. A ROI that fails is reported and skipped, so one bad region cannot lose the rest of the run. Each ROI's patterns are cut out of the stack, and its positions are shifted to a non-negative origin on the reconstruction canvas.

Defocus

The starting defocus is the mean of the defocus map (data/defocus/map, written by the defocus step) over the ROI, or the manual pty_dF. Torchslice and GSharp use the opposite sign convention to the one 4d stores, so these two engines negate the value on the way in and log both numbers. With pty_defocus_local on, a low-order surface is fitted to the per-position defocus, and every exposure's probe is propagated to its own defocus. This keeps a tilted specimen in focus at the edges of the ROI.

Preprocessing

  1. Hot pixel and bright-field disc. The hottest detector pixel is masked first, because one stuck pixel can outshine the whole disc. The bright-field disc is then fitted on the average pattern (PACBED). Its radius sets the mrad per detector pixel, and through it the pixel size of the reconstruction.
  2. Smooth descan. The disc's centre of mass in every exposure is fitted with a Zernike polynomial of the scan position: the slow drift of the disc across the field.
  3. Row-direction descan. On a serpentine scan the disc also sits a fixed distance off that smooth field, and the offset flips sign from one scan row to the next, with the direction the beam travels along the row. A polynomial of position cannot carry a term that changes sign from row to row. It is measured on the disc rim (or taken from the CBED calibration, data/descan_direction) and applied with each exposure's travel sign.
  4. CBED alignment. Torchslice fits the shift field that best aligns every pattern with the average. The fit is accepted only if it sharpens the disc edge (10–90 % width) on the compensated average by at least 5 %; otherwise the raw disc centre and the centre-of-mass model stay. Its memory grows with the number of patterns, so on large ROIs it runs on a fixed random subset within a 12 GB budget.
  5. Aperture fit. A generalised circle is fitted to the PACBED. The starting probe is built through that soft-edged aperture instead of a hard disc, whose sharp edge would put ringing into the probe.
  6. Descan model. The whole-field model stored by the CBED calibration (data/descan_model) is preferred. Otherwise the ROI's own fit is used, and only if it passes a correlation check against the measured shifts. The model is applied per exposure inside the probe generator.
  7. Scan-coil lag (optional, off by default). The scan coils trail the commanded position along the direction of travel, which prints stripes two rows apart. A fixed or refined lag moves every position back along its own travel direction before the solve.

Recursive levels

A multi-pass scan exposes the same area several times, so the earliest passes carry the least damaged signal. The levels spend that dose in steps. With the default scheme “all, two thirds, one third” on a 9-pass stack, level 0 uses all 9 passes, level 1 the first 6 and level 2 the first 3. Level 0 starts from an empty object. Every later level starts from the damage-weighted Fourier sum of the levels already solved. Each level runs sharp_lvl_iter iterations (default 10) of minibatch optimisation with the chosen loss (poisson_approx), refining the object, the probe and the scan positions with learning rates that fall from level to level (sharp_lvl_lrnet, sharp_lvl_lrprobe, sharp_lvl_lrgrid). The solver runs on whatever GPU the machine has: CUDA on Linux and Windows with an NVIDIA card, Metal on a Mac, otherwise the CPU.

Damage model and level fusion

Every level gets a resolution cutoff from the damage model. The default Grant & Grigorieff smooth roll-off weights each spatial frequency by exp(−0.5 · dose / Nc(q)), with Nc the critical dose at that frequency. The alternatives are a hard stop at the optimal exposure (2.51284 × Nc), a hard stop where the weight falls to 0.1, or fixed Butterworth cutoffs (sharp_lvl_dmg). In the example above the cutoffs come out at 23.8, 18.3 and 11.4 Å. The levels are registered onto level 0 first, because a Fourier merge damps whatever is misaligned between its inputs. They are then fused:

Single-pass data (one layer in mask/frame_id) has no dose to split. It is reconstructed in one level on all exposures, with no damage model and no fusion.

Outputs of the stage

ptycho_torchslice_amplitude.mrc and ptycho_torchslice_phase.mrc (the fused object, with the pixel size stamped as the voxel size), the figures ptycho_torchslice_levels.png, _summary.png and _spectrum_gold.png, and the preprocessing diagnostics ptycho_prep_cbed_align.png and ptycho_prep_aperture_fit.png. The single levels are not written. The other engines write the same set under their own names.

The three engines

EngineSolverHow the dose is usedSpecific to it
py4DSTEM py4DSTEM single-slice ptychography Nested levels, then Fourier fusion Detector-orientation search; a centre-of-mass plane is removed from the patterns themselves; the probe is re-fitted to an aberration basis after every iteration (sharp_probe_cor); complex or potential object; optional upsampling; defocus used with the stored sign.
Torchslice default Torchslice BaseReconstructor (PyTorch) Nested levels, then Fourier fusion Descan and row-direction terms applied in the probe generator; CBED alignment; aperture fit; scan-coil lag; defocus negated.
GSharp Torchslice GSharpReconstructor Disjoint dose fractions in one joint solve, no fusion Shares all inputs and preprocessing with Torchslice; the fractions are cut by pty_gsharp_fractions.
scan pass 1 2 3 4 5 6 7 8 9 dose Torchslice and py4DSTEM: nested levels, solved one after another level 0 50.7 e/Ų · solved first level 1 33.8 e/Ų · second, seeded by level 0 level 2 16.9 e/Ų · third three objects Fourier fusion GSharp: disjoint dose fractions, solved together fractions 0 – 16.9 e/Ų 16.9 – 33.8 33.8 – 50.7 gradient keeps fine + coarse medium + coarse coarse only one object, one joint solve, minibatches interleaved one object no fusion step
Figure 4. Two ways to spend the dose of a 9-pass scan. The recursive engines solve nested subsets of the passes one after another and fuse the three objects afterwards. GSharp cuts the passes into disjoint fractions (default: the first 3 passes, then triples). It solves one object from all of them at once, and filters each fraction's object gradient in Fourier space by the damage weight of the dose it was taken at. The least exposed fraction thus writes the fine detail, and the most exposed only the coarse.

Stage C – amplitude–phase merge

Amplitude and phase carry different information about the same object. The phase resolves the fine detail; the amplitude mostly reports large-scale mass-thickness and is noisy at high frequencies. The merge (dependents/ptycho_merge.py) therefore takes each channel only where it is reliable and adds the two.

phase only both channels neither phase: high-pass, periods finer than 200 Å amplitude: band-pass, 10 – 200 Å slow background, removed from both apmerge_amp_lowpass = 10 Å …_amp_highpass, …_pha_highpass = 200 Å 1 2 5 10 20 50 100 200 500 1000 period in Å (log scale): finer ← → coarser
Figure 5. Which spatial periods each channel contributes with the default cutoffs. The filters are Butterworth roll-offs, not the hard edges drawn here. Between 10 and 200 Å both channels add; finer detail comes from the phase alone. Anything coarser than 200 Å (uneven illumination, specimen tilt, the reconstruction's own low-frequency offset) is removed from both.
  1. Source. When stage B has run, the merge reads the files of that engine. Otherwise apmerge_source chooses: an engine by name, or latest for the engine the reconstruction ran last (pty_engine_used), failing that the newest files on disk.
  2. Filter. Each channel is filtered at full size, as in Figure 5, so the transforms keep the context outside the later crop.
  3. Crop. Each ROI's canvas is sized to its own scan extent, so every result is centre-cropped to the smallest canvas among them, which keeps the outputs directly comparable.
  4. Z-score. Both channels have their mean subtracted and are divided by their standard deviation, so neither dominates just by its numerical range.
  5. Flip the phase. With apmerge_invert on (the default) the phase changes sign, so features read dark in both channels and reinforce instead of cancelling.
  6. Add. The sum is written as a_p_merge.mrc (float32, unclipped, the pixel size as the voxel size) and a_p_merge.png (filtered channels and their sum on top, the unfiltered inputs below). In potential mode the py4DSTEM engine writes no amplitude, and the merge stops with a note.

Inputs and outputs

StageReadsWritesReturns to 4d
A data/dpatterns, the positions, scan_hole, CPE mask/patches/ptycho (in the stack), ptycho_roi.png pty_dose
B each ROI's patterns and positions, mask/frame_id, data/defocus/map, data/descan_model, data/descan_direction; KV, CSA, pacbed_PIX_itr, pty_dose ptycho_<engine>_amplitude.mrc, _phase.mrc, the engine's figures, ptycho_prep_*.png pty_engine_used
C ptycho_<engine>_amplitude.mrc and _phase.mrc a_p_merge.mrc, a_p_merge.png –

With more than one ROI every file name gains a _roiN suffix. Each stage's own page lists every parameter and figure in full: ROI layout, py4DSTEM, Torchslice, GSharp and amplitude–phase merge.