4d — Image file formats

← Back to 4D-STEM explained · 4d User Manual

Why the format matters

A file format is not just a container: it decides what survives of your measurement. Four properties matter for electron microscopy, and the rest of this page is mostly about them.

Bit depth and number type. Detector counts need 16 bits, reconstructions and phase maps need 32-bit floating point. An 8-bit format (JPEG, GIF) throws away the dynamic range that makes the data quantitative, and cannot represent a negative value at all.
Lossy or lossless. Lossless formats return exactly the numbers you wrote. JPEG does not: it discards high spatial frequencies — which, in a noisy micrograph or a diffraction pattern, is where the signal lives.
The origin. Row 0 sits at the top in every ordinary picture format and at the bottom in MRC. Getting this wrong mirrors the image, which flips the handedness of everything derived from it.
The header. Pixel size, voltage, dose and camera settings only travel with the data if the format has somewhere to put them.

Comparison at a glance

File sizes are for a single 1024 × 1024 image. "Raw" is the pixel data alone; the compressed figures assume noisy detector data, which compresses far less than an ordinary photograph — noise is incompressible, and that is the single most common surprise when people first archive micrographs.

Format	Compression	Bit depths	Origin	Metadata in the header	1024 × 1024
HDF5	Optional, lossless, per chunk: gzip, Blosc2, bitshuffle+LZ4, Zstd, LZF	anything: 8/16/32/64-bit integer or float, complex	no convention — whatever the writer used	unlimited, user-defined attributes anywhere in the tree	2.1 MB at 16 bit raw; counting data often 5–10× smaller compressed
MRC	None in the format (files are often gzipped whole)	int8, int16, uint16, float16, float32, complex	bottom-left	fixed 1024-byte header + optional extended header	2.1 MB at 16 bit, 4.2 MB at 32-bit float, + 1 KB header
TIFF	Optional, lossless: LZW, Deflate, PackBits (JPEG-in-TIFF is lossy)	1–64 bit, integer or float, any number of samples	top-left (Orientation tag 274 can say otherwise)	tags: size, bit depth, resolution, description, date, private tags	2.1 MB at 16 bit raw; ≈ 1.6–2.0 MB with LZW/Deflate on noisy data
EER	Lossless, event-based — the point of the format	event positions, not pixel values	top-left (TIFF container)	TIFF tags plus an XML metadata block	scales with electrons, not pixels — see below
PNG	Lossless, Deflate with per-row prediction filters	8 and 16 bit, grey or RGB(A); no float	top-left	size, bit depth, optional physical pixel size and text chunks	≈ 1.5–2.0 MB at 16 bit, ≈ 0.7–1.0 MB at 8 bit, on noisy data
JPEG	Lossy, DCT-based, quality-dependent	8 bit per channel	top-left (EXIF Orientation may rotate/flip)	JFIF and EXIF blocks: camera, date, orientation, thumbnail	≈ 0.2–0.7 MB at quality 90 — noise inflates it
GIF	Lossless LZW, but only within a 256-colour palette	8-bit palette (≤ 256 colours)	top-left	screen descriptor, palette, per-frame delays	≈ 0.6–1.0 MB after quantising to 256 greys

For reference, the uncompressed pixel data of a 1024 × 1024 image is 1.05 MB at 8 bit, 2.10 MB at 16 bit and 4.19 MB at 32-bit float. Any format that claims much less than that has either thrown information away or found genuine redundancy — worth knowing which.

Where the origin sits

Top-left versus bottom-left image origin, and the vertical flip between them — **The most common conversion bug.** Ordinary picture formats store the top row first; MRC data are conventionally displayed with row 0 at the bottom, the crystallographic convention its viewers (IMOD, RELION, EMAN) follow. The same bytes therefore appear mirrored top to bottom depending on which convention the reader applies. A vertical mirror is not harmless: it reverses handedness, so defocus gradients, tilt axes, drift directions and the chirality of a reconstruction all come out wrong. If an image looks right but a derived quantity has the wrong sign, suspect this first.

HDF5 (.h5, .hdf5) — the native format of 4d

HDF5 is not an image format but a hierarchical container: a file holds a tree of groups (like directories), datasets (n-dimensional arrays of any type) and attributes (metadata attached to either). That generality is why it carries a whole 4D-STEM acquisition — scan positions, diffraction patterns, masks, precomputed maps — in one file.

Compression	Per-chunk filters, all lossless: gzip/Deflate, Blosc2, bitshuffle + LZ4, Zstd, LZF, plus byte-shuffle pre-filters. Chunking also decides read speed: a chunk shaped like one diffraction pattern lets 4d load a single pattern without touching the rest.
Bit depth	Anything the machine has, including 64-bit floats and complex numbers. Datasets can be extendable and sparse.
Origin	Undefined by the format. An array is just an array; the convention comes from the layout the writer follows.
Header	A superblock plus B-trees and heaps, all handled by the library. Scientific metadata lives in user-defined attributes, so the layout must be agreed on — hence the conventions: 4d's own, EMD, NeXus.
1024 × 1024	Raw size plus a few kilobytes of structure. Compression on counting-mode data, where most pixels are zero, routinely gives 5–10×; on dense 16-bit data expect 1.1–2×.

4d stores datasets in its version 2.0 convention with Blosc2 compression; the group layout is described in §7 of the main manual.

MRC (.mrc, .mrcs, .map)

The workhorse of cryo-EM: a fixed 1024-byte header, an optional extended header whose length the fixed header states, then raw pixel data. It stores 2D images, image stacks (.mrcs) and 3D volumes (.map) in the same structure, which is why a reconstruction and a particle stack look alike to a reader.

Compression	None. The format writes raw numbers, which is exactly why it is fast to memory-map and why files are large. Whole files are often gzipped for transfer (`.mrc.gz`); variants such as MRCZ add Blosc compression but are not part of the standard.
Bit depth	Chosen by the mode field: 0 = int8, 1 = int16, 2 = float32, 3 and 4 = complex, 6 = uint16, 12 = float16. Mode 2 is the default for reconstructions, mode 6 for raw camera frames.
Origin	Bottom-left by convention — see the figure above.
Header	1024 bytes of fixed fields: image size (nx, ny, nz), mode, start indices, sampling (mx, my, mz), the unit-cell dimensions from which the pixel size follows, axis order, min/max/mean and RMS of the data, space group, size of the extended header, the origin, the `MAP` stamp, a machine stamp giving the byte order, and up to ten 80-character text labels. An optional extended header follows, which Thermo Fisher (FEI1/FEI2) and SerialEM use for per-frame metadata such as tilt angle, dose and stage position.
1024 × 1024	1.05 MB (mode 0), 2.10 MB (modes 1/6), 4.19 MB (mode 2), each plus the 1 KB header.

The header specification. The current definition is MRC2014, maintained by CCP-EM: www.ccpem.ac.uk/mrc_format/mrc2014.php gives the field-by-field table (word number, type, meaning). It is described in Cheng et al. (2015), Journal of Structural Biology, "MRC2014: Extensions to the MRC format header for electron cryo-microscopy and tomography". The closely related CCP4 map format is documented at www.ccp4.ac.uk/html/maplib.html. When a file opens with scrambled values, check the machine stamp: MRC files can be written big- or little-endian, and old files sometimes lie about which.

TIFF (.tif, .tiff)

TIFF is a tagged container rather than one format: an 8-byte header points to a chain of image file directories, each a list of tags describing one image. That flexibility makes it the usual export format of camera software — and the reason two TIFF files can need completely different code to read.

Compression	Optional and normally lossless: LZW, Deflate (ZIP), PackBits run-length; newer writers add Zstd and LERC. JPEG compression inside a TIFF is possible and lossy — avoid it for data. On noisy micrographs lossless compression typically saves only 5–20 %.
Bit depth	1 to 64 bits, unsigned, signed or floating point, one or many samples per pixel — one of the few ordinary formats that stores 32-bit float directly.
Origin	Top-left by default. Tag 274 (Orientation) can declare any of eight arrangements; many scientific readers ignore it, so a file that relies on it may appear flipped.
Header	Byte-order mark (`II` or `MM`), magic number 42 (or 43 for BigTIFF), and the offset of the first directory. Tags then give width, height, bits per sample, sample format, compression, strip or tile layout, X/Y resolution and unit. Free-text tags — ImageDescription, Software, DateTime — are where ImageJ, SerialEM and camera vendors hide pixel size, dose and acquisition settings, often as XML or a private tag block.
1024 × 1024	1.05 / 2.10 / 4.19 MB raw at 8 / 16 / 32 bit, plus a few hundred bytes of tags; ≈ 1.6–2.0 MB for a noisy 16-bit image with Deflate.

Files above 4 GB need BigTIFF, which uses 64-bit offsets — a classical TIFF simply cannot address that far, which is why long movies silently fail to save in some software.

EER (.eer) — Electron Event Representation

EER takes a different view of what an image is. Instead of integrating the electrons that arrive during a frame into pixel counts, a Falcon 4-class detector records each detected electron as an event: its position, refined to a sub-pixel grid, and the frame in which it arrived. The file is a list of those events, losslessly compressed.

Compression	Lossless and intrinsic: the gaps between successive events are run-length encoded and entropy coded. Since a low-dose frame is almost entirely empty, this is far more compact than storing mostly-zero pixel arrays — an order of magnitude or more against an equivalent raw movie.
Bit depth	Not applicable: there are no pixel values, only event positions. The image appears when you render the events onto a grid, and you choose the grid — the native sensor sampling, or 2× / 4× finer using the sub-pixel positions, and as many or as few frames per rendered image as the dose requires.
Origin	Top-left, as the TIFF container implies; the rendering library fixes the convention in practice.
Header	EER is wrapped in a TIFF structure, so ordinary TIFF tags describe the frames, with private tags carrying the compressed event bitstreams and an XML block holding the acquisition metadata: sensor size, number of frames, frame rate, dose rate.
Size	Proportional to the number of detected electrons, not to the pixel count — roughly one to two bytes per electron. A whole low-dose exposure typically lands in the low hundreds of MB where the equivalent unpacked movie would be several GB. Expressed per 1024 × 1024 area at 1 Å pixels and 40 e⁻/Å², the events of the complete exposure amount to a few tens of MB, against 2.1 MB for the single 16-bit integrated image they render into.

The practical consequence: keep the EER and decide later. Dose weighting, motion correction and the choice of fractionation can all be redone from the events, which is impossible once they have been integrated into fixed frames.

PNG (.png)

Compression	Lossless Deflate, preceded by a per-row prediction filter (each row is coded as its difference from the row above or the pixel to the left). Excellent on smooth images and line art, poor on noise.
Bit depth	8 or 16 bit, greyscale or colour, with an optional alpha channel. No floating point and no negative values, so a phase map has to be scaled into an integer range before it can be stored.
Origin	Top-left; rows are stored top to bottom (optionally interlaced).
Header	An 8-byte signature followed by chunks. `IHDR` carries width, height, bit depth and colour type; `pHYs` can record the physical pixel size; `tEXt`/`iTXt` hold free text. There is no standard place for microscope metadata, so it is usually lost.
1024 × 1024	≈ 0.7–1.0 MB for a noisy 8-bit micrograph, ≈ 1.5–2.0 MB at 16 bit; a smooth or largely empty image can drop to a tenth of that.

PNG is the right choice for figures, screenshots and anything going into a manuscript or a manual — it is lossless, universally readable, and does not invent texture the way JPEG does. It is the wrong choice for data, because the calibration does not travel with it.

JPEG (.jpg, .jpeg)

Compression	Lossy. The image is divided into 8 × 8 blocks, cosine-transformed, and the high-frequency coefficients are quantised away. At quality 90 the damage is invisible in a photograph; in a micrograph it removes precisely the fine detail you measured, and adds block edges and ringing around strong features.
Bit depth	8 bit per channel. A 16-bit detector image must be scaled down to 256 levels first, which alone rules it out for quantitative work.
Origin	Top-left. The EXIF Orientation tag may ask for a rotation or mirror, and viewers disagree about honouring it.
Header	Marker segments: JFIF (version, pixel density), optionally EXIF (camera, date, exposure, orientation, GPS, thumbnail), the quantisation and Huffman tables, and the frame header with the dimensions.
1024 × 1024	≈ 0.2–0.7 MB at quality 90 for a noisy image — noise defeats the compression, so a micrograph is several times larger than a photograph of the same size. Quality 50 halves it again and the artefacts become obvious.

Use JPEG for a quick look, a slide or an e-mail, never as the archive of a measurement. Re-saving a JPEG compounds the loss each time.

GIF (.gif)

Compression	Lossless LZW — but only after the image has been reduced to a palette of at most 256 colours, which for a micrograph means 256 grey levels. The quantisation, not the compression, is where the data is lost.
Bit depth	8-bit palette indices. One palette entry can be transparent; there is no alpha channel and no 16-bit mode.
Origin	Top-left, with each frame positioned by its own left/top offsets inside the logical screen.
Header	`GIF87a`/`GIF89a` signature, a logical screen descriptor (size, background colour, palette flags), the global colour table, and per-frame descriptors with delay times and disposal methods for animation.
1024 × 1024	≈ 0.6–1.0 MB for a noisy greyscale image — LZW can even exceed the raw size on pure noise; a few tens of kilobytes for a plot or a diagram.

The one thing GIF still does well is a short animation that plays anywhere, such as a tilt series or a through-focus series for a talk. For anything else, PNG is better in every respect.

Other formats you may meet

DM3 / DM4	Gatan DigitalMicrograph. A tagged tree carrying the image plus very complete microscope metadata. DM4 uses 64-bit offsets for large files.
SER / EMI	Thermo Fisher (formerly FEI) TIA. The `.ser` holds the data, the companion `.emi` the metadata — keep the pair together.
EMD	Electron Microscopy Datasets: a documented convention inside HDF5, readable by any HDF5 tool. Also the native format of Velox.
NeXus	Another HDF5 convention, from the neutron and synchrotron world, increasingly used for electron data.
SMV / ADSC	Simple header-plus-data format from X-ray detectors, used by some electron-diffraction pipelines.
TVIPS	Movie format of TVIPS cameras, a header followed by concatenated frames.
OME-TIFF	TIFF with a standardised XML metadata block in the ImageDescription tag — the light-microscopy answer to the metadata problem.

Rules of thumb

Keep the original. Archive what the detector wrote — EER events, the raw movie, the HDF5 — and treat everything else as derived. No conversion adds information.
Data in HDF5 or MRC, pictures in PNG. HDF5 when the metadata and the structure matter, MRC when the cryo-EM tool chain expects it, PNG for anything a human looks at.
Never put measurements through JPEG or GIF. Eight bits and lossy compression are not a storage decision, they are a measurement decision — and it is irreversible.
Check the flip once, deliberately. Convert a deliberately asymmetric image between MRC and TIFF/PNG and look at it. Do it once per pipeline and write down what you found.
Expect compression to disappoint. Detector noise is incompressible: lossless compression of a dense micrograph saves tens of percent, not factors. Counting-mode and event data are the exception — there the emptiness is real and the savings are large.
Carry the pixel size. If the format cannot hold it (PNG, JPEG, GIF), record it beside the file, or the image becomes a picture rather than a measurement.