Group of pictures and frame architecture

The three picture types

A modern video bitstream does not describe every picture from scratch. It describes the first picture in a run fully, and then describes the following pictures mainly by reference to the ones already sent. This is the foundation of temporal compression, and it produces three kinds of picture.

An I-frame, or intra-coded picture, is compressed using only information inside itself. It is the most expensive picture to send, but it is also the only one that can be decoded without any earlier picture. A P-frame, or predicted picture, is described by reference to an earlier picture; the encoder searches for matching patches in that reference and records where each patch came from, together with a correction for the small differences that remain. A B-frame, or bi-predicted picture, may be described by reference to pictures both before and after it, which lets the encoder average two estimates and spend fewer bits on the correction.

Decode order is not display order

Because a B-frame refers to a picture that appears later in the finished sequence, the encoder must send that later picture first. The transmission order is therefore rearranged so that a reference is always available before it is needed. A small delay buffer in the decoder holds the received pictures and releases them in display order. This reordering is invisible to the viewer but it explains why an encoder reports two orders, one for coding and one for presentation, and why the difference between them sets a limit on how many B-frames may be used.

Reference chains

The pattern of references need not be a simple line. An encoder may keep several earlier pictures available as references, and a later picture may draw on any of them. This enlarges the search space and lets the encoder find a better match for a patch that was obscured and later revealed. The cost is memory in both the encoder and the decoder, since every retained reference must be stored. The choice of how many references to keep is a direct trade between compression gain and hardware cost.

Choosing a group length

The distance between two I-frames is the group-of-pictures length, often abbreviated GOP. A long group reduces cost, because only one expensive picture is sent in each run. But a long group has three practical objections. First, random access becomes slower, since a viewer joining the stream must wait for the next I-frame. Second, an error that corrupts one picture propagates through every picture that refers to it, until the next I-frame stops the damage. Third, editing becomes harder, because a cut can be placed cleanly only at an I-frame unless the material is re-encoded.

Broadcast practice tends to favour short groups, because a viewer who tunes in at any moment should see a picture within about half a second. Archival practice favours longer groups, because the material will not be joined at random and every saved bit improves the quality of the master. The correct choice is therefore not a constant but a decision that follows from how the material will be used.

A related picture type, the instantaneous decoder refresh picture, marks a point after which no earlier picture may be referenced. It gives the same clean start as an I-frame but with a firmer guarantee, and it is the natural place to set an entry point in a streaming delivery chain.