Reading the AVC sequence_parameter_set: a field-by-field guide to the H.264 SPS
You have two H.264 files on disk. Before a single frame is decoded, before a single pixel reaches the screen, one small structure already tells you the resolution, the profile, the chroma format, the frame rate, and the display aspect ratio. That structure is the sequence_parameter_set - the SPS. Learn to read it and most of what a decoder does stops being a mystery.
In this guide we walk through the SPS field by field using two real files, both parsed by VTCLab Media Analyzer:
- Example A - High profile, 1280x720, square pixels, 25 fps.
- Example B - Main profile, 720x480 anamorphic (SAR 32:27 -> 16:9), 29.97 fps.
The two differ in just the right ways to expose every interesting corner of the format.
What the SPS is and why a decoder cannot start without it
The SPS is a NAL unit of type 7. It carries the parameters that apply to a whole coded video sequence: profile and level, picture dimensions (in macroblocks), chroma format and bit depth, reference-frame limits, frame-numbering and picture-order-count configuration, and an optional VUI block with aspect ratio, timing, and bitstream restrictions. Without it, a decoder literally cannot size its frame buffers - it has no idea how big a picture is.
H.264 splits its parameters across two sets. The SPS (type 7) holds whole-sequence parameters. The PPS - picture parameter set, NAL type 8 - holds per-picture coding parameters such as the entropy mode, default reference-index counts, deblocking control, and the initial QP. They are activated through a chain:
slice header -> pic_parameter_set_id -> selects a PPS
PPS -> seq_parameter_set_id -> selects an SPSEvery slice names a PPS, every PPS names an SPS, and the decode parameters cascade down from the SPS. In both of our examples all the IDs are 0, so the wiring is as simple as it gets.
One last detail before the fields: the NAL header byte. An SPS NAL usually starts with 0x67. Decoded, that byte is forbidden_zero_bit = 0, nal_ref_idc = 3, nal_unit_type = 7 - the 7 in the low five bits is what marks it as an SPS.
Where the SPS hides
The SPS lives in one of two places depending on how the stream is packaged.
In an Annex-B elementary stream (.h264, .264, or inside a transport stream) it is in-band: a start code 00 00 01 followed by a NAL header byte whose low five bits equal 7 (typically 0x67). In MP4 and MOV it is out-of-band, stored in the avcC box (the AVCDecoderConfigurationRecord) inside the sample description (stsd). Both of our example files are MP4, and the analyzer found their SPS in the avcC box rather than in the picture data.
There is one gotcha that trips up every hand-written parser: emulation prevention. Inside a NAL the byte sequence 00 00 03 has the 03 inserted so the payload can never contain a false start code. Before you read a single field you must strip those 03 bytes after every 00 00, or every later field misaligns and the whole parse turns to garbage. VTCLab Media Analyzer does this automatically.
Identity, profile, and level
The first fields establish what kind of stream this is.
profile_idc selects the subset of coding tools the stream is allowed to use. The common values:
profile_idc | Profile |
|---|---|
| 66 | Baseline |
| 77 | Main |
| 88 | Extended |
| 100 | High |
| 110 | High 10 |
| 122 | High 4:2:2 |
| 244 | High 4:4:4 |
Example A is 100 (High); Example B is 77 (Main).
Next come the constraint_set0_flag through constraint_set5_flag bits, which signal extra conformance constraints. In A they are all 0. In B, constraint_set1_flag = 1. That flag advertises Baseline-compatible / Main-constrained conformance intent - it is a flag, not a profile change. The profile stays Main; the bit just promises the stream honors a tighter set of rules.
level_idc is ten times the level number and caps resolution times frame rate, bitrate, and the decoded picture buffer. A is 31 (Level 3.1); B is 30 (Level 3.0).
Finally seq_parameter_set_id is this SPS's own id, the value a PPS uses to reference it. Both are 0.
The High-profile-only block: chroma, bit depth, scaling
Here is the field that catches naive parsers. For profile_idc 100, 110, 122, or 244, the SPS includes an extra block right after seq_parameter_set_id:
chroma_format_idc-0monochrome,1is 4:2:0,2is 4:2:2,3is 4:4:4. In Example A it is1(4:2:0).bit_depth_luma_minus8andbit_depth_chroma_minus8- both0in A, meaning 8-bit luma and chroma.qpprime_y_zero_transform_bypass_flag-0.seq_scaling_matrix_present_flag-0, so the flat default quantization matrices are used.
The key contrast is Example B. Because B is Main profile, this entire block is absent. The bitstream goes straight from seq_parameter_set_id to log2_max_frame_num_minus4. A parser that always reads the chroma and bit-depth block will desync on every Main or Baseline stream it sees. Presence of this block is gated entirely on profile_idc - nothing else.
Frame numbering and picture order count
The next group configures how the decoder tracks frames.
log2_max_frame_num_minus4 sets MaxFrameNum = 2^(v + 4). The frame_num field counts reference frames in decode order and wraps at this modulus. Both examples use 0, so frame_num wraps at 16.
pic_order_cnt_type selects how the picture order count (POC) is derived. POC describes display / output order, which is distinct from decode order. Type 0 uses explicit least-significant-bit signalling; both examples use it. log2_max_pic_order_cnt_lsb_minus4 then sets MaxPicOrderCntLsb = 2^(v + 4); both use 2, so the POC LSB wraps at 64.
max_num_ref_frames is the maximum number of reference frames the decoder must keep buffered. It drives the decoded picture buffer (DPB) size and is the single biggest driver of decoder memory and latency. Both examples use 4. gaps_in_frame_num_value_allowed_flag is 0 in both.
Why two separate counters? Once B-frames enter the picture, decode order and display order diverge. frame_num keeps the decode order straight; POC puts the pictures back into the right display order.
From macroblocks to pixels
The SPS does not store width and height in pixels. It stores them in 16x16 macroblocks, and you compute the pixel size:
width = (pic_width_in_mbs_minus1 + 1) * 16
height = (pic_height_in_map_units_minus1 + 1) * 16 * (2 - frame_mbs_only_flag)frame_mbs_only_flag is the interlace switch. When it is 1, the picture is progressive, a map unit is one macroblock row, and the height factor is 1. When it is 0, field coding is allowed (interlaced), the factor becomes 2, and the height doubles.
Run the formulas on our files:
- Example A:
pic_width_in_mbs_minus1 = 79gives(79 + 1) * 16 = 1280;pic_height_in_map_units_minus1 = 44gives(44 + 1) * 16 * 1 = 720. Progressive 1280x720. - Example B:
44gives720;pic_height_in_map_units_minus1 = 29gives(29 + 1) * 16 = 480. Progressive 720x480.
Because coded dimensions are always whole macroblocks, any size that is not a multiple of 16 is coded larger and then trimmed. That is what frame_cropping_flag and its four offsets - frame_crop_left_offset, frame_crop_right_offset, frame_crop_top_offset, frame_crop_bottom_offset - do. The offsets are counted in chroma-subsampling units: for 4:2:0 progressive, CropUnitX = CropUnitY = 2 luma samples per step.
The canonical case is 1920x1080. Its height, 1080 / 16 = 67.5, is not a whole number of macroblock rows, so it rounds up to 68 rows and a coded height of 68 * 16 = 1088. That leaves 1088 - 1080 = 8 excess luma rows; with CropUnitY = 2, the encoder writes frame_crop_bottom_offset = 8 / 2 = 4 to trim them. Both of our examples have frame_cropping_flag = 0, because 1280, 720, and 480 are all exact multiples of 16 and need no trimming.
VUI in depth: aspect ratio, timing, and restrictions
Both examples set vui_parameters_present_flag = 1, so the VUI (Video Usability Information) block is present. This is where the SPS stops describing the decode process and starts describing how the picture should be shown.
Aspect ratio. With aspect_ratio_info_present_flag = 1, the field aspect_ratio_idc indexes a fixed table of Sample Aspect Ratios (the pixel shape). Index 1 is 1:1, square pixels. The special value 255 is Extended_SAR, which means explicit sar_width and sar_height fields follow.
- Example A:
aspect_ratio_idc = 1- square pixels. Display shape is just the coded shape, 1280x720 = 16:9. - Example B:
aspect_ratio_idc = 255, thensar_width = 32,sar_height = 27- a pixel aspect ratio of 32:27. The display aspect ratio is(720 / 480) x (32 / 27) = 1.5 x 1.18519 = 16:9. This is classic NTSC-DVD anamorphic: a 16:9 picture stored in a 720x480 grid using wide pixels.
Because the SPS stores the pixel ratio, not the display ratio, you always compute the DAR yourself. If that distinction is new, see our companion guide on pixel vs display aspect ratio (opens in new tab), which walks through SAR, PAR, and DAR in detail.
Timing. When timing info is present, the frame rate is:
fps = time_scale / (2 * num_units_in_tick)- Example A:
num_units_in_tick = 1,time_scale = 50->50 / 2 = 25fps. - Example B:
num_units_in_tick = 1001,time_scale = 60000->60000 / 2002 = 29.97fps.
The factor of 2 reflects that H.264 timing counts field periods, not frame periods. Both examples set fixed_frame_rate_flag = 0.
Bitstream restrictions. Both set bitstream_restriction_flag = 1 and carry two numbers a decoder uses to size its buffers before decoding anything:
max_num_reorder_frames = 2- the maximum number of frames that precede a given frame in decode order but follow it in output order. This is your reorder latency.max_dec_frame_buffering = 4- the total number of frames the DPB must hold.
Together they let a decoder bound its memory and its latency up front.
Practical: what to check when something is wrong
The SPS is the first place to look when a file misbehaves:
- File will not play on a device -
profile_idcorlevel_idcis too high for the hardware decoder. - Combing or interlace artifacts -
frame_mbs_only_flag = 0when you expected progressive. - Stretched or squished video - a wrong or missing
aspect_ratio_idc/ SAR (again, see the aspect-ratio guide (opens in new tab)). - Reported resolution disagrees with the container - compare the computed width and height against the MP4
tkhddimensions and anypaspbox. The coded whole-macroblock size and the display size can legitimately differ; a genuine mismatch points to a mux bug or a wrong crop. - High latency - large
max_num_ref_frames,max_num_reorder_frames, ormax_dec_frame_buffering.
Read it yourself in VTCLab Media Analyzer
You do not have to do any of this by hand. Open an MP4 in VTCLab Media Analyzer - drag and drop or use the file picker; everything runs in the browser. Then either expand the MP4 track down to the sample description (stsd) and its avcC box, or just find the AVC Sequence Parameter Set (SPS) entry directly in the packet list. Click it.
The detail panel on the right shows every field with its raw value and a human interpretation: profile_idc 100 (0x64) High, level_idc 31 (0x1F) 3.1, aspect_ratio_idc 1 (1:1 square), and so on.
This is where the analyzer earns its keep. It strips the emulation-prevention bytes, knows that the chroma and bit-depth block only exists for High-family profiles, and does the macroblock-to-pixel arithmetic for you. No hand-parsing, no hex math, no desync on the next Main-profile file you open.
Once you can read an SPS, you can answer the most important questions about an H.264 stream - resolution, profile, frame rate, aspect ratio - before you ever decode a frame. Open one of your own files and check the fields against the two examples here; they cover most of what you will meet in the wild.
Comments