QCOW image format versions 2/3 analysis


#1

Background:
QCOW 3 has been released though currently MiQ only supports 1/2.
This topic aims to provide an overview of the differences between
versions 2 and 3 so as to possibly support both.

From [3]:
“The proposal for QCOW3 […] includes a version number increase in order to introduce some incompatible features, however it’s strictly an extension of QCOW2 and keeps the fundamental structure unchanged, so that a single codebase will be enough for working with both QCOW2 and QCOW3 images.”

Main Features:

  • Improved performance over qcow2
  • Fully backwards-compatible qcow2 feature set
  • Support an easy 2->3 upgrade path
  • Additions to support extensibility

Additions/Differences (from [4]):

  • Header version # bumped to 3
  • The header has been expanded with the following fields (with their offsets):
    • 72-79: incompatible_features (bitmask):
      “an implementation must fail to open image if an unknown bit is set”
    • 80-87: compatible features (bitmask):
      “an implementation can safely ignore any unknown bits”
    • 88-95: autoclear_features (bitmask):
      “An implementation may only write to an image with unknown auto-clear
      features if it clears the respective bits from this field first”
    • 96-99: refcount_order:
      width of refcount block entry (may not exceed 6; in bits: 1 << refcount_order)
    • 100-103: header_length:
      length of header structure
  • The previous fields are assumed to be 0 for version 2 with the exception
    of the following:
    • refcount_order: 4 (eg 16 refcount bits)
    • header_length: 72
  • Immediately after header, header extensions are stored, each with the
    following structure:
    • 0-3: header extension type
    • 4-7: length of header extension data
    • 8-n: header extension data
    • n-m: padding to round up header extension size to multiple of 8
  • “Unless stated otherwise, each header extension type shall appear at most once
    in the same image”
  • If image has a backing file, name will be stored in remaining space after header
    extensions and end of first cluster. No other data is permitted to be stored here.
  • If the least significant bit of a cluster descriptor in the L2 table is 1, the
    cluster reads as all 0. eg the cluster will be ‘preallocated’ but won’t actually
    be read from disk image or backing file. In version 2 this is always 0.
  • “Extra” data in snapshot header must include:
    • 40-47: size of vm state (in bytes, 0 if no vm state saved)
    • 48-55: virtual disk size of snapshot (in bytes)

Values as of QCOW3:

  • incompatible_features (bitmask bits):
    • 0 - dirty: rescan L1/L2 table to repair refcounts
    • 1 - corrupt: some data structure is corrupt, image must not be written to
  • compatible_features (bitmask bits):
    • 0 - lazy refcounts: if set image may be marked dirty and postpone refcount updates
  • autoclear_features (bitmask bits):
    • n/a
  • header extension types:
    • 0x00000000: end of header extension area
    • 0xE2792ACA: backing file format name
    • 0x6803f857: feature name table

Feature Name Table:

Optional header extension containing names of features used by image.
Can be used to display useful errors in apps that are unaware of specific features.

Total length is given by header extension data length, each entry has same format:

  • 0: type of feature
    • 0: incompatible
    • 1: compatbile
    • 2: autoclear
  • 1: bit number in corresponding feature bitmap
  • 2-47: name (padded w/ 0’s, not necessarily null terminated if full length)

External References:

[1] QCOW: https://people.gnome.org/~markmc/qcow-image-format-version-1.html
[2] QCOW2: https://people.gnome.org/~markmc/qcow-image-format.html
[3] QCOW3: http://wiki.qemu.org/Features/Qcow3
[4] 2/3 Spec: https://github.com/qemu/QEMU/blob/master/docs/specs/qcow2.txt