Reader#

class Reader(bits: Bits | BitArray, pos: int = 0)#

Wraps a Bits or BitArray object with a bit position for sequential reading. The bitstring must already be a Bits or BitArray, and pos must be within it.

A Reader is a cursor over bit data. The data itself belongs to the wrapped Bits or BitArray; the reader adds a current bit position and a set of methods that read from it and move it along.

>>> r = Reader(Bits('0x160120f'))
>>> r.read_value('u12')
352
>>> r.pos
12

Every method is anchored at pos. None of them take an absolute range to work on, so a reader works forwards through the data unless you move it back yourself, either by setting the position or with seek_back_to. Whole-bitstring operations remain available on the wrapped object, which is exposed as Reader.bits and is the original object rather than a copy. If it is a BitArray then it can be mutated through the reader, for example r.bits.append('0xff'). Mutating the wrapped object never updates pos automatically.

Reading#

There is no plain read method. Each reading method instead says what it returns:

>>> r = Reader(Bits('0x160120f'))
>>> r.read_bits(12).hex
'160'
>>> r.pos = 0
>>> r.read_value('u12')
352
>>> r.read_list('u12, bin3')
[288, '111']
>>> r.remaining
1

read_value takes one dtype, either as a string such as 'u12' or as a Dtype object, and returns one value. A format with more than one token in it is an error - use read_list for those, which also takes the keyword arguments that a format may need for its lengths.

Reading raw bits can be done either way. If the length is known when the code is written then r.read_value('bits12') works, as bits is an ordinary dtype; read_bits additionally allows a length that is only known at run time:

n = r.read_value('u8') * 8      # a length read from the data itself
payload = r.read_bits(n)

If there are not enough bits left for a read then a ReadError is raised and pos is left where it was. That is true of every method on this class: the position moves only when the operation succeeds.

Looking ahead#

Looking ahead is usually a matter of a single value - a tag, a length or a marker that decides how the bits after it should be read. Reader.peek_value and Reader.peek_bits do that, returning what the matching read would have returned but leaving the position alone:

>>> r = Reader(Bits('0x01ff'))
>>> r.peek_value('u8')
1
>>> r.pos
0

There are deliberately no peeking versions of read_list and read_array. To look ahead over more than a single value, use Reader.bookmark. It is a context manager that restores the position when the block ends, whether or not an exception was raised, and any combination of reads and seeks can be used inside it:

>>> r = Reader(Bits('0x01ff02'))
>>> with r.bookmark():
...     kind, size = r.read_list('u8, u16')
>>> r.pos      # unchanged by the block above
0

This is also the way to undo a read that turned out to be the wrong one, by reading inside a bookmark and only repeating it outside once the data has been identified.

Moving the position#

Reader.pos is an ordinary read/write property, so it can be set directly or adjusted with +=. Setting it outside the data raises a ValueError. Reader.byte_pos is the same position measured in bytes, and Reader.remaining and Reader.at_end describe how much is left:

>>> r = Reader(Bits('0x160120f'), pos=8)
>>> r.byte_pos
1
>>> r.remaining
20
>>> r.pos += 20
>>> r.at_end
True

Reader.align moves forwards to the next multiple of a given boundary and returns how many bits it skipped. It defaults to byte alignment, and raises a ValueError if there are not enough bits left to reach the boundary:

>>> r = Reader(Bits.from_zeros(64), pos=3)
>>> r.align()
5
>>> r.align(32)
24
>>> r.pos
32

Searching#

Searching moves the reader to a marker in the data. Searches run forwards from the current position, and return True or False rather than a position, so that a match at bit zero isn’t mistaken for a failure. On a failure the position is not moved.

Reader.seek_to leaves the position at the start of the match, and Reader.seek_past leaves it just after the match. The search starts at the current position, so a match already under the cursor is found where it is and nothing moves. That makes while r.seek_to(...) an infinite loop, and seek_past the one to loop on:

>>> r = Reader(Bits('0x0000010c0000011f'))
>>> while r.seek_past('0x000001', byte_aligned=True):
...     print(r.read_value('u8'))
12
31

Reader.read_to and Reader.read_past do the same two moves but also return the bits passed over, the difference between them being whether the match itself is included:

>>> r = Reader(Bits('0xaabbcc00dd'))
>>> r.read_to('0x00', byte_aligned=True).hex
'aabbcc'
>>> r.read_past('0x00', byte_aligned=True).hex
'00'

The two families differ in how they report a missing match. The seeks return False, as not finding something is a normal outcome of looking for it. The reads raise a ReadError, as with any other read that cannot be satisfied.

Reader.seek_back_to is the only method that searches backwards. Only matches that end at or before the current position are considered, so the position always ends up further back than it started and while r.seek_back_to(...) does make progress. Like seek_to it leaves the position at the start of the match.

All six searching methods take the same two optional arguments: byte_aligned to match only on byte boundaries, and mask, where only the bits set in the mask need to match:

>>> r = Reader(Bits('0x00120034'))
>>> r.seek_to('0x0000', mask='0xff00')
True
>>> r.byte_pos
0

To search without moving the position, use the wrapped object directly, which gives the full Bits.find interface including an explicit range:

>>> pos = r.bits.find('0x00', start=r.pos)

Errors#

Exception

Raised when

ReadError

A read needs more bits than are left, or a searching read does not find its match.

ValueError

A position is set outside the data, Reader.byte_pos is read while the position is not byte aligned, align cannot reach its boundary before the end, or an argument does not make sense, such as a negative length, an empty bitstring to search for, or a multi-token format given to read_value.

TypeError

An argument is of the wrong type for the method, such as an integer given to read_value or a bitstring to read_bits.

In every case the position is left unchanged, so a failed read can be caught and retried differently.

Methods#

Reader.align(boundary: int = 8, /) → int#

Moves Reader.pos forwards to the next multiple of boundary bits and returns the number of bits skipped. If the position is already on a boundary then nothing is moved and 0 is returned. This covers byte alignment as align() and generalises to 16-bit or 32-bit boundaries for free.

Raises a ValueError if boundary is not positive, or if aligning would move past the end of the data, in which case the position does not move.

>>> r = Reader(Bits.from_zeros(64), pos=3)
>>> r.align()
5
Reader.bookmark()#

Returns a context manager that restores Reader.pos to its current value when the block ends. The position is restored whether the block completes normally or raises.

Any mixture of reads and seeks can be used inside the block, so this is the general way to look ahead. Reader.peek_value and Reader.peek_bits are shorthand for the single-value case.

>>> r = Reader(Bits('0x160120f'))
>>> with r.bookmark():
...     header = r.read_list('u12, u12')
>>> r.pos
0
classmethod Reader.from_file(source: str | Path | BinaryIO, /, *, length: int | None = None, offset: int = 0) → Reader#

Creates a reader over the contents of a file, positioned at the start. The arguments are the same as for Bits.from_file, and the file is read as an immutable Bits.

>>> r = Reader.from_file('data.bin')
>>> magic = r.read_bits(32)
Reader.peek_bits(n: int, /) → Bits#

As Reader.read_bits, but leaves Reader.pos unchanged.

Reader.peek_value(dtype: str | Dtype, /) → int | float | str | Bits | bool | bytes | None | tuple#

As Reader.read_value, but leaves Reader.pos unchanged.

>>> r = Reader(Bits('0x01ff'))
>>> r.peek_value('u8')
1
>>> r.pos
0
Reader.read_array(dtype: str | Dtype, /, count: int | None = None) → Array#

Reads count items of type dtype and returns them as an Array, advancing Reader.pos past them.

If count is not given then as many whole items as will fit in the remaining bits are read. Any bits left over at the end are not read, and can be checked with Reader.remaining.

Raises a ReadError if count is given and there are not enough bits for that many items, in which case the position does not move, and a ValueError if count is negative.

>>> r = Reader(Bits('0x0102030405'))
>>> r.read_array('u8', 3)
Array('u8', [1, 2, 3])
Reader.read_bits(n: int, /) → Bits#

Reads n bits and returns them as a Bits, advancing Reader.pos by n.

Raises a ReadError if fewer than n bits remain, and a ValueError if n is negative. For a length that is fixed when the code is written, read_value('bits8') is equivalent to read_bits(8).

>>> r = Reader(Bits('0x160120f'))
>>> r.read_bits(12).hex
'160'
Reader.read_list(fmt: str | list[str | int | Dtype], **kwargs) → list[int | float | str | Bits | bool | bytes | None]#

Reads one or more format tokens and returns a list of values, advancing Reader.pos past all of them.

>>> r = Reader(Bits('0x160120f'))
>>> r.read_list('u12, u12, bin3')
[352, 288, '111']
Reader.read_past(bs: BitsType, /, byte_aligned: bool = False, mask: BitsType | None = None) → Bits#

Searches forwards for bs and reads up to and including it, leaving Reader.pos just after the match. A loop of read_past calls therefore always makes progress.

Raises a ReadError if bs is not found, in which case the position does not move, and a ValueError if it is empty. Use Reader.seek_past instead if a missing match is expected.

>>> r = Reader(Bits('0xaabbcc00dd'))
>>> r.read_past('0x00', byte_aligned=True).hex
'aabbcc00'
Reader.read_to(bs: BitsType, /, byte_aligned: bool = False, mask: BitsType | None = None) → Bits#

Searches forwards for bs and reads up to but not including it, leaving Reader.pos at the start of the match. The match itself is left to be read next.

Raises a ReadError if bs is not found, in which case the position does not move, and a ValueError if it is empty.

>>> r = Reader(Bits('0xaabbcc00dd'))
>>> r.read_to('0x00', byte_aligned=True).hex
'aabbcc'
Reader.read_value(dtype: str | Dtype, /) → int | float | str | Bits | bool | bytes | None#

Reads one dtype from Reader.pos and returns its interpreted value, advancing the position past the bits used.

dtype is given either as a string such as 'u12' or as a Dtype. A format containing more than one token raises a ValueError; use Reader.read_list for those. A dtype with no length, such as 'u', uses all of the remaining bits, which must be a whole number of items.

Raises a ReadError if fewer bits remain than the dtype needs, in which case the position does not move.

Variable-length dtypes such as 'ue' read as many bits as the value needs:

>>> r = Reader(Bits('ue=12, ue=3'))
>>> r.read_value('ue')
12
>>> r.read_value('ue')
3
Reader.seek_back_to(bs: BitsType, /, byte_aligned: bool = False, mask: BitsType | None = None) → bool#

Searches backwards for the previous occurrence of bs. Only matches that end at or before Reader.pos are considered, so the position always ends up further back than it started and while r.seek_back_to(bs) makes progress. If it is found then the position is moved to the start of the match and True is returned, otherwise the position is left alone and False is returned.

Raises a ValueError if bs is empty.

>>> r = Reader(Bits('0x00ff00ff'), pos=32)
>>> r.seek_back_to('0xff')
True
>>> r.pos
24
Reader.seek_past(bs: BitsType, /, byte_aligned: bool = False, mask: BitsType | None = None) → bool#

Searches forwards from Reader.pos for bs. If it is found then the position is moved to just after the match and True is returned, otherwise the position is left alone and False is returned.

Raises a ValueError if bs is empty. This is the method to use for loops, as it always makes progress:

>>> r = Reader(Bits('0x0000010c0000011f'))
>>> while r.seek_past('0x000001', byte_aligned=True):
...     print(r.read_value('u8'))
12
31
Reader.seek_to(bs: BitsType, /, byte_aligned: bool = False, mask: BitsType | None = None) → bool#

Searches forwards from Reader.pos for bs. If it is found then the position is moved to the start of the match and True is returned, otherwise the position is left alone and False is returned.

Raises a ValueError if bs is empty. A match already under the cursor is found where it is and nothing moves, so repeating this call without reading anything finds the same match each time - see Reader.seek_past.

>>> r = Reader(Bits('0xaabbcc00dd'))
>>> r.seek_to('0x00', byte_aligned=True)
True
>>> r.byte_pos
3
Reader.__len__() → int#

len(r) returns the length in bits of the wrapped bitstring. Use Reader.remaining for the number of bits still to be read.

Properties#

Reader.at_end: bool#

Read-only. True if the position is at the end of the data, so that no further bits can be read.

Reader.bits: Bits | BitArray#

Read-only. The wrapped bitstring object. This is the original object, not a copy, so a BitArray can still be modified through it. To read different data, create a new Reader.

Reader.byte_pos: int#

The current position in bytes. Reading this property requires Reader.pos to be byte aligned and raises a ValueError otherwise. Setting it sets Reader.pos to eight times the value, and raises a ValueError if that is outside the data.

Reader.pos: int#

The current bit position. It must be between zero and the length of the wrapped bitstring, and setting it outside that range raises a ValueError.

If you are reading from a BitArray while also growing it, append the new data first and then set the position.

Reader.remaining: int#

Read-only. The number of bits between the current position and the end of the data, that is len(r) - r.pos.