Streaming API

aiohttp uses streams for retrieving BODIES: aiohttp.web.BaseRequest.content and aiohttp.ClientResponse.content are properties with stream API.

class aiohttp.StreamReader

The reader from incoming stream.

User should never instantiate streams manually but use existing aiohttp.web.BaseRequest.content and aiohttp.ClientResponse.content properties for accessing raw BODY data.

Reading Methods

StreamReader.read(n=-1)

:async:

Read up to a maximum of n bytes. If n is not provided, or set to -1, read until EOF and return all read bytes.

When n is provided, data will be returned as soon as it is available. Therefore it will return less than n bytes if there are less than n bytes in the buffer.

If the EOF was received and the internal buffer is empty, return an empty bytes object.

Parameters:

n (int) – maximum number of bytes to read, -1 for the whole stream.

Return bytes:

the given data

StreamReader.readany()

:async:

Read next data portion for the stream.

Returns immediately if internal buffer has a data.

Return bytes:

the given data

StreamReader.readexactly(n)

:async:

Read exactly n bytes.

Raise an asyncio.IncompleteReadError if the end of the stream is reached before n can be read, the asyncio.IncompleteReadError.partial attribute of the exception contains the partial read bytes.

Parameters:

n (int) – how many bytes to read.

Return bytes:

the given data

StreamReader.readline()

:async:

Read one line, where “line” is a sequence of bytes ending with \n.

If EOF is received, and \n was not found, the method will return the partial read bytes.

If the EOF was received and the internal buffer is empty, return an empty bytes object.

Return bytes:

the given line

StreamReader.readuntil(separator='\n')

:async:

Read until separator, where separator is a sequence of bytes.

If EOF is received, and separator was not found, the method will return the partial read bytes.

If the EOF was received and the internal buffer is empty, return an empty bytes object.

Added in version 3.8.

Return bytes:

the given data

StreamReader.readchunk()

:async:

Read a chunk of data as it was received by the server.

Returns a tuple of (data, end_of_HTTP_chunk).

When chunked transfer encoding is used, end_of_HTTP_chunk is a bool indicating if the end of the data corresponds to the end of a HTTP chunk, otherwise it is always False.

Return tuple[bytes, bool]:

a chunk of data and a bool that is True when the end of the returned chunk corresponds to the end of a HTTP chunk.

Asynchronous Iteration Support

Stream reader supports asynchronous iteration over BODY.

By default it iterates over lines:

async for line in response.content:
    print(line)

Also there are methods for iterating over data chunks with maximum size limit and over any available data.

async StreamReader.iter_chunked(n)

Iterates over data chunks with maximum size limit:

async for data in response.content.iter_chunked(1024):
    print(data)

To get chunks that are exactly n bytes, you could use the asyncstdlib.itertools module:

chunks = batched(chain.from_iterable(response.content.iter_chunked(n)), n)
async for data in chunks:
    print(data)

async StreamReader.iter_any()

Iterates over data chunks in order of intaking them into the stream:

async for data in response.content.iter_any():
    print(data)

async StreamReader.iter_chunks()

Iterates over data chunks as received from the server:

async for data, _ in response.content.iter_chunks():
    print(data)

If chunked transfer encoding is used, the original http chunks formatting can be retrieved by reading the second element of returned tuples:

buffer = b""

async for data, end_of_http_chunk in response.content.iter_chunks():
    buffer += data
    if end_of_http_chunk:
        print(buffer)
        buffer = b""

Helpers

StreamReader.exception()

Get the exception occurred on data reading.

aiohttp.is_eof()

Return True if EOF was reached.

Internal buffer may be not empty at the moment.

See also

StreamReader.at_eof()

StreamReader.at_eof()

Return True if the buffer is empty and EOF was reached.

StreamReader.read_nowait(n=None)

Returns data from internal buffer if any, empty bytes object otherwise.

Raises RuntimeError if other coroutine is waiting for stream.

Parameters:

n (int) – how many bytes to read, -1 for the whole internal buffer.

Return bytes:

the given data

StreamReader.unread_data(data)

Rollback reading some data from stream, inserting it to buffer head.

Parameters:

data (bytes) – data to push back into the stream.

Warning

The method does not wake up waiters.

E.g. read() will not be resumed.

aiohttp.wait_eof()

:async:

Wait for EOF. The given data may be accessible by upcoming read calls.