web3.py Patterns: WebSocketProviderV2

We're happy to announce the release of a new WebsocketProvider within web3.py v6.7.0! This post will give you a quick introduction to why that was a priority, how to use it, and what comes next.

What are WebSockets?

Most of the web operates via HTTP(S) – a stateless request from a client produces a response from a server. Stateless here means that each request is self-contained and has no context of requests that have come before it or will come next. Each request includes some overhead to establish a new connection.

WebSockets were standardized as an alternative messaging protocol intended to serve long-lived connections, where clients and servers may need to exchange data bi-directionally on an ongoing basis. Chat/messaging apps are an obvious use case example here: after establishing a connection once, messages can flow continuously between server and client until that channel is closed.

What was wrong with the old WebsocketProvider?

In short, technical debt. The first WebsocketProvider was implemented in 2018 and hasn't changed substantially since. A primary issue is the lack of support for eth_subscribe, which is among the best reasons to reach for such a provider. In the end, a rewrite was easier than a refactor.

What's eth_subscribe?

It's an efficient means to monitor events on-chain, like new block headers, contract event logs, and pending transactions. Instead of polling for updates, the server will push an update to you when it occurs.

Note that not everything can be subcribed to via eth_subscribe. The list of examples above is nearly exhaustive. If you're using a remote node provider, you may also find slight differences between what is supported by those providers.

How do I use the new provider?

⚠️ Note: WebsocketProviderV2 is considered beta software and its API may change in smol or large ways before the dust settles.

A couple code samples which subscribe to new block headers can be found below. There are two patterns you can leverage when establishing a subscription: an async context manager or an async iterator. We'll start with the context manager pattern (async with). Here are a few concepts you'll find in that code:

• A WebSocket is an asynchronous communication protocol, so requires that you import the AsyncWeb3 module in addition to the provider.
• To initialize the provider, utilize a new method on the AsyncWeb3 module: persistent_websocket.
• If you'd like web3.py to automatically unsubscribe after a block of code, use async with when initializing the provider.
• Code within a listen_to_websocket() block will get executed every time a new message is received from the WebSocket. If subscribed to a subscription, these messages may be identified and parsed here.
• You can use arbitrary logic to conclude (i.e.,unsubscribe from) any subscription. For the sake of this example, the subscription is cancelled at the next block ending in zero.

import asyncio
from web3 import AsyncWeb3, WebsocketProviderV2

WS_URL = "your-provider-ws-url-here"


async def ws_v2_subscription_example():
    async with AsyncWeb3.persistent_websocket(WebsocketProviderV2(WS_URL)) as w3:
        # subscribe to new block headers
        subscription_id = await w3.eth.subscribe("newHeads")

        unsubscribed = False
        while not unsubscribed:
            async for header in w3.listen_to_websocket():
                print(
                    f"[Block: {header['number']}]"
                    f" Gas limit %: "
                    f"{header['gasUsed'] / header['gasLimit'] * 100:.2f}%"
                )

                # unsubscribe from new block headers
                if await w3.eth.block_number % 10 == 0:
                    unsubscribed = await w3.eth.unsubscribe(subscription_id)
                    break

        # still an open connection:
        latest_block = await w3.eth.get_block("latest")
        print(f"Latest block: {latest_block}")

        # the connection closes automatically when exiting the context
        # manager / `async with` block


asyncio.run(ws_v2_subscription_example())

Sample output:

[Block: 17792825] Gas limit %: 60.31%
[Block: 17792826] Gas limit %: 41.11%
[Block: 17792827] Gas limit %: 49.00%
[Block: 17792828] Gas limit %: 61.86%
[Block: 17792829] Gas limit %: 60.97%
[Block: 17792830] Gas limit %: 69.38%

The async iterator version of this code looks similar, but leverages the async for pattern. The iterator may be preferable if you'd like to handle errors, but reconnect, re-subscribe, and carry on with the subscription. The abbreviated version:

import asyncio
from web3 import AsyncWeb3, WebsocketProviderV2
import websockets


async def ws_v2_subscription_iterator_example():
    async for w3 in AsyncWeb3.persistent_websocket(
        WebsocketProviderV2(WS_URL)
    ):
        try:
            ...
        except websockets.ConnectionClosed:
            continue


asyncio.run(ws_v2_subscription_iterator_example())

What comes next?

WebsocketProviderV2 is beta software. User feedback will help us determine which features to prioritize and what the stable API ends up looking like. If the new version serves everyone's needs well enough, we'll evaluate phasing out the old WebsocketProvider in the next major release of web3.py (v7).

In the meantime, please take the provider out for a spin and let us know if your pain points are addressed or report any issues you run up against! We want to hear from you in the GitHub issues and in the Discord server. Happy building! 🚀