web3.py Patterns: WebsocketProviderV2

[Last update: Nov. 1, 2023]

We're happy to announce the release of a new WebSocket provider, WebsocketProviderV2, within web3.py v6.7.0! This post will give you a quick introduction to why that was a priority, how to use it, and what comes next. [Update: based on user feedback, important breaking changes have been made as late as v6.11.2.]

What are WebSockets?

Most of the web operates via HTTP(S) – a stateless request from a client produces a response from a server. Stateless here means that each request is self-contained and has no context of requests that have come before it or will come next. Each request includes some overhead to establish a new connection.

WebSockets were standardized as an alternative messaging protocol intended to serve long-lived connections, where clients and servers may need to exchange data bi-directionally on an ongoing basis. Chat/messaging apps are an obvious use case example here: after establishing a connection once, messages can flow continuously between server and client until that channel is closed.

What was wrong with the old WebsocketProvider?

In short, technical debt. The first WebsocketProvider was implemented in 2018 and hasn't changed substantially since. A primary issue is the lack of support for eth_subscribe, which is among the best reasons to reach for such a provider. In the end, a rewrite was easier than a refactor.

Of particular note, the WebsocketProviderV2 lives under the Web3Async namespace and includes the asyncio dependency. WebSockets are an inherently asynchronous messaging pattern and the websockets library is typically run on top of asyncio, so we've chosen to bake that relationship into the provider.

What's eth_subscribe?

It's an efficient means to monitor events on-chain, like new block headers, contract event logs, and pending transactions. Instead of polling for updates, the server will push an update to you when it occurs.

Note that not everything can be subscribed to via eth_subscribe. The list of examples above is nearly exhaustive. If you're using a remote node provider, you may also find slight differences between what is supported by those providers.

How do I use the new provider?

⚠️ Note: WebsocketProviderV2 is considered beta software and its API may change in smol or large ways before the dust settles.

There are multiple patterns for using the provider and you'll need to choose one depending on your use case. You may need slight variations, for example, if you're creating a Discord bot vs. a simple script. To give you the gist, one code sample using an async iterator pattern can be found below.

In this example, you'll find:

• Imports of AsyncWeb3, asyncio, and websockets in addition to the provider.
• A new method required to initialize the provider: persistent_websocket.
• The creation of a subscription to monitor new block headers: await w3.eth.subscribe("newHeads").
• The async for iterator pattern, used in combination with w3.ws.listen_to_websocket() . Code within this block will get executed every time a new message is received from the WebSocket. If subscribed to a subscription, these messages may be identified and parsed here.
• You can use arbitrary logic to conclude (i.e.,unsubscribe from) any subscription.

import asyncio
from web3 import AsyncWeb3, WebsocketProviderV2
import websockets


async def ws_v2_subscription_iterator_example():
    async with AsyncWeb3.persistent_websocket(
        WebsocketProviderV2("wss://...")
    ) as w3:
        # subscribe to new block headers
        subscription_id = await w3.eth.subscribe("newHeads")
    
        async for message in w3.ws.listen_to_websocket():
            header = message["result"]
            print(
                    f"[Block: {header['number']}]"
                    f" Gas limit %: "
                    f"{header['gasUsed'] / header['gasLimit'] * 100:.2f}%"
                )

            # if some cancel condition:
            #     await w3.eth.unsubscribe(subscription_id)
            #     break


asyncio.run(ws_v2_subscription_iterator_example())

Sample output:

[Block: 17792825] Gas limit %: 60.31%
[Block: 17792826] Gas limit %: 41.11%
[Block: 17792827] Gas limit %: 49.00%
[Block: 17792828] Gas limit %: 61.86%
[Block: 17792829] Gas limit %: 60.97%
[Block: 17792830] Gas limit %: 69.38%

This simplified example works as described, but when you're ready to dig deeper, you'll find additional context and pattern options in the documentation. We've also created a sample Discord bot repo that leverages another set of patterns.

What comes next?

WebsocketProviderV2 is beta software. User feedback will help us determine which features to prioritize and what the stable API ends up looking like – and already has to a great degree! If V2 serves users' collective needs well enough, we'll evaluate phasing out the old WebsocketProvider in the next major release of web3.py (v7).

In the meantime, please take the provider out for a spin and let us know if your pain points are addressed or report any issues you run up against! We want to hear from you in the GitHub issues and in the Discord server. Happy building! 🚀