web3.py Patterns: WebSocketProvider

Included in the (beta) release of web3.py v7 are some important and exciting updates to the asynchronous providers:

• The AsyncIPCProvider is live.
• WebsocketProviderV2 has graduated from beta and been renamed to WebSocketProvider (note the capital S).
• The original WebsocketProvider has been deprecated and renamed to LegacyWebSocketProvider.

In this post, we're diving into the new and improved WebSocketProvider. We'll introduce the WebSocket protocol, why the WebSocketProvider got a makeover, and how to use the new features.

Let's zoom out first.

What are WebSockets?

Most of the web operates via HTTP(S) – a stateless request from a client produces a response from a server. Stateless here means that each request is self-contained and has no context of requests that have come before it or will come next. Each request includes some overhead to establish a new connection.

The WebSocket protocol was standardized as an alternative messaging protocol intended to serve long-lived connections, where clients and servers may need to exchange data bi-directionally on an ongoing basis. Chat/messaging apps are an obvious use case example here: after establishing a connection once, messages can flow continuously between server and client until that channel is closed.

Why a new WebSocketProvider?

In short, the rewrite resulted in big gains in reliability, performance, and first-class support for eth_subscribe. Of particular note, the WebSocketProvider lives under the AsyncWeb3 namespace and includes asyncio as a dependency. WebSocket is an inherently asynchronous messaging pattern and the Python websockets library is typically run on top of asyncio, so we've chosen to bake that relationship into the provider.

What is eth_subscribe?

It's an efficient means to monitor events on-chain, like new block headers, contract event logs, and pending transactions. Instead of polling for updates, the server will push an update to you when it occurs.

Note that not everything can be subscribed to via eth_subscribe. The list of examples above is nearly exhaustive. If you're using a remote node provider, you may also find slight differences between what is supported by those providers.

How do I use the new provider?

Perhaps unsurprisingly, a basic familiarity of asynchronous programming in Python is valuable when navigating any of the async providers (AsyncHTTPProvider, AsyncIPCProvider, WebSocketProvider). If it's your first time wandering into these waters, you can likely still get off the ground just fine by referencing the examples in the documentation and this post.

There are multiple patterns for using the provider and you'll need to choose one depending on your use case. Some patterns are better suited for creating a Discord bot vs. a locally running script, for example. As a starting point, we'll look at a code sample that leverages an asynchronous context manager. It's a fancy name for a relatively straightforward usage pattern. Note that AsyncWeb3 is instantiated with the provider using the async with keywords, then you're free to use (await) the w3 instance within that code block:

async def example():
  async with AsyncWeb3(WebSocketProvider("wss://...")) as w3:
    ...
    block = await w3.eth.get_block("latest")
    ...

asyncio.run(example())

Let's now turn that scaffold into a small program that alerts us each time a new block is added to the blockchain and include some detail about the amount of gas used in each block. In this example, you'll find:

• Imports of AsyncWeb3, WebSocketProvider, and asyncio.
• Instantiating AsyncWeb3 via the asynchronous context manager pattern (async_with) described above.
• The creation of a subscription to monitor new block headers: w3.eth.subscribe("newHeads").
• The async for iterator pattern, used in combination with w3.socket.process_subscriptions() . Code within this block will get executed every time a new message is received by the WebSocket. If you have any active subscriptions, these messages may be identified and handled here.
• You can use arbitrary logic to conclude (i.e.,unsubscribe from) any subscription.

import asyncio
from web3 import AsyncWeb3, WebSocketProvider


async def ws_subscription_example():
    async with AsyncWeb3(WebSocketProvider("wss://...")) as w3:
        # subscribe to new block headers:
        subscription_id = await w3.eth.subscribe("newHeads")

        # listen for messages related to a subscription:
        async for message in w3.socket.process_subscriptions():
            header = message["result"]
            print(
                    f"[Block: {header['number']}]"
                    f" Gas limit %: "
                    f"{header['gasUsed'] / header['gasLimit'] * 100:.2f}%"
                )

            # if some cancel condition:
            #     await w3.eth.unsubscribe(subscription_id)
            #     break


asyncio.run(ws_subscription_example())

Sample output:

[Block: 19486646] Gas limit %: 41.48%
[Block: 19486647] Gas limit %: 46.91%
[Block: 19486648] Gas limit %: 52.80%
[Block: 19486649] Gas limit %: 40.90%
[Block: 19486650] Gas limit %: 68.51%
[Block: 19486651] Gas limit %: 30.09%

Hopefully this example is an effective starting point, but when you're ready to dig deeper, you'll find additional context and pattern options in the documentation. We've also created a sample Discord bot repo that leverages another set of patterns.

Upgrade path from WebsocketProviderV2

There are only a couple of notable user-facing changes to the API in the v7 release.

• Instantiation of the provider is simplified by no longer requiring the persistent_websocket method:

# WebsocketProviderV2:
AsyncWeb3.persistent_websocket(WebsocketProviderV2('...'))

# WebSocketProvider:
AsyncWeb3(WebSocketProvider('...'))

• Handling of subscription messages has a more flexible namespace:

# WebsocketProviderV2:
async for message in w3.ws.process_subscriptions():
  ...

# WebSocketProvider:
async for message in w3.socket.process_subscriptions():
  ...

The namespace update from ws to socket supports the AsyncIPCProvider in addition to the WebSocketProvider, i.e., socket may refer to an IPC socket in another context.

Wrapping Up

If you're new to websockets or asynchronous programming in Python, there are plenty of opportunities to get tripped up along the way. Note that web3.py largely mimics the established patterns of the websockets library. If you spend some time in their documentation, many concepts learned will be transferable.

Download the latest beta version and take the provider out for a spin. Let us know if your pain points are addressed and report any issues you run up against – we want to hear from you in the GitHub issues and in the Discord server! Happy building! 🚀