web3.py Patterns: Middleware

The latest major version (v7) of web3.py includes a reimagining of the middleware architecture. The short version: the functional programming paradigm has been replaced with class-based middleware. The refactor provides clearer control over request and response processing, and clears the path for new features, including batch transactions.

This post will cover a quick introduction to middleware within web3.py, how to configure the middleware stack, and how to create custom middleware of your own. If you'd like to write your own middleware and are ready to skip straight to sample code, you can find that here.

The big picture

Within web3.py, middleware is just some logic that gets executed before a request is made, after a response is received, or both.

When you create an instance of the Web3 class, you get a set of default middleware under the hood, but you can add or remove middleware as needed. The default middleware adds convenience functionality, e.g., converting to and from human-readable data types, support for ENS names, and basic input validation.

Conceptually, we use an onion as an analogy for how middleware gets executed as a request makes it way through web3.py:

Let's run through the ENS example. By default, Ethereum nodes have no notion of ENS addresses; the registry of names and associated addresses lives at the smart contract layer. In order to look up the balance of a .eth address, you must first convert the domain (e.g., example.eth) to a hex address (e.g., 0x51ABa26...). Thanks to the ens_name_to_address middleware, web3.py will resolve any ENS domains under the hood before making a request to your node.

In this case, the ens_name_to_address middleware does not apply any logic to the return value in the response; the integer value passes through that layer of the middleware unchanged.

The pattern

Each middleware is a class that defines a request_processor, response_processor, or both.

Note that asynchronous middleware is also supported via async_request_processor and async_response_processor, but are omitted here for brevity. You'll find both the synchronous and asynchronous methods documented here.

from web3.middleware.base import Web3Middleware

class CustomMiddleware(Web3Middleware):
    def request_processor(self, method, params):
        # Pre-request processing goes here before passing to the next middleware.
        return (method, params)

    def response_processor(self, method, response):
        # Response processing goes here before passing to the next middleware.
        return response

If you need even greater control, you can "wrap" the make_request function instead, by overriding the wrap_make_request method. Defining the middleware function within will look familiar if you've needed to write custom middleware in prior version of web3.py. TL;DR: within one method, you can include request pre-processing, conditionally make the request, then process the results:

from web3.middleware.base import Web3Middleware

class CustomMiddleware(Web3Middleware):
    def wrap_make_request(self, make_request):
        def middleware(method, params):
            # pre-request processing goes here
            response = make_request(method, params)
            # response processing goes here
            return response

        return middleware

Again, you'll find the asynchronous pattern (async_wrap_make_request) available in the docs here.

The middleware stack API

Whether its middleware provided by web3.py, or that you create yourself, you can add or remove it from your middleware stack using the methods available on the middleware_onion:

w3 = Web3(...)

# add a middleware to the stack:
w3.middleware_onion.add(CustomMiddleware, name="custom_middleware")

# remove a middleware from the stack:
w3.middleware_onion.remove("custom_middleware")

The API also includes inject, replace, and clearing the middleware stack. You may also choose to instantiate your Web3 object with a custom list of middleware, but note that it will replace the default middleware. For example:

w3 = AsyncWeb3(AsyncHTTPProvider("..."), middleware=[my_middleware1, my_middleware2])

Next steps

Want to learn more about middleware included in the box? The middleware docs have additional context, including lists of default and other available middleware. 📚

Writing your own middleware? This repo includes minimalist synchronous and asynchronous examples. Print statements are used to simulate more sophisticated logging middleware. 🧱

Still have questions? Join the community! The Ethereum Python Community Discord server is a great place to give and receive support as you build. 🤝