Ethereum Python Roadmap - Part 2

A month ago I posted some information on the python ecosystem for Ethereum development.  You can read that post here on the /r/ethereum subreddit.

Since then a lot has happened so I believe it is time for an update.

Ethereum TestRPC and Tester Client

• https://github.com/pipermerriam/eth-testrpc/
• https://pypi.python.org/pypi/eth-testrpc
• https://github.com/pipermerriam/ethereum-tester-client
• https://pypi.python.org/pypi/ethereum-tester-client

These two libraries are both abstractions on top of the ethereum.tester EVM from the pyethereum repository.  

If you would like to interact with the test EVM directly from your python code I recommend using the ethereum-tester-client library.  It implements the vast majority of the APIs that geth exposes over its various interfaces.

If you need to test RPC interactions then I suggest using eth-testrpc which implements the vast majority of the JSON-RPC inferface that is exposed by geth. Under the hood, it is merely a layer on top of the ethereum-tester-client library.

Both libraries support python 2.7, 3.4, and 3.5.

Web3.py

• https://github.com/pipermerriam/web3.py
• https://pypi.python.org/pypi/web3

The web3.py codebase is now ready for public consumption.  The entire codebase has been overhauled, adding significant test coverage, and expanding the library to implement very close to 100% of the functionality that users of the Javascript Web3.js library are familiar with.

The one exception to this is the contract interface which I chose to implement according to the EIP68 specification. I believe that this specification provides a much cleaner manner of interacting with contracts and makes it much clearer to users whether they are sending a transaction or just calling a contract function.

This library supports python 2.7, 3.4, and 3.5.

One of my favorite parts of the library is the TestRPCProvider.  This provider automatically spins up the eth-testrpc server for you so that you can test your web3.py code against an in memory EVM.  This provides lightning fast tests with a high guarantee that any code you write will also work when interacting directly with the geth backed JSON-RPC server.

py-geth and py-solc

• https://github.com/pipermerriam/py-geth
• https://pypi.python.org/pypi/py-geth
• https://github.com/pipermerriam/py-solc
• https://pypi.python.org/pypi/py-solc

Both of these libraries provide python wrappers around the underlying executables.

py-solc provides the high level compile_files and compile_sources functions for Solidity compilation from within your python code.  It also provides a link_code function to handle library linking.  If you would like to do lower level things you can also dig into the code and should be able to find access to whatever parts of the underlying solc api that you need to access via the solc.wrapper.solc_wrapper function.

In the same vein, py-geth provides high level tools for running the go-ethereum geth client from your python code.  It provides the classes LiveGethProcess and TestnetGethProcess that are pre-configured to run the main and morden networks respectively as well as the DevGethProcess which is intended for running local private blockchains for things like testing.

All of these classes have the following friendly APIs.

• GethProcess.accounts returns the list of account addresses
• GethProcess.data_dir returns the path to the underlying data directory for the geth process.
• GethProcess.rpc_enabled/rpc_port/rpc_host for information about the JSON-RPC interface.
• GethProcess.is_rpc_ready which returns whether the JSON-RPC server is up and listening.
• GethProcess.wait_for_rpc(timeout=0) which blocks for up to timeout seconds waiting for the JSON-RPC server to come online.
• GethProcess.ipc_*/is_ipc_ready/wait_for_ipc(timeout=0) for identical functionality to the corresponding rpc methods and properties.

The py-geth library also provides a few mixin classes that can be used to do things like writing all of the geth output to stdout/stderr, or piping it into a logger via the geth.mixins module.  Examples available in the project readme.

Both libraries supports python 2.7, 3.4, and 3.5.

Populus

All of the above libraries were precursors to the main work I've been wanting to do.  Populus is currently in a beta release cycle for a 1.0 stable release.  

Part of this release involved trying to establish a baseline for stability in the library so that some of the more advanced features I've been dreaming up are possible.  This meant removing two major commands, populus web and populus attach.  These command respectively ran a webserver and spun up a python console.  They were both poorly supported, sloppily implemented, and very fragile.

In addition, there were large parts of the populus codebase that would be better served as standalone python packages.  The py-solc and py-geth libraries both removed large pieces of complex subprocess based logic.  The work on web3.py has allowed removal of very large portions of code from the  codebase while simultaneously making the blockchain interactions much more robust and reliable.

All in all these refactors and repackaging allowed me to remove 13,000 lines of code, tests, and fixtures from the populus codebase.

The optimist in me is hoping to get 1.0 released this week.  The realist in me thinks it's more likely to be another week away.  I'm working hard on populus migrate which was conceptually inspired by truffle migrate.  Anyone who is familiar with Django migrations should immediately recognize the architectural similarity.  The goal is to provide a framework to automate complex deployments as well as setting the stage for a full packaging and releasing system.

Populus supports python 2.7, 3.4, and 3.5.

Deprecation

Part of this process has been deciding to deprecate some of the libraries that were initially part of this refactor but became obsolete.  ethereum-rpc-client, ethereum-ipc-client, and ethereum-contract will no longer be supported.  The role they once filled has been replaced by better things in other libraries.

I hope you'll join me in continuing to advance the python toolchain for the Etherum ecosystem.  Happy Hacking!