November 29, 2022: Titanoboa Testing 🐍🕵️
$crvUSD successfully ports Brownie tests into Vyper's Titanoboa
If you’re among those asking “Wen $crvUSD,” we invite you to marvel at the complexities of the tests directory in its Github repo for a sense. A product of this complexity requires testing many unusual edge cases. Once it launches, there’s no going back, so you have to carefully consider the nearly infinite complexities which could emerge.
Thoroughly stress testing the least probable nooks and crannies of cutting-edge formulae is how Curve smart contracts stay relatively safe, while the rest of DeFi seemingly suffers nonstop hacks.
One of the most notable innovations on display in the new $crvUSD repository is the first known use of Titanoboa for comprehensive testing. Previously most Curve repositories used Brownie, with newer registries like metaregistry moving to the newer Ape Framework. To handle some of the complexities of $crvUSD, the newest repository primarily rebuilt testing from the ground up using Titanoboa to handle its complex tests.
What follows is your cheat sheet for studying these tests.
Both the underlying math and the corresponding tests for $crvUSD are quite in-depth. The tests make frequent use of parameterization to test over a wide range of potential values in search of an error. Using Titanoboa to execute tests locally and efficiently makes a big difference here in terms of processing time. Such tests might ordinarily run over several hours, but here the entire testing suite runs in a smooth 0:46 minutes on a Mac M1.
For users who want to use Titanoboa for your testing, the Github repository indeed provides a masterclass. The suite only needed a few native boa functions mixed with Pytest to replicate the functionality of a full smart contract testing suite. Since the functions are too new to be thoroughly documented, here are a few of the functions on display in the repo:
tests/conftest.pyThe conftest.py file in Pytest hosts fixtures, or functions which can be quickly loaded into other tests. For example, the handy approx() function allows tests to be run to a desired level of precision. This function proves so useful it gets upgraded to sit in the global scope, making it easily accessible throughout.
This conftest.py file thereby serves as the global foundation of the entire test suite, and thus serves as a good demonstration of how to set up boa for smart contract testing.
Up top we see some basic table setting:
boa.interpret.set_cache_dir()preserves compilations across sessionsboa.reset_env()starts the repo with a clean “environment”
Within Titanoboa, the concept of an “environment” references all the loaded contracts, and contains some useful built-in functions. The environment functionality can be accessed through the boa import directly as boa.env
Continuing through the top level conftest.py, we compare other testing suites like Brownie generate test accounts for you by pulling test accounts directly created from ganache. Since we’re working directly against the EVM, we have to generate the accounts directly with the code on line 25 to populate a list inline with ten items.
Note these generated addresses therefore come out not like the standard semi-randomized Ethereum addresses from Ganache, but are generated from scratch, meaning you get sequential addresses mostly left-filled with zeroes like above.
Finally, from lines 33 to the end of the file we directly load a few helpful contract stubs for testing. These are some simple test contracts representing mock ERC-20 tokens and oracles loaded directly in memory using the boa.load() functionality.
We also distinguish boa.load() from a similar function boa.load_partial(), demonstrated in another conftest.py file:
tests/lendborrow/conftest.py
The two functions work in similar ways to read and process the raw Vyper bytecode. The difference comes in that running load() returns a deployed Contract. Executing load_partial() returns a VyperDeployer object, which looks as follows:
So later, when we hit testing, objects loaded partially have these functions available. Here’s an example:
tests/lendborrow/test_factory.py
Incidentally, the presence of testing a factory that can deploy a market using any mix of collateral token by passing an oracle and monetary policy strongly suggests that $crvUSD will be mintable with more than just Ethereum.
With all our contracts loaded in assorted conftest.py files, we can flesh out the full suite of tests. Most of these are just python and calls to the loaded smart contracts, but a few additional boa keywords makes this easy.
tests/lendborrow/test_create_repay.py
This example is a simple test to make sure that a loan has its basic parameters zeroed out on repayment — all these assertions are contained in two indented blocks of boa code, which demonstrate the prank and anchor keywords.
boa.env.prank()
Any code executed within a boa.env.prank() block the compiler will LARP as the particular user. Here’s the source:
Who knows what mischief you can get into when you masquerade as random crypto wallets, making prank indeed a good name.
boa.env.anchor()
Anchor is a fun one. It deploys an anchor, then when the block of code is done it returns everything back to the anchor. Allows for making several state changing operations and quickly reverting.
boa.env.time_travel()
For another example, we need to see how the LLAMMA will respond when the oracle price changes. Applying an exponential moving average (EMA) to the oracle price is a good way to smooth out sudden spikes, so here the code observes the effects of a sudden change in oracle price.
tests/amm/test_price_oracles.py
Here the test checks the price, cuts the price in half, and see how it looks in both the near and far future. The Titanoboa source code here is quite simple:
The magic number “12” refers to the fixed number of seconds per block since the merge made calculating block time relatively easy.
boa.reverts()
Fairly self-explanatory for anybody who’s used other smart contract testing suites, the revert statement checks if the execution of the smart contract reverts, and optionally if it throws the correct error message.
By way of example:
tests/test_math.py
This higher level test is reviewing some of the new Vyper math libraries created for optimizing gas efficiency when computing some of the more advanced methods seen in the white paper. Here it plugs in random integers within a uint256 and looking for failure cases. The boa.reverts() keyword is utilized to see if large powers are returning an “exp overflow”
The related Titanoboa source:
Even beyond the usage of boa, the suite of tests deserves acclaim for showing off the more advanced features of PyTest. For instance, stateful testing allows you to define many basic interactions as rules and then iterate through many combinations of such rules to try to detect edge cases from the sort of bizarre user behavior that can actually happen.
Using stateful testing, we can define all basic ways a user could interact with $crvUSD, rules like repay, add_collateral, create_loan, borrow_more, time_travel, and change_rate. Once this legwork is done, you can then iterate over many permutations of these behaviors, as shown here:
tests/lendborrow/test_create_repay_stateful.py
We imagine as Titanoboa development continues many of the aforementioned features could get further streamlined. For instance, we see no usage of the Vyper inline gas profiler which could be a gas golfer’s dream. Even at this early stage, it’s already proving hearty enough to safeguard potentially large quantities of Ethereum. Bullish boa!
For more reading on Titanoboa, the best resource is the Vyper Discord. We also have some additional prior content on the subject:
Happened.
I'm amazed at the complexity of the tests.
👍😁