The above figure shows the overview of ContraMaster, which is driven by a grey-box fuzzing loop. Given a set of initial seeds, ContraMaster randomly synthesizes a set of transaction sequences and picks one from the pool in each iteration of the fuzzing loop. ContraMaster runs each transaction in sequence on an instrumented EVM. When the transaction is finished, ContraMaster verifies the contract state against our semantic test oracle. If a violation is detected, ContraMaster reports a vulnerability and presents the generated attack contract and transaction sequence which can be used to reproduce the exploit. Otherwise, ContraMaster collects runtime execution information to guide the test sequence generation for the next iteration. The information collected mainly includes control-, data-flow graph and contract state information.
Furthermore, ContraMaster is equipped with a number of new mutation operators customized for smart contracts. In addition to function inputs used in traditional fuzzers, we also use gas limits, fallback functions, transaction sequences and contract states as the mutation targets. Many vulnerabilities in smart contracts require the interplay of several contracts and can only be exploited by a particular transaction sequence with the correct gas limit. Therefore, the customized mutation operators are important for triggering vulnerabilities. The newly generated inputs are added to the pool and the fuzzing process continues until it exceeds the allocated resource limits.
The table shows the time overhead taken by ContraMaster and ContraAFL, where ContraMaster performs better than ContraAFL in all contract programs. In particular, ContraMaster averagely takes 61.52 seconds to successfully launch an attack while ContraAFl spends 218.94 seconds on average, achieving a speedup of 3.56X. Furthermore, the variance in ContraMaster and ContraAFL is 2126.49 and 19998.92, respectively. Thus, ContraMaster is more stable than ContraAFL. Specially, 3 exploitable vulnerabilities cannot be found by ContraAFL in the given timeout (600 seconds) in all the experiments.
Following Klees et al.’s recommendation, we apply the Mann Whitney U-test on the time used to find the vulnerabilities. As shown in table, in most experiments, the p-values are smaller than or close to a significance level of 0.05. Thus, we conclude there exists a statistically significant difference in the time used to find the vulnerabilities, compared to ContraAFL.
To determine the extent to which ContraMaster outperforms ContraAFL, we also use Vargha and Delaney’s A12 statistical tests. From the table, we can see among benchmark experiments the resulting A12 statistic exceeds the conventionally large effect size of 0.71 in 18 out of 23 cases (78.3 %). Therefore, we conclude that the time usage in ContraMaster to find vulnerabilities is statistically different from that in ContraAFL.
There are three different types of new attacks found by ContraMaster in 26 smart contracts.
A code snippet from the smart contract CreditDepositBank is vulnerable and was detected by ContraMaster. The contract CreditDepositBank has a public function takeOver(), which can transfer ownership to the sender of the message. A sender who acquires the ownership can then call the withdraw() function to withdraw Ether of any other participant. Furthermore, the bookkeeping variable “balances” is not updated after the “send” (Line 10), thus violating the balance invariant.
The whole contract program can be obtained here.
The contract has a minimum deposit value, which is set by the owner of contract. When the participant deposits less Ether than the minimum value by invoking the payable function “deposit” (false branch at Line 3 in Fig. 9), the Ether is still deposited into the contract without any record. Therefore, the participant cannot withdraw his contribution again. As a result of this transaction, the balance of participant is reduced, but the bookkeeping variable is not changed. This violates the transaction invariant. The expected behavior should be to return the fund to the participant and roll back the transaction, if the contribution is less than the minimum requirement.
The whole contract programs, such as ETH_VAULT and WhaleGiveaway, can be obtained.
A code snippet from LZLCoin illustrates this vulnerability. The vulnerable function takes two parameters _tkA and _etA, which represent the balance a participant is able to withdraw and the Ether actually being sent out, respectively. The values of these two parameters should always be equal, otherwise, the adversary is able to choose a smaller value for _tkA and a larger value for _etA, to withdraw more than pledged. This behavior violates our balance invariant.
The whole contract programs, such as BountyHunt, LZLCoin and PowerCoin, can be obtained.
We illustrate the reasons why some vulnerabilities detected by existing techniques are not exploitable.
This contract first checks whether the balance of the message sender is zero at Line 11. If so, it throws an exception and reverts the program state. Otherwise, it sets the balance of the sender to zero at Line 12, and then uses withdrawEtherOrThrow(), the safe withdraw function, to fetch Ether at Line 13. Through this safe withdraw function, the program may re-enter the function refund(). When reentering function refund(), the balance will be set to zero. Thus, the reentrancy cannot pass the check at Line 11 again, and the program state is reverted. As a result, an adversary cannot steal Ether from this contract.
The whole contract programs, such as DaoChallenge and FunFairSale, can be obtained.
Considering the code snippet from the contract Store. At Line 5, the contract pays out Ether to the message sender and the send() operation may fail. When the send() operation fails, the contract reverts the program states at Line 8. This is in fact a correct way to handle the exception. However, it is reported as a vulnerability by both ContractFuzzer and Zeus. There is no easy way to precisely detect exception disorder without semantic understandings.
The whole contract programs, such as Store, PassDao and MillionEther, can be obtained.
Considering code snippet from JamCoin as an example. In this contract, before the integer operations at Lines 5 − 6, it checks whether the operations would produce overflow/underflow at Lines 3 − 4. However, these checks are ignored by Zeus. It requires program contexts being considered to accurately identify integer overflow/underflow.
The whole contract program can be obtained here.