Echidna Workshop Streaming Series Part 1

This is the first part of a six-part series on Echidna, where the basics of Echidna are covered. The speaker talks about some housekeeping items and provides an overview of what will be covered in this workshop.

Introduction

• The workshop begins with music.

• The speaker introduces himself and welcomes everyone to the first Echidna Workshop streaming series.

• The speaker talks about the agenda for today's workshop and what will be covered in each section.

How to Find Bugs

• The speaker discusses different methodologies that can be employed to find bugs in code.

Fuzzing Methodologies

• The speaker explains what fuzzing is and how it fits into testing methodologies.

Interactive Exercises

• The speaker demonstrates how to use Echidna for fuzzing through interactive exercises.

Defining Good Invariance

• The last section covers how to define good invariance, which sets up for next week's workshop.

Overall, this transcript covers an introduction to Echidna, including its basics, methodologies for finding bugs, fuzzing techniques using Echidna, and defining good invariance.

Introduction to Testing Techniques

This section introduces different testing techniques that can be used to find bugs in smart contracts. The speaker discusses the benefits and limitations of each technique.

Unit Testing

• Unit testing is a well-known technique that covers expected values.

• It has limitations as it may miss unexpected values, leading to edge cases.

• A unit test is just a dot on the input space line, unable to cover the entire breadth of input space.

• Edge cases are usually found through other testing methodologies.

Manual Review

• Manual review theoretically finds every possible bug in the code base.

• However, it's time-consuming, requires specific skill sets, and does not track code changes.

• Security audits are an example of manual review but are snapshots in time.

Fully Automated Analysis

• Fully automated analysis is quick and easy to use.

• Slither is an example of fully automated analysis but has limitations on how many types of bugs it can find.

Semi-Automated Analysis

• Semi-Automated Analysis is great for logic-related bugs that are hard to find with previous methodologies.

• Echidna is a semi-automatic analysis technique where humans guide the automated piece.

Introduction to Fuzz Testing

In this section, the speaker introduces fuzz testing and explains how it differs from traditional testing. They also introduce property-based testing as a way to approach fuzz testing in the smart contract world.

Fuzz Testing vs Traditional Testing

• Fuzz testing is different from traditional testing because it doesn't aim to crash the program with random inputs.

• Smart contracts don't technically have crashes, so we need a different approach to fuzzing.

• Property-based testing is a way to approach fuzzing in the smart contract world.

What is Property-Based Testing?

• Property-based testing involves defining properties that should always be true given some preconditions.

• The fuzzer generates random values and tests whether the properties hold or not.

• The goal of property-based testing is to test whether the system behaves as expected.

Examples of Properties

• An example invariant for an ERC20 token is that no user's balance should exceed the total supply of tokens.

• System properties define the expected behavior of a system or function.

Introduction to Echidna

In this section, the speaker introduces Echidna, an open-source tool used for semi-automated analysis in auditing smart contracts. The speaker explains that Echidna requires a target contract and properties to test.

How Echidna Works

• Echidna needs a Target contract and properties to test.

• The Target contract is what Echidna is fuzzing, while the properties are what it's trying to validate.

• With these two inputs, Echidna can try to validate or invalidate the system behavior by breaking the invariant.

Summary of Finding Bugs with Semi-Automated Analysis

In this section, the speaker summarizes how to find bugs using semi-automated analysis and property-based testing.

Key Considerations for Using Echidna

• Think of Echidna as an externally owned account that sends out transactions really fast with random values.

• Echidna calls a sequence of functions with random inputs on both the Target and inherited contracts.

• Properties resolve to a truthy value, which means they must be true.

Benefits of Property-Based Testing

• Property-based testing is good at validating system behavior and finding hard-to-find bugs in codebases.

Exercise One: Checking for Correct Arithmetic

In this section, the instructor introduces exercise one and explains that they will be using Solidity 0.7 to check for correct arithmetic. They also discuss the Target contract and the transfer function.

Introduction to Exercise One

• Exercise one involves checking for correct arithmetic using Solidity 0.7.

• The Target contract is introduced, along with the token contract which inherits ownership.

• The transfer function is discussed as being important for exercise one.

Goals of Exercise One

• Goal 1: Add a property to check that Echidna caller cannot have more than an initial balance of 10,000 tokens.

• Echidna caller is defined as message.sender or the person sending transactions to the Target contract.

• Goal 2: Fix any bugs found and recheck properties with Echidna.

Setting Up Test Token Contract

• A new file called niche.soul is created with Solidity version set to 0.7.

• The token.sol file is imported into niche.soul.

• Inheritance is used so that test token can inherit all functions from token and ownership from plausible.

Setting Up Echidna

• Setup involves defining a constructor and setting up Echidna collar's balance at ten thousand tokens.

• Properties are defined in a specific format that includes a public function returning a Boolean value.

Introduction to Property-Based Testing

In this section, the speaker introduces property-based testing and explains how it differs from traditional testing methods.

What is Property-Based Testing?

• Property-based testing involves defining properties that a function or program should satisfy, rather than writing specific test cases.

• A property is either true or false. If the computation in a function returns true, the property holds. If it returns false, the property does not hold.

Writing a Property Test

• The speaker challenges Echidna to write a property test for the balance of Echidna caller. The test should validate that Echidna caller's balance will always be less than or equal to ten thousand.

• Echidna can either validate this or find an edge case where this is not valid.

Using Echidna for Property-Based Testing

In this section, the speaker demonstrates how to use Echidna for property-based testing.

Setting Up Solidity Version and Running Echidna

• Use solc-select to set up the right solidity version.

• Call Echidna test, which is the binary for Echidna.

• Compile everything in niche.sol using test token as the target contract.

• Echidna will parse through all state-changing functions and identify those with function signatures that match its second input (i.e., properties).

Finding Bugs with Echidna

• When running Echidna test, Echidna will call a sequence of functions to find bugs.

• In this example, Echidna calls the transfer function and sends a huge amount of tokens to an address.

• Because underflow and overflow happen silently in Solidity 0.7.0, if the value sent is more than 10,000, it will underflow and result in a balance greater than 10,000.

• The speaker explains how to fix the bug by taking the balance of msg.sender and the recipient beforehand and ensuring that the new balances are less than or equal to the previous ones.

Conclusion

In this section, the speaker emphasizes that property-based testing is important for testing edge cases and preventing vulnerabilities in smart contracts.

Importance of Property-Based Testing

• The speaker explains that although concretizing values may seem trivial, what matters is what we're testing: can someone steal funds from the system or violate properties?

• Property-based testing indirectly tests for vulnerabilities by checking if initial balances can be increased without explanation.

Running Echidna on a Target Contract

In this section, the speaker discusses how to run Echidna on a target contract and provides properties for testing.

Setting up Exercise 1

• The speaker shows the template that was set up for Exercise 1.

• A property was set, and Echidna broke it by sending a value of more than 10,000 which caused an underflow.

Validating Rob Bytecode with Echidna

• Echidna needs an ABI in addition to bytecode to validate Rob bytecode.

• Providing both the bytecode and ABI is equivalent since the ABI helps determine state-changing functions and their input parameters.

Resetting Contract State

• The contract state is reset after a sequence of function calls except for what was constructed initially in the constructor.

Using Echidna with Python

• Homebrew can be used on Macs instead of running the Docker image. More information can be found on the Echidna readme or by opening an issue on the Echidna issue tracker.

Exercise 2: Checking Access Controls

In this section, the speaker discusses exercise 2 where access controls are checked.

Setting up Exercise 2

• Same token setup as before but checking for access controls.

• The goal is to pause the contract and revoke ownership in setup.

• Pausing sets pause to true while resuming sets it to false.

• Ownership sends owner to mesh.center.

• Since Echidna deploys this system, it is also its owner allowing pausing in setup.

Adding Property to Check Unpausing Contract

• Once paused, no one can resume since no one owns it after revoking ownership.

Introduction to Testing with Echidna

In this section, the speaker introduces Echidna and explains how it can be used for testing smart contracts. They also explain the importance of property-based testing and how it differs from assertion testing.

Using Echidna for Property-Based Testing

• The speaker shows an example of importing a solidity file into Echidna.

• They explain the setup constructor and why it's important to pause the system before revoking ownership.

• The speaker demonstrates how to write a property function in Echidna using the test plausible function.

• They challenge Canada to prove that isPaused is false.

Finding Bugs with Echidna

• The speaker explains that by setting isPaused to false, they can take access of the owner and unpause the system.

• They demonstrate how Echidna found a sequence of calls that was able to make isPaused false, allowing them to identify a bug in the code.

Assertion Testing vs Property-Based Testing

In this section, the speaker discusses assertion testing and how it differs from property-based testing.

Understanding Assertion Testing

• There are no bullet points associated with timestamps in this section.

Understanding Property-Based Testing

• The speaker explains that property-based testing is more general than assertion testing because it tests properties rather than specific inputs or outputs.

Assertion Testing

In this section, the speaker explains the difference between assertion mode and property mode in Echidna. Assertion mode allows for more flexibility as it can take input arguments and fuzz values between zero and 256 sub Max.

Assertion Mode vs Property Mode

• Assertion mode is used to assert a property while returning a property is done in property mode.

• Assertion mode allows for more flexibility as it can take input arguments and fuzz values between zero and 256 sub Max.

• To run Echidna in assertion mode, specify test mode as assertion when calling test possible.

Exercise Two Summary

This section summarizes exercise two, which focuses on understanding how to run Echidna, write properties, understand what a setup is, and abstract simple examples into real-world code.

Key Takeaways

• Exercise two helps understand how to run Echidna, write properties, understand what a setup is, and abstract simple examples into real-world code.

• These principles can be applied to more realistic scenarios such as loss of funds or access control.

Comparison with Foundry

The speaker compares Echidna with Foundry in terms of speed, functionality, and smartness in choosing inputs.

Differences Between Echidna and Foundry

• Echidna has more features than Foundry at the moment but lacks some capabilities like vm.prank or cheat codes.

• Echidna focuses on providing more capabilities and power to the end-user while Foundry is slightly more developer-friendly.

• Echidna uses a Corpus to determine its next sequence of calls, which is unique and provides more coverage into the codebase.

Conclusion

In this transcript, we learned about assertion testing in Echidna, summarized exercise two, and compared Echidna with Foundry. The speaker provided valuable insights into the differences between these two tools and their respective strengths.

Introduction

In this section, the speaker introduces Echidna and its features.

Features of Echidna

• Echidna can fuzz block timestamp and block number.

• It is possible to configure Echidna to jump up to a week in time or a day and time in 24 hours.

• Echidna has dap mode optimization mode and other features.

Defining Good Invariance

This section discusses how to define good invariance for testing with Echidna.

Defining Good Invariance

• Start by defining the invariant in English.

• Convert the invariant into solidity by creating a Boolean statement that can be defined as a truthy value.

• Run Echidna, investigate if the invariant breaks, fix it, and repeat the process until all controls pass.

Types of Invariants

• Functional invariants are recommended for starting since they test a single function without relying on much of the system.

• System level invariants require deploying the whole system to test them.

Functional and System Level Invariance

This section discusses functional and system level invariance, which are important concepts in testing smart contracts. Functional invariance refers to ensuring that a function performs as expected, while system level invariance involves checking the overall behavior of the system.

Function Level Invariant

• Functional invariance is tested by checking if a function performs as expected.

• A simple way to test functional invariance is through assertion mode, where values are fuzzed and arithmetic operations are checked for correctness.

• An example of this is using assertion mode to check if adding two values produces the same result regardless of their order.

System Level Invariant

• System level invariance involves checking the overall behavior of the system.

• This requires proper tracking of balances and understanding how other functions affect them.

• Initializing and setting up systems for fuzzing is crucial for testing system level invariance.

• Leveraged unit test frameworks can be used for more complex systems.

Finding Bugs with Echidna

This section discusses how Echidna can be used to find bugs in smart contracts.

Using Echidna to Find Bugs

• Echidna tracks coverage by creating an array of zeros that corresponds to each byte of bytecode.

• When a test case executes a new byte, it updates the corresponding element of the coverage map from 0 to 1.

• Echidna uses this information to generate new tests that explore untested parts of the codebase.

• By repeating this process, Echidna can eventually find bugs or vulnerabilities within a contract.

Introduction to Echidna

In this section, the speaker introduces Echidna and explains how it works.

What is Echidna?

• Echidna is a property-based fuzzer for Ethereum smart contracts.

• It uses symbolic execution to generate test cases that explore the behavior of a contract.

How does Echidna work?

• When Echidna finds new coverage, it stores that transaction sequence in a different data structure called the Corpus.

• The Corpus is the interesting stuff that Echidna found to be cool.

• To find something that's cool, you check whether coverage increases.

Mutation and Splicing

• When Echidna wants to reuse something from the Corpus, it will take that thing from the data structure and deserialize it into an object that it can work with.

• Then, it'll mutate stuff along the way using educated randomness based on information from previous runs.

• There's also something called splicing where you can add stuff in the middle of a sequence or after a sequence.

Testing Smart Contracts with Echidna

In this section, the speaker explains how to use Echidna to test smart contracts.

Revisiting an Example

• The speaker revisits an example of a buy function for tokens in a smart contract.

• The valid buy function is stateless and just takes in two values and does some arithmetic to pass or fail.

Functional Invariance

• The idea of functional invariance is to start with the valid buy function and build up from there.

• One important invariant for this function is that if you give no money, you should not be able to get any tokens out.

Testing with Echidna

• To test the valid buy function, you can use assertion mode and assert that if way sent is zero, desired tokens should also be zero.

• A precondition for testing this invariant is that desired amount must be greater than zero.

Finding Bugs with Property-Based Testing

In this section, the speaker discusses how property-based testing can help identify logical flaws in code bases. They use Echidna to test a smart contract and find a bug that allows users to mint tokens for free.

Property-Based Testing

• Property-based testing is an important technique for finding difficult logical flaws in code bases.

• Echidna is a tool used for property-based testing of smart contracts.

• Properties are Boolean statements that are either true or false.

• You can test properties in property mode or assertion mode.

Finding a Bug with Echidna

• The speaker uses Echidna to test a smart contract and finds a bug that allows users to mint tokens for free.

• The bug occurs due to rounding errors when dividing by 10 in Solidity.

• This bug indirectly impacts other functions in the smart contract as well.

Next Steps

• In the next workshop, they will focus on fuzzing aptk math, which is a fixed-point math library.

• They will also discuss function-level testing and debugging and optimizing fuzz tests.

• Participants are encouraged to complete exercises 3 to 6 in building secure contracts if they feel comfortable with the material.

Conclusion and Future Workshops

In this section, the speaker concludes the workshop and invites participants to attend future workshops.

Workshop Conclusion

• The speaker thanks the participants for joining the workshop.

• The speaker invites participants to attend future workshops.

• The next stream will be on Tuesday, November 22nd at 12 p.m. with the same speaker.

• The date for the third stream will be announced next week.

Final Thoughts

• No further questions were asked by participants.

• The speaker hopes that everyone enjoyed the workshop.