Insights from my Software Engineering Radio interview with Zac Hatfield-Dodds
Introduction
In a recent episode of the Software Engineering Radio podcast, I had the pleasure of interviewing Zac Hatfield-Dodds, a leading researcher, software engineer, and maintainer of open-source software. Zac currently leads the assurance team at Anthropic, an artificial intelligence (AI) safety and research company. He is also known for his contributions to open source projects like Hypothesis, Pytest, and HypoFuzz. We discussed property-based testing and how it can be effectively implemented using Hypothesis, an advanced testing library for Python. I’m really thankful that Zac took the time to share his knowledge with me!
Insights
I learned a lot from Zac during this interview. Here are some of his key insights:
What is property-based testing and how does it work with the Hypothesis tool?
“Property-based testing gives us a different attitude to what we check and a much richer set of tooling to help computers generate those inputs for us. … So when I think about writing software tests and the kind of unit test style, maybe they’re testing small units, maybe they’re testing whole programs. There are basically two problems that you have to solve in order to have a software test. The first is that you need to work out what kind of input data you run, and the second is that you need to check in some way that your code didn’t do the wrong thing.”
What challenges or limitations may developers face when using Hypothesis?
“The first thing I do wanna say before I get into the downsides is that I think almost every good test suite will involve some or maybe many example based tests. … The problem with basing your entire test suite on example based tests is just that it takes a long time and it gets really tedious to write them. … And so I think most of the downsides are a combination of requiring a lot of manual effort to define a rigorous test suite like this. And also that it’s really easy to miss the edge cases that you didn’t think of.”
How does Hypothesis work and how can developers effectively use it?
“What you do with Hypothesis is you write a test function which should work for any valid input and then you describe to Hypothesis what kind of inputs are valid. … And then Hypothesis can try to generate many valid inputs according to that specification you gave it looking for one which makes your test fail.”
Listen
In conclusion, if you want to explore property-based testing for Python programs and the Hypothesis tool, I strongly suggest that you listen to Zac Hatfield-Dodds on Property-Based Testing from Software Engineering Radio! You can find it on your favorite podcast player, Apple Podcasts, Spotify, YouTube, or listen to it with this handy podcast player.