“There are five practical reasons that we write tests. Whether we realize it or not, our personal testing philosophy is based on how we judge the relative importance of these reasons.” - Sarah Mei This episode discusses the factors.
Transcript for episode 178 of the Test & Code Podcast
This transcript starts as an auto generated transcript.
PRs welcome if you want to help fix any errors.
00:00:00 There are five practical reasons that we write tests. Whether we realize it or not. Our personal testing philosophy is based on how we judge the relative importance of these reasons.
00:00:10 Thank you to PyCharm for sponsoring this episode.
00:00:13 Pycharm helps me to understand.
00:00:14 End and play with my code. The refactoring tools are amazing. A simple one is just to rename a method and it just gets renamed everywhere. There’s a whole bunch of other cool refactoring tools as well. If I changed a bunch of code, I can visually see the difference of my code and the get repo code, and I can even visually walk through the local history to see all of my changes. I actually love refactoring and PyCharm helps me have fun while I’m doing it. Try PyCharm Pro for four months by going to testandcode.com.
00:00:45 Pycharm support for test and code comes from Sauce Labs, the end to end testing solutions provider that helps development teams build a digital application so they work exactly as they should on every browser, OS and device. Every single time. They offer full stack and full software development lifecycle testing, including low code, mobile app, mobile beta, API, error monitoring and reporting, cross browser, UI, visual and automation. Visit saucelabs.com Testbetter for more information and a free trial. Saucelabs Test continuously test smarter.
00:01:17 Develop with confidence.
00:01:33 Welcome to Test & Code.
00:01:43 That quote in the intro was from Sarah May. She wrote an article called Five Factor Testing, which I’ll link in the show notes. Of course, describing these five factors. There’s a lot of great stuff in here as a setup for the problem. In understanding why we test. Sarah has a great intro. Here are some of the snippets from it. When I took my first real Dev job in the late 1990s, it was not common for developers to write their own automated tests. But times have changed. Today on most teams, writing automated tests is a normal part of the software developers job. Changes to a code base usually aren’t considered complete until there are at least some automated tests exercising them, usually written by the developer who changed the code.
00:02:28 In the late 1990s, when having any developer written tests was weird, it was hard to imagine that 20 years later, having no developer written tests would be weird. But here we are. This mostly matches my experience, but I know the experience varies drastically across companies and even across teams within a company, but there’s no doubt that developer written tests is way more common now. But many still don’t like to write tests. Sarah goes on, Sometimes we write tests just to do it, whether or not it’s actually useful. After that’s been going on for a while, you’ll hear things like the tests are so slow or flaky or unpredictable. We don’t have the test coverage we need, but we don’t have time to update it. Writing tests just doubles the amount of time the story takes for no useful reason. The story is finished, I just have to write some tests. This is terrible and concerning, and it makes my heart ache.
00:03:25 Okay, so there’s a fix. Sarah recommends fixing this by reconnecting with the underlying needs that originally drove us to write tests in the first place. So what do tests give us? Here are Sarah’s five factors on what tests do for us. One, they verify the code is working correctly. Two, they prevent future regressions. Three, they document the code’s behavior. Four, they provide design guidance, and five, they support refactoring. There’s discussion of each in the article, but of course, I’m going to give you my own take on these. Five, for the most part, we agree on these. However, I’m going to rearrange them in my order of importance.
00:04:10 For the code I write first, verify the code is working correctly. Duh. That’s why we have tests, but it’s not so simple. I want to make sure the code is working correctly from the perspective of the user of the code, be that the end user or another Dev that’s using my library, but almost always from some interface or API. Secondarily, for utilities and algorithmic bits that I squirrel away to make my code simpler, I’m the customer, so I write test to verify that the code does what I wanted to do. I’m just trying to be specific on what I mean by working correctly. Second, one is prevent future regressions. I was going to put support refactoring second, but I feel guilty doing that. So sure, let’s do prevent future regressions. That can be. Second, we put all this time into getting the code right once. It’s really nice to make sure that it keeps working in the future.
00:05:07 Regressions is something we forget about, but it’s really nice to be able to catch things and make sure things don’t get broken again. Third, support refactoring. This was last on her list. I’m putting it third. Now we’re talking. This is where it gets fun. I like refactoring more accurately. I like building the first draft of my code as fast as possible, just to make sure that things can work, that I can solve the problem in some way. I like developing tests at the same time, inching my way through the test cases to make sure the happy path and the edge cases are covered. And then I can go back and make the code something I’m proud of, or at the very least, something I’m cool with. Submitting for a PR and a code review. Trusting my test suite and utilizing version control. It allows me to wholesale refactor and rewrite chunks of my code without worrying about breaking something. And I can really kind of make it a design that I’m happy with. It’s so fun to solve a problem you’ve already figured out. Also, have you ever taken the time to get through a video game level and then gone back and done it again and again? Maybe just because it’s fun trying to replay, maybe for more points, or for a better time, or just because you enjoy it. Let coding be like that and refactor with the supportive tests. Next is provide design guidance.
00:06:26 I’m begrudgingly keeping this in fourth place, but for a different reason, as Sarah. Sarah correctly describes this as the most controversial claim of testing advocates. And I agree many testing advocates, especially people advocating unit tests, wave this flag.
00:06:44 The idea is that if you test all the small bits, you’ll naturally produce more modular systems. And somehow that’s a better design. It might be. I just personally don’t follow the logic. I think that the best designs are done by someone who’s already solved the problem once, not necessarily because of modularity. Modularity might be part of it. Modularity is great, but isolated bits aren’t necessarily better. They actually might make things harder to refactor. If you’ve got tests around all bits, if you’ve got a bunch of unit tests wrapped around each bit of code, then how do you refactor? How do you change things around without deleting some tests? So why am I leaving design guidance here at number four? Because tests allow you to actually use your code and you’ll build better APIs, better function signatures, et cetera. Here’s the idea. Your code will have at least one user. A library interface will have other code calling it a function will have some other code calling it an API, will have some other system using the API. Writing tests against these interfaces forces you to use the interfaces. If it sucks to write the test against it, you’ll have time to fix the interface to make the test writing, not suck. This will improve the interface so that your end user, your customer, your fellow Dev, has a better experience.
00:08:01 You having to deal with the bad interface first while writing the tests and fixing it. Save others, saves others from this horrible interface. To that extent, your design will be better and that the interfaces will be better. I’ve written perfectly reasonable API methods. I thought that when testing, I realized that one of the parameters to the function wasn’t knowable by the caller in some circumstances, and I had to go back and refactor the code to make that parameter either optional or just take it out and move it somewhere else. The other way that testing improves design is aiding refactoring. Test allows you to play with the code like clay until you have something that you’re proud to give to your mum for Mother’s Day. Okay, big tangent. I think it’d be really cool to have shops that do primary school art parties for grownups. You sit down, have some wine and cheese and glue macaroni and string to a picture we can give to our kids as presents for upcoming holidays. Maybe stab a hunk of clay with a stick a handful of times, Bake it in a kiln, and there you have a pen holder for your kid. Brilliant. Anyway, back to testing last on the list for me is document your code’s behavior. I can’t remember where Sarah had it up on the list but it was higher up. I’m putting it last because I hardly ever think about this when writing tests. I should it’s a good idea. I love it when I’m a bit confused about a library. Look at the tests and go oh that’s how you’re supposed to use it. Tests can make great executable examples of how to use some code and in what order the methods should be called or at least one order that works. It’s a cool idea and I should do it more. Anyway, thanks Sarah for writing this great article. I love it.
00:09:49 Thank you SauceLabs for sponsoring Sauce labs test continuously test smarter develop with confidence Visit saucelabs.com Testbetter for more information and a free trial thank you PyCharm for sponsoring visit Test And Code. Compaijarm for a four month free trial of PyCharm save time use PyCharm thank you Patreon supporter join them at testandcode.com support all of those links are in the show notes at testandcode.com that’s all for now.
00:10:17 Now go out and test something.