“Code is read much more often than it is written.” - Guido van Rossum This is true for both production code and test code. David Seddon came up with 6 principles to help us write more readable tests. We discuss these, as well as more benefits of readable tests.
Transcript for episode 112 of the Test & Code Podcast
This transcript starts as an auto generated transcript.
PRs welcome if you want to help fix any errors.
00:00:00 Gleeband Rossum is quoted as saying code is read much more often than it is written. And I definitely find that to be true, and it’s definitely true for production code. But I think it might even be more so true for test code. When you’re trying to understand why a test is failing, you’ll be very grateful to the test author if they’ve taken the time and care to make it readable, David said, and came up with six principles to help us write more readable tests. We’ll discuss these today, as well as more benefits of readable tests. Thank you, Patreon supporters, for your continued support. And thank you, PyCharm, for sponsoring this episode.
00:00:51 Welcome to Testing Code, Python Testing for Software Engineers.
00:00:58 I am excited today on test and code. Have David settle on and we’re going to talk about readable tests, but let’s find out who you are first. So, David, who are you?
00:01:07 Hi, Brian. Well, I’m a Django developer. I’ve been working pretty much exclusively with Django for maybe ten years. And the sites that I tend to work on are quite sort of business logic heavy. So I currently work in the energy sector. The monolith I work on has in terms of numbers of Python modules in the thousands. So it’s sort of big complex back ends. And as a result, I’ve sort of got a side obsession of how to organize code so that you don’t get into a terrible mess. I sort of give talks and write blog entries. Got a couple of open source packages out there to try and help people with those problems.
00:01:54 Okay. So that’s how to organize. So if you’ve got thousands of modules, that’s a big project. Yeah.
00:01:59 It is pretty complicated.
00:02:02 That’s all under a Django project.
00:02:05 That’s a Django project.
00:02:06 Yeah.
00:02:06 Okay.
00:02:07 So that would be interesting in and of itself.
00:02:09 How to organize it.
00:02:10 Well, having me back again.
00:02:11 Yeah.
00:02:12 Have to have you back again. Yeah. That’s good. So how do we get from there to readable tests? Do you do a lot of testing on this?
00:02:19 Yeah. I think that testing and test driven developments are really big part of code design. And I guess the idea of what I want to talk about today is about test readability.
00:02:32 And this is just sort of a side thing that I’ve noticed that I haven’t really heard many people point out. And I gave a talk on it at PyCon UK last year, and people seem to be saying, yeah, that shines with our experience. So essentially it’s the idea that when you go to the production code, it can be quite good quality, but you head on over to the test and your heart sinks a bit because it’s all a bit difficult to read and understand and you don’t really want to go there. You’re certainly not going to go to the tests to find out how your software works. And I think that’s a shame. And so I sort of come up with a few principles to help talk about how we can make our tests more readable.
00:03:26 Oh, cool. I like lists.
00:03:29 Should we go through these principles and how they help?
00:03:32 Yeah, I guess so. I mean, it’s a bit of a grab bag really, but it’s good to throw things into a list even if they don’t really fit into a list.
00:03:39 Sure.
00:03:40 Well, before we get into it, I just want to, I guess, say that I do agree with you. It’s a shame.
00:03:47 I don’t really buy the notion that some people say the benefits of tests and testing with development is that your tests become living documentation. That would be sad if that was your only living documentation, because there’s a lot of stuff you want to do with tests that are possibly not the best way to learn how to use a piece of software. But if we can make tests more like documentation for how to use the API, that’s a cool thing.
00:04:16 Yeah, I totally agree. I think there’s kind of three main benefits to test being readable because obviously tests bring you other things even if they’re not readable, like tells you if there’s been a regression or something. But I think that you can sort of think of it as the sort of lifecycle of development. So the first thing that they bring is when you’re trying to kind of understand what to build in the first place.
00:04:41 So I don’t know if you’ve ever been in a situation where someone sort of said, okay, we’ve got this feature, but it hasn’t really been that clear.
00:04:50 When it gets down to it, what you’re meant to be building sort of a bit vaguely specified. I guess that’s something I would say is the case for almost every feature I build, there’s a sort of discovery process where you’re like, yeah, we’re trying to solve this problem. Maybe this is the solution, but you haven’t kind of worked out all the details of exactly what the solution is. So something that I found works quite well is to write a test first that clarifies exactly what you’re trying to achieve and then even review that before you go into implementing it because you might have misunderstood what you’re trying to build. So does that make sense as a sort of first reason it kind of clarifies the scope?
00:05:37 Yeah, definitely.
00:05:38 You can bring in multiple team members to try to understand, to make sure that you’re really looking at the right scope as well.
00:05:45 Exactly.
00:05:46 Because I think once you start to drill down into scope, it can open new questions and you can say, well, do we need to do it like this?
00:05:55 That’s the first reason. I think also, as you say, there’s the living documentation side of things. Like if tests are something that you can go to that actually explain how things work, that’s really helpful. And some of the tests that I’ve written, I do go to and look at to remind me, like in a year’s time or whatever. And I really appreciate those tests because I can rely on them actually being a real reflection of reality rather than some sort of outdated Doc string or something. And then the third reason is that they help you change your code. If you’ve got a nice readable test, then you can go to it and you can make a little tweak that embodies the change to the feature or the change in behavior. And it’s really obvious what you’re doing.
00:06:45 If that test is well factored, then it’s easier to edit and change in a meaningful way. Yeah.
00:06:50 And if you’re changing behavior, hopefully you don’t have to change tons of tests.
00:06:53 Exactly. Yeah. Agreed.
00:06:55 Yeah. I was just thinking, one of the times when I use tests to help me understand the code and understand the behavior and the intent is often with things like an open source project. And a lot of open source projects require the tests to be part of a pull request.
00:07:15 Looking at how somebody added tests to a change can tell you a lot about which corner cases they’ve already thought about and stuff like that.
00:07:25 Yeah, absolutely.
00:07:26 That’s an interesting one. I’ve not really done that, but I think you’re right.
00:07:32 Thank you, PyCharm for sponsoring this episode. Pycharm 2021 is out and oh, what a treat. Get integration was already amazing, but now it’s even better. You can do interactive rebasing. That’s cool. You can choose to have the commit window appear as a tool window next to your code instead of a pop up. I really like that branch searching is now easier. You can even install Get right from PyCharm. Heck, you can now even install Python right from PyCharm. There’s improvements to virtual environment support and improved support for adding packages to requirements that text files. If you use those. There’s even a new Charm command line tool that opens a light edit version for quick edit jobs that don’t require all the bells and whistles. You can split the terminal window now to see the output of two commands at once. You can configure the status bar. There’s improved database support with SQL script run configurations. They’ve even improved the font with the JetBrains Monotype face. Now that’s attention to detail, you got to check this out and you may as well start with the Pro version. It’s free for four months only if you go to testandcode.com piechar savetime use PyCharm.
00:08:36 So if those are the reasons, here comes the list. So it’s six principles, I think probably rather than rattling them off, we could just talk through them one by one.
00:08:46 Yeah, sounds good.
00:08:47 So the first principle, I feel like this one is preaching to converted. Probably for people that listen to this podcast, it’s profit from the work of others. So this is really just about using third party tools. And the classic example is Pi test which sometimes I’ve had colleagues who weren’t familiar with pytest sort of asked me, why do we use Pyte? What actually is it? And my best answer is, apart from many other things, like its biggest value add is that it removes loads of the boilerplate and it allows you to write more readable code.
00:09:24 I get the impression from listening to other episodes that pytest is something that everyone likes and uses. So maybe I don’t need to talk much more about that.
00:09:35 Yeah, but I like the benefit of pytest allowing more readable tests. There’s definitely people that would argue. I would not argue against that, but I guess the only downside is you have to make sure that people on the team know where to find the fixtures. If you’re using fixtures.
00:09:51 I would totally agree. And actually this is a matter of opinion, but I find fixtures. I’m not sure about fixtures.
00:09:59 I don’t like the way in which you can’t tell where these things have come from. I’d rather import it personally, but there are some amazing things about fixtures. But I guess what I’m talking about is more like being able to do natural assertions rather than having to do self assert equals and things like that. I think that’s where it really shines in terms of readability.
00:10:21 Yeah, definitely.
00:10:23 Agreed.
00:10:23 And a couple of other ones. Web Test I don’t know if you’ve featured this on the podcast before, but it’s a really nice.
00:10:30 I don’t think so.
00:10:31 It’s great.
00:10:33 It’s a Python API around interacting with web applications, and it’s just got a lovely simple API. Like, for example, you make an Http request and you get the response object and then if there’s a form in that response object, then you can just do response form and then you can fill out the form as if it was a dictionary and then just do response dot form, dot submit.
00:10:58 There’s just lots of lovely stuff like that in Web Test. That’s worth a look. And Factory Boy is another great one for creating state. I think.
00:11:08 So. That’s the first principle. Just use stuff that other people have done profit from the work of others.
00:11:13 Yeah. So slight divergence. Looks like Web Test was part of the pilot’s project, but I assume since you’re bringing it up that you can use it for really just anything.
00:11:22 There’s Django Web Test, which is, I think it’s a plug in to Web Test somehow. So that’s specifically the flavor we use.
00:11:30 Okay. Oh, yeah. See, that too neat. All right. So that’s principle one.
00:11:34 Principle two is put naming to work. So we know that naming is really important in code factoring here’s just a couple of nice tips. So when you name your test functions, how do you name them? They’re never being called by anything, so it’s tempting, I think, just to sort of call it anything. I’ll give an example. Let’s say there’s a test called Test number available. That’s the kind of sort of typical test that I think you see a lot that I’m tempted to write as well. It doesn’t really say very much. And what you can do is craft it as a more of a Bose sentence.
00:12:14 So you could say instead test add to cart reduces number available. So you kind of craft it almost like it’s the comment, but it’s the name of the function. That’s the pattern I’ve seen some people use.
00:12:28 I love it.
00:12:29 It’s really simple. It’s no extra code. Another thing is how you name your variables. So for example, let’s say we need to get some dates to do some tests on these dates. So if I were to name three dates and they’re all different dates and I call them date one, date three, what are they? I don’t know how to interpret this data. Can they just be any date? Is there a relationship between the dates? Is there some significance if say, one of the dates of the first of the month or in elite year? So if you just call in date one, date two, and date three, it doesn’t say anything unless you put some comments around or something. But what you can do is name your variables so that it communicates the intent of what that data means. So, for example, if it just needs to be a date, it doesn’t matter what date it is. I’ll often just call it some date or arbitrary date or something, and then the reader knows it doesn’t matter. And then that could be date one, for example. And then maybe the other two dates are earlier date and later date, and suddenly all the meaning comes out. There’s no extra work, but it’s so much more expressive code.
00:13:48 Then people can visually inspect your assertions. So if your assertion is like assert that earlier date is less than date or some date or something, then it’s like obvious. Oh, yeah, of course that should be true.
00:14:02 Yeah, exactly.
00:14:04 So that’s naming. I mean, there’s loads of you can do with that, but just making sure we do it in tests as well. I think tests suffer from something that production code doesn’t suffer from, which is like, what the hell does this sample data mean?
00:14:19 It’s often really not obvious when you come back to it later. Yeah.
00:14:23 And I suffer from using something like date one or something because I put the intent that it doesn’t matter. But that’s a great one. Just to use like some or any date or some day.
00:14:38 That’s nice.
00:14:39 One of the things that I was doing this morning was renaming my test functions to reflect what they’re actually testing. When I’m starting exploring a feature, writing a test around it. The intent of the test that I started might not be the same as the test that I end up with. And so it’s one of the last steps I want to make sure I do.
00:15:02 And that’s a good time for variable names, too, is to go through and read the test and what it’s really testing and make sure that the test name is reflective of what we’re really testing.
00:15:12 Yeah.
00:15:12 I had a parameter range test that was supposed to ended up I wanted to just check the boundaries of something, and while I was down that rabbit hole realized that two or three of the parameters interact with each other. And I wanted to explore all the maximums to make sure that the different sets that could go in that hit the maximum memory value could be done.
00:15:40 At that point, I still needed to write the original test somewhere else. But that test now might kind of transformed into something else.
00:15:49 Validating test names before you merge it in and have other people review. It’s a good thing to check for.
00:15:55 Yeah, definitely.
00:15:56 And it’s even more important than function names and variable names are important in all code. But in tests, the time when people are staring at it is when something’s broken and they’re trying to figure out what it’s supposed to do. So helping people with that stressful situation is worthwhile putting a little bit more effort.
00:16:17 But there’s something about tests which is different from production code in that production code kind of links with the outside world. So in theory, you should be able to follow the chain somehow of like, in what way is this useful? Who’s using it? Like, what are people’s expectations of it? Well, the test sort of exists in and of itself, so you can’t actually get any clues apart from the test. Do you know what I mean?
00:16:42 Yeah. And with tests, you also have intentional magic values. You’re picking some data points. So that doesn’t happen as much in production code.
00:16:52 Yeah, exactly. So that’s why we need more clues.
00:16:55 So what’s our third principle?
00:16:56 So the third principle is show only what matters. This is probably my favorite principle. There are a few different examples of this, but essentially the idea is, I think when you read a test, there’s a lot of kind of clutter that obscures the intention of the test.
00:17:15 For example, let’s say you’ve got a test that’s testing that when you complete an order on a shopping cart, that the cart is emptied.
00:17:25 Like a classic test that I see and I have written a lot of myself as well is like, you’ve got a few lines where you’re setting up the state for that shopping cart. Maybe you’re creating like a product that can be added to the shopping cart and you’re saying what the price is and you’re creating a customer as well. And then you remember we’re testing what happens when you complete the order. So then maybe there’s a couple of steps to completing the order, so you sort of call those functions and then finally you want to check that the cart’s empty. Maybe that involves a couple more statements. Who knows. And what I find is that you read through these and you’re not quite sure what codes under test, what’s important, it’s just confusing. So what I started doing is I try to hide everything except the thing I actually care about.
00:18:17 You can do that in a really simple way just by breaking it out into private functions. For example, you could rather than have all this set up stuff. If you have got specific set up for this specific test, you don’t want to kind of generalize it and go to all the work of that, just pull it out into a create full cart, private function that’s in the same module and then you’ve just got one line which is cart equals create full cart and that’s all you need to know and then you do it for the other bits as well. So you end up with almost just kind of like English, that’s just describing what the important bits of the test and it hides everything else. I find this really helps. Does that make sense?
00:19:04 Yes, I personally push most of that set up into a fixture if possible.
00:19:08 Yeah, absolutely. That’s definitely a way to do that. Would you put your fixture just module level or would you have like some if it’s intended for just for that test function.
00:19:19 It can just stay right in that module then of course.
00:19:23 So, yeah, that’s absolutely a way to achieve this.
00:19:26 One of the scopes that I like in that case is a class scope. So I don’t use test classes a lot. But if it helps the readability of a test to have a fixture just for that test, grouping them into a class makes it obvious that they go together.
00:19:43 Absolutely, yeah, I do that lots as well. I think that’s the main reason for using classes actually. Otherwise it can be a bit confusing if you’re testing lots of things in a module.
00:19:51 Yeah. In the thing that you’re testing also, I don’t know if I’m jumping ahead or not, but making sure that there’s not too many asserts that are certing on things that don’t matter to what you’re.
00:20:00 Absolutely, yeah. Can I just say one other thing about showing you what matters? So I think it’s important to realize to not go too far. So it’s kind of like a balance. So I’ll give an example, let’s say a similar sort of ecommerce example, but let’s say we’ve got a test which creates a full cart and then it applies a voucher to the cart and then it asserts that the total of the cart is £11.63.
00:20:30 Is it okay if I talk in British currency?
00:20:34 Yes.
00:20:34 Eleven pound, 63. That’s a bit of a weird. Where did that number come from? And this is an example of where you’re hiding too much. Like if you’ve got a test which is trying to test about discounts applying the right percentages, say then what you want to do is make sure that what you could do is you could pull up in your fixture or private function, whatever the ability to specify what the total amount of the cart is as an argument and also what the discount percentage of the voucher is.
00:21:04 So then in your test you’ve hidden all the other set up stuff, but you keep the stuff that’s relevant to the assertion you’re making at the top level.
00:21:14 So then you can see these are the two numbers which go towards making the number that I’m asserting on. Does that makes sense?
00:21:21 Yeah, definitely.
00:21:22 If you want to go even further than with math stuff, I quite like to show my working so that you really know, you could sit there with a calculator and go, okay, that’s the number that’s that number that times that. Okay. And then that gives me the unrounded number. Then this is the moment where rounding happens, and that’s why I get that final number. I always try and show my working when doing mathematical tests.
00:21:48 That’s good. Definitely. And no tricks either. It should be just exactly the same basic math you could do with a calculator.
00:21:56 Out of interest, I haven’t really decided on this one, but do you ever see tests where they do the maths in the assertion? So it’s like assert input times output equals expected results?
00:22:09 Oh yeah. And I’m on the fence as to whether that’s a good thing or not.
00:22:12 Yeah, me too.
00:22:13 Well, let’s stay on the fence.
00:22:15 However, I do like that’s a place where being very verbose, I think is fine, and being able to name the variables of each individual math component so that when I can turn on show locals with the assertion if there’s a failure and I can see actually all the steps with all the values, the intermediate values, it’s one way to easily check that if things are going wrong, if the test fails, some part of this went wrong, and if there’s a bunch of math in there, you have to go through and figure out which part it is anyway. So kind of predoing that work is sometimes helpful.
00:22:53 Yeah, absolutely. That makes sense. I think you moved on to making assertions about you’re asking about assertions of complex data.
00:23:01 You were saying show what matters, but I guess I was going to assert on only what matters.
00:23:07 Yeah. So I think you’re absolutely right. A problem that I’ve run into as a Django developer. Like a lot of your tests, you have quite complex objects, so I’ll assume that people don’t know Django. Really. So a Django model is kind of an object that maps onto a table in a database, and you might have foreign keys between the tables and the database which will be in the Python object will be like attributes on the Python object. So you can end up with all these kind of separated lookups on a Python model and you want to check that some of the attributes on some of the things in this object graph have certain values. And I find that what happens is that you end up making loads and loads of different assertions about this.
00:24:03 As an example, let’s say you’ve got an order which has multiple lines on the order, and then each line might point to a product. Say you might end up making seven, eight different assertions where you like, check the order total, you check the total of each line, check the number of lines on the order, you check that the product on each line is same as the expected product.
00:24:29 I find that stuff quite difficult to read. And also you only know about the assertion that fails. But really, you’re making a single assertion about data.
00:24:40 You know what I mean? So one technique that I found really helps with readability of this is that you come up with a value object. So when I say a value object, I mean something that if all its values are the same, then it is the same. So for example, a date is a value object. If the month and the year and the day are the same, then it’s the same date. And what you do is you make some value objects like this, or you design a value object that’s like a simplified version of your more complex production data, and you transform your production data into the simplified value object. And then you compare that transformed object with a value object that you just instantiate in the test class.
00:25:31 So it’s a bit difficult to explain.
00:25:33 But you end up having instead of comparing like 15 different properties, you can compare two valuable objects that each have 15 properties.
00:25:44 Exactly, yeah.
00:25:45 And then the assertion for quality will hopefully show you exactly all of the differences. Not just the first one that’s different.
00:25:53 Exactly, yeah. If it’s different, then you can go into the debugger or whatever, and then you can navigate just the data that you’re interested in on the value object instead of having to do some database lookups or whatever it is in your complex object graph. And actually this sounds complicated, but it’s actually really easy to do in Python because you’ve got data classes. So all you do is you knock together a few data classes for that test and then transform them and it makes the test so much more readable.
00:26:26 Yeah, I love data classes for this, because even if you just slurp in all the data for something, then you can explicitly state which ones are not part of the equality operation.
00:26:39 Yeah, exactly.
00:26:40 That’s handy.
00:26:41 Yeah, I recommend everyone gives that a try at least once. It may not go back.
00:26:46 Which principle was that part of?
00:26:47 Well, it sort of crowbard into show only what matters. Okay, but it is the last one of those. So we’re onto principle four now, which is. Don’t repeat yourself. The most controversial of the principles I don’t know what your reaction is to hearing me say that you shouldn’t repeat yourself in test.
00:27:06 It’s a different rule than in production code. I’ll just say that.
00:27:10 Yeah, well, I’m interested to know what you think.
00:27:14 I’ve heard a lot of people say to me if I’m pairing with someone on a test, say, I might point out this is very repetitive code. Can we kind of refactor this? So it’s drier. People often say to me, well, I don’t think you should do that in tests and I think each test should spell things out individually. And when I gave this talk at PyCon UK, I asked for a show of hands. Lots of people put their hands up and were like, yeah, I agree, you should repeat yourself in tests. So it’s definitely a thing that a lot of people think that we should discard that principle when we’re testing. Now this is my argument for why we shouldn’t. So I think there are two reasons for not repeating yourself in production code and the first reason is to have a single source of truth. So for example, if you’re like calculating tax, say you want one place in your code where the tax rate is defined. If it’s defined in loads of different places, then that’s not very maintainable. And if you change it in one place, then maybe you forget somewhere and the system will lack integrity. But sometimes in order to do that, you have to sacrifice readability.
00:28:29 You have to sacrifice code being intuitive to read because you’ve had to introduce some kind of indirection or abstraction. So that’s a big reason for doing it in production code and that doesn’t apply to tests. You shouldn’t be trying to have a single source of truth in tests. But there’s another reason for writing dry code which is more readable and that you wouldn’t be surprised to know is something I do think there is a reason why we should be writing dry code and tests. So for example, if you’ve got five or six different tests which are all basically doing the same thing, but just with a couple of different tweaks of different things in each one, I find it really hard to go through and even spot what the difference is between each of them. And I think it’s much better to refactor that.
00:29:17 Yeah, I definitely agree with this. With that aspect, it’s really annoying to have not even just a couple of tests that are almost identical. But I’ve seen it where there’s like, wait, there’s tons of tests that are almost identical and it is a mental effort to figure out what in there. Well, that kind of goes back to show what’s important. The lines of code in the test should all be important. You shouldn’t be doing things that just don’t matter or just irrelevant to what you’re testing. I’ve seen people abuse it when there’s API calls to something to set it up, and every test is having an error check, say check the error logs or something like that, and then they’ll replace that one line with another function that contains that one line.
00:30:09 But I mean, if all the tests all call that same function that helper function, and that helper function only has one line of code in it, you haven’t really saved anything.
00:30:19 Now I have three lines of code instead of one.
00:30:23 It didn’t really help anything. So there are times where using helper functions and hiding so hiding information makes it harder to read. So don’t repeat yourself as great if it adds readability, your tests should still tell a story. You should still be able to tell what’s going on.
00:30:42 Yeah, I totally agree. Don’t repeat yourself. Doesn’t equal readability, it’s just that it’s a tool that’s really important in achieving readability. Sometimes I don’t think we should be afraid of using it.
00:30:53 And also, like you said, it helps highlight the show what’s only what matters, show what’s important, don’t repeat yourself. Might help.
00:31:01 I totally agree. It’s all the standard ways that we’re used to for not repeating ourselves. I sometimes even go object oriented with tests. I’m not sure everyone would agree with this, but you can do that. For example in PyCharm, not PyCharm. You can do that. For example in Pi test by having a kind of a base test class which doesn’t begin with the word test, and then that won’t be run. You can have a whole suite of tests on it and then have certain attributes on the base test class, and then you can extend that base test class with sort of concrete test suites which sort of populate the attributes in such a way where the tests run. Now that could be a complete unmaintainable mess, but I have used it in ways which have made it much easier and kind of more declarative style of writing tests.
00:31:53 What do you think about that?
00:31:55 I think it’s brilliant and I don’t know if I could discover this from you or somebody else, but this idea of having a base class that has tests in it that don’t get run because the test classes to name something different. I think this is a brilliant use of inheritance within tests. You have to be careful because test classes and test objects are different kinds of things than other objects. I mean, the implementation is the same, but it’s hard to think about them because test frameworks often use classes just as sort of containers for tests, even though I mean they are instantiated, but they’re instantiated way more than you realize.
00:32:43 Yeah.
00:32:44 So if you’ve got a class that has a bunch of tests in it, it’s going to get instantiated one for each test.
00:32:52 Yeah. I feel like you want to use your test classes as name spaces. Really?
00:32:56 Yes.
00:32:57 More than anything else, that’s what they’re useful for.
00:33:00 If you’re setting loads of attributes on self, then I wouldn’t say it’s necessarily wrong.
00:33:07 But it’s a bit of a smell and the behavior is going to surprise you.
00:33:11 Yeah, absolutely.
00:33:13 Well, we mentioned it before, but parameterization in pytest is another good way of not repeating yourself because you’re only right one test function, but you give in loads of different inputs to it.
00:33:24 Yeah. Okay.
00:33:26 Parameterization is brilliant. And I guess that’s one of the things we should hop back to. Good naming that complex. Parameterizations often have silly dumb names. So spending a little bit of time writing an ID function or writing Identifiers for parameterization is helpful for watching the test run so that your parameterizations are meaningful.
00:33:48 Yeah, absolutely.
00:33:51 It’s annoying when you see that, like one failed true false, one false, none. Great.
00:34:00 Yeah. Or if you pass in a list of lists, like if you parameterize a fixture and you just get like that just names the objects that come in.
00:34:12 Oh, yeah.
00:34:13 So less than helpful.
00:34:14 Yeah. It’s a bit of a danger error Parameterization, because I actually find I end up just putting comments by each case above it. But probably I should do what you said.
00:34:26 Not as hard as it sounds.
00:34:28 It’s more about just the way in which it ends up getting formatted and can be hard to read, I guess. Yeah.
00:34:35 One of the tricks I often do is to just have one of the parameters, like if I’m doing a multi parameter object or something, have one of the elements be something that’s identifiable or even a comment just as a string value, and then have the ID function pull that element out.
00:34:52 All right. And maybe not even use it in the test.
00:34:56 Yeah, it’s just for identifying.
00:34:58 I like that. I’m going to use that. Great.
00:35:02 Anyway, that’s the don’t repeat yourself principle. Okay. Are you ready for principle five?
00:35:06 Yeah, we only got two more. Let’s go.
00:35:08 Yeah. Don’t worry if your head hurting like we’re nearly done. So principle five is quite a well known principle, actually, but it’s a good principle which is arranged after Cert. This is just a way of structuring tests. It’s also sometimes called given. When then. And it’s just a sort of pattern, particularly for unit tests, I’d say for sort of lower level tests where you’re running lots of fast ones. So a test has kind of three structures, or it should do, it arranges, it sets all the data up, then it acts, it does the thing that we’re testing and then it asserts. And a really simple way of making your tests easier to read, first of all, conform to that structure, but also just put like a blank line between the three different sections and it just immediately signals what bit of the test is actually exercising the purpose of each bit of the test. It’s just a really easy thing to do.
00:36:10 Yeah. I even sometimes either stick arrange active Cert or Given when then as comment words. Yeah, I’ve seen that it can be annoying if it’s a small test, but for a larger test, it’s worth it to me to split that up. To say, yeah, this is the part that I’m intending to be testing. So that if an assertion happens in the arrange part, it tells you different information.
00:36:37 Yeah.
00:36:37 Partly means the test name isn’t going to tell you much about why it’s failing.
00:36:42 Of course, with some tests you might want to diverge from that principle a bit. Like if it’s a sort of bigger end to end test that uses a database or something and it’s quite expensive, you might more think of it as telling a story or going on a journey through the data. So maybe you’d have an arranged stage and then you act, then you assert.
00:37:06 I think that’s okay for a certain kind of test, but still it’s good to signal that’s what’s going on.
00:37:12 Yeah. Even at higher level tests, it’s fun to try to figure out how. If it’s not natural to put it in this state, how could I? There’s been many times where the real answer is to split the test up into multiple tests. So if I’ve got really five different things that I’m checking or even three things that I’m really checking, maybe I can push the arrange part into a fixed year to get run at a module level or a class level and then have the tests be just different tests that test different things. So do the complicated, expensive work and then arrange a certain active move those into their own tests. Because I’m really testing different aspects. Sometimes it’s more work than it’s worth, sometimes the entire having a nice story. But having most of your tests be these story type tests that have a bunch of act and asserts in them is a smell and is something that you should try to not have all of your tests be like that.
00:38:16 Absolutely. I mean, I feel like I increasingly think about tests in terms of like signal and people talk about fast test, slow tests. I actually find it more relevant for me anyway, is what kind of a signal do I want to get from the test? Do I want something really detailed that says this is your bug or do I want something that gives me a lot more confidence about the wider functioning of the system? I’ve heard people refer to them as high gear, low gear tests. I think that’s really nice.
00:38:46 Are you like climbing a Hill in first gear in your car or are you driving down the motorway really fast? Not very much control.
00:38:56 Maybe if the test fails you won’t get very much information. And I think the way to think about it is if there are some failures in your test suite, you should always be going to the unit tests first because they’ll give you better information.
00:39:08 And that’s how you should think about unit tests is like they tell you important stuff, but they’re a bit less reliable. Yeah.
00:39:15 I like the high gear, low gear thing. And also, if you are in a hurry, you can do the high gear type test for most of your system and do the low gear stuff for the problematic stuff.
00:39:26 Absolutely. Yeah.
00:39:27 You’ll find out where the problem owners are. If you’re having to debug a test a lot, maybe split it up.
00:39:34 I totally agree with that. Yeah, of course. Also, high gear tests allow you to refactor better, usually because they’re more distant from the implementation.
00:39:45 It speeds up that as well. So, yeah, that’s a range at a certain one. The final principle is a very simple principle which is aim high. So I guess main argument is that we tend to treat tests like a second class citizen. And my first tip is simply treat tests like a first class citizen. It’s just an attitude change. No more than that. So for example, I think how do we make code goods? How do we improve the readability of code? We do thorough peer review. Now, are we doing as thorough peer review on our test code as our production code? I don’t think so. I mean, I speak personally, I find it harder to review tests and I do to review the code that’s running on production, I have to force myself sometimes to really give it attention and to say, well, why don’t we maybe we should change the test like this to people a bit like, but this is the test. Well, who cares? And also refactoring tests. Going back to tests later and changing the way they’re organized is another way to improve their readability. But it’s not something we may be in the habit of doing.
00:40:54 Yeah. I try to tell people on my team to review tests actually code also, but review tests as if everybody else is on vacation. You’re the only one working and the CI system shows a failure in this test and you need to figure out what’s wrong.
00:41:11 Nice.
00:41:11 Yeah, that’s really what we want. I want a test failure for anybody on the team to be able to jump in. And maybe you do have to do some. Maybe you have to. I think it’s fine if you have to rerun the tests or look at the local variables or debug it a little bit, but it shouldn’t be that hard to figure out what’s wrong from a test failure.
00:41:30 Yeah, I like that. I really like that. There’s another way you can sort of hack yourself.
00:41:35 Basically. I think reviewing tests first is a good idea. And I don’t know if you’ve noticed, but on a pull request in GitHub, tests are usually down the bottom. I mean, this might be just because they’re in alphabetical order or something, but they’ve always been down the bottom in the projects I’ve worked on.
00:41:55 So I end up reading at the end. And I think that reading them at the beginning, like scrolling down to the bottom and looking for the test first is a good way to get yourself to engage with whether or not a test actually explains what’s going on. And then you go and you look at the implementation.
00:42:11 I think that we have a good reason here to petition really the world that T should be the first letter of the Alphabet.
00:42:20 Definitely.
00:42:21 There is actually a hack you can do, which is I’m a real fan of breaking up pull requests into smaller, really small commits and then encouraging people who are reviewing my code to work their way down the commit list rather than just look at all the changes in a pull request. So what you can do is you can start your pull request with the tests failing tests, Mark them as X, fail using pytest, Mark XFL, and then you’re like, okay, this is the first thing you need to look at. It’s just the test. Maybe you can stub out the functions, but you leave the functions empty and then people are actually encouraged to think about. Right. Does this scope out what you’re trying to achieve? So it kind of becomes a sort of introduction to the pull request and then you can build up the implementation and eventually that final implementation slots in and the reviewer can also see the Xfail got removed. Great. The test pass. Job done.
00:43:17 Nice. Yeah.
00:43:19 Like it.
00:43:20 So those are my six principles for Readable tests. Hope you enjoyed them.
00:43:24 I did enjoy them. I’m the kind of person that enjoys talking about testing. So it’s good.
00:43:29 You should start a podcast.
00:43:32 Yeah.
00:43:33 And I can have on people that also think about tests a lot like you. That would be good.
00:43:39 Now this has been really fun and I think it would be great to I haven’t watched your talk, but presentation, but the Python UK talk, we will link to it in the Show Notes.
00:43:50 Yeah.
00:43:51 I’ve basically just read out the slides of that talk. So I recommend if you sort of didn’t understand anything I said or if any of the readers. There’s lots of code examples in the talk, so you should be able to follow along and hopefully make a bit more sense.
00:44:07 If you disagree with one or more of these principles, I think it would be a good idea for you to come on the show and talk about why you disagree with it, because that would be interesting.
00:44:18 Well, thanks so much for joining us today. This is fun.
00:44:21 Thanks for having me. It’s great.
00:44:25 Thank you, David, for the principles and also reminding us to take care to make our tests readable. Thank you, Patreon supporters join them by going to Testincode.com Support. Thank you, Pie Charm for sponsoring this episode. The link to the extended profile is at Testing Code.com Pie Charm. That link is also in our Show Notes at Testing Code.com one, one, two. That’s all for now. Now. Go out and test something. But make sure your test is readable.