A discussion about mocking in Python with the original contributor of unittest.mock, Michael Foord.
Transcript for episode 145 of the Test & Code Podcast
This transcript starts as an auto generated transcript.
PRs welcome if you want to help fix any errors.
00:00:00 The mock library that comes with Python as unit test. Mock started with Michael Ford. In this episode, Michael and I talk about mocking, of course, but also testing philosophy, unit testing, and what a unit is in TDD and even where Michael Howell is and what color. Michael was instrumental in the building of testing tools for Python, and he’s got a lot of great advice about testing. I hope you enjoyed the episode.
00:00:39 Welcome to Test and Code.
00:00:48 Michael Ford. Did I get the last name right?
00:00:51 You got it right.
00:00:51 Yeah.
00:00:52 It’s the English version or the British version?
00:00:56 Yeah, that’s right. I’m AOO.
00:00:59 Is it just Americans that have trouble with that?
00:01:06 The thing is, the single o is so much more common. People sort of assume it’s Dutch for some reason, it’s just English. Back in the days when we used to have telephone books, there was always a few in every telephone book. So not common, not rare.
00:01:19 Whenever I see your last name, I don’t know if it’s spelled the same, but I think of Ford Prefect. But did you spell it with one o or two?
00:01:27 He spelled it with one o’there’s. Also the Hoopie fruit in the work of Douglas Adams. And I love it. I’ve always felt sort of an affinity with those works, both because of Ford Prefect and because I am a Hoopie fruit. Who knows where his towel is?
00:01:41 Definitely.
00:01:42 So do you have your towel with you?
00:01:44 I don’t have it with me, but I know where it is.
00:01:47 Yeah, it’s perfect.
00:01:48 It’s upstairs, one of these microfiber ones so you can pack. It really small for traveling. It’s great.
00:01:55 Perfect.
00:01:57 One of the things. Okay, so we want to talk about marking eventually, but what are you doing now? Do you have, like, a consulting company or something? Right.
00:02:07 Yeah. Contracting and training.
00:02:09 Training.
00:02:11 I started training Python, doing Python training about ten years ago now with David Beasley teaching his Practical Python and Advanced Python Mastery courses alongside my regular work. I just do it a couple of times a year, and I really love doing it and enjoying it. And it’s a good trade to be in.
00:02:32 And then I worked for my recent career history. My last job was with Red Hat on Ansible Tower, their enterprise Web application for managing Ansible and for managing computer infrastructure with Ansible. And I worked on the test automation team there, helping them build out a test system. I worked for them for a year, and the fact I really wanted to go self employed, it was time to do that.
00:03:01 So I’ve done half contracting and half training, and I really enjoy that. I still teach David BS courses. I’ve also taught some Flask and some testing and object oriented theory for the Royal Ordnance Survey. Object oriented theory with Python. They didn’t just want a Python course. They wanted something a bit more fancier because they’re the Royal Ordnance Survey. So I did object oriented theory with Python. That was great fun contracting. My most interesting project has been for the United Kingdom Atomic Energy Authority, and that was from January to June last year, and I was working on software for designing fusion reactors.
00:03:41 Oh, wow.
00:03:45 It’s like a dream job. He was actually a PhD guy and it was his PhD project, so he’d already committed to make it open source before he sort of accepted the job with the UK. So they are going to make it all open source. It’s blueprint. And I worked with him on turning it into an engineering project, and at one point, he is phenomenally clever guy. I mean, I was literally actually removing descriptors off metaclass.
00:04:11 No, removing metaclasses off descriptors, which does not need to exist, but still, you have to be really clever to make as much of a mess as that.
00:04:23 That was great fun. Yeah. So the normal way that the fusion reactor design is done, the academic world, they’re all silo, everyone has their field of expertise, but there’s a whole series of things that need to be done. You know, they want to work out the magnetic containment, the plasma containment Chamber, so they’ve got place, they’ve got overall size, power requirements, positioning of the magnets. The really cool thing they’re doing now is breeder blankets, basically fusion reactors. You put hydrogen in, you ignite it with a laser, enough heat and pressure to trigger fusion, and then using magnets, you shape and contain the plasma, usually in a Taurus.
00:05:04 And then this produces a stream of neutrons, which hit big metal absorber blocks around the side of the lining.
00:05:14 The reactor, turns it into heat, and from there, it’s normal turbine technology, which is very well understood. What they’ve done is they found that if they put lithium into these blankets, that the neutron stream bombarding the lithium every now and then will hit the lithium, split into two tritium, atom split into tritium, which is the fuel for the reactor. So you need enough tritium, which is very expensive to produce and radioactive, pretty dangerous. An isotope of hydrogen. And once you’ve got ignition, it’s then selffeeding in terms of the Trojan. So that’s a brilliant innovation.
00:05:53 Anyway, you used to design these things by sort of getting your spec, sending to the first guy to do the first bit of design. He takes a month, send you back a bunch of numbers, which you send to the next guy. So the whole process took months, so you can’t really do an iterative process of trying a bunch of things that way. And this guy, his genius, is an interdisciplinary approach, and he essentially wrote one application that does all of these different things, and it’s for initial design work. It’s not the detailed design, but you can now do an initial design with fancy OpenGL drawings, which are beautiful, and everyone thinks it’s the whole point, but of course it’s not. It’s all the numbers, it spits out. You can do that in 40 minutes, so you can do iterative. Design processes. So it is and is going to revolutionize fusion reactor design. It was a fantastic opportunity to work on it and so much fun.
00:06:45 That’s so cool. Did you say that was open source stuff?
00:06:50 Well, I emailed him recently to ask him if it was open source yet and he didn’t reply to that bit.
00:06:56 They were due to do it before now, so I think the answer is no.
00:07:00 It’s called Blueprint.
00:07:03 There is a PayPal he’s released on it.
00:07:11 Yeah. I’ll be all over Twitter with it when it comes out because it’s fantastic to play with.
00:07:18 You can design fusion reactors in your living room.
00:07:24 Thank you, Pie Charm for sponsoring this episode. I first tried PyCharm when they started supporting pytest many years ago. Their support for pytest is now amazing. I was a longtime Vim user, so next I needed to test the idea Vim plugin, so all of my finger muscle memory still worked while editing. Check. It works great. There’s lots of reasons to live by term, but for me it is because they have the absolute best user interface for test automation. Then I learned many more ways by Charm can save me time. Like, really great support for editing Markdown, HTML, CSS, JavaScript, remote connections to database, and amazing version control support. Really. It’s the best Get diff tool I’ve ever used. And now version 2023 is out and the Shift shift, the Find anything key sequence even lets you search, get commit messages. What even that is so awesome. Tons of other cool features have been added in 2023. Check it out and I hope you enjoy it at testandcode.com. Pycharm, I know you’ve got a site. Agileobstractions.com. Is that the.
00:08:33 Yeah, that’s my professional site. I haven’t added any of the projects I’ve worked on since 2019 and 2020. I don’t think so. It’s a little out of date, but that’s my professional site.
00:08:43 If somebody wanted to hit you up for training or something, they could go there, right?
00:08:47 Yeah. Oh, Michael at python.org.
00:08:49 Okay.
00:08:51 A Python.org email. Yeah.
00:08:56 Instant credibility. I got it because I helped out a lot on the web. I was one of the Webmasters on various mailing lists administration.
00:09:05 I asked for Michael, and nobody else had taken it. But it’s a beautiful email address. I’m very proud of it.
00:09:09 Yeah. So there’s a Python feed thing. I can’t remember where it is. Python. Do you remember what that is?
00:09:17 Python. Is that what you mean?
00:09:19 Say that again.
00:09:20 Python feeds. Do you mean Planet Python?
00:09:22 Yeah, planet Python.
00:09:24 I just remember that because I think that’s the first time I ran across your name, because when I started blogging about Python, I heard somebody say, well, you got to get your blog on Planet Python to get listed or people won’t pay attention to it. And then so I requested it, and I think you replied and said, okay, it’s there it was.
00:09:46 One of the things I was looking after that was back in what I think of as the golden days of the Python community, back when Python was about to explode with the web revolution and Google adopting Python.
00:10:00 Prior to that, Python had mostly been used only by enthusiasts, only by people who really loved the language, which made a beautiful community full of passionate people really eager to teach you. And then Python just exploded. So back in the day, having your blog on Planet Python, you could get 1000 views for a blog entry.
00:10:23 The glory days.
00:10:24 Yeah.
00:10:26 Then I ran across your name next when I started researching testing stuff and you had a bunch of testing articles, but then also your name is attached to the Mock Library and Unit Test.
00:10:42 That’s right, yeah. I originally wrote what’s now? Unit Test. Mock maintained that as a library for quite a while, and that came out of my first program, Gig for Resolving Systems in London. It started with them back in 2006 and they were doing all Extreme programming. It was all pair programming, fully Test Driven Development, customer representative, doing prioritization estimations, tracking, velocity, all of this kind of stuff. We did that rigorously for four years and that was an amazing experience.
00:11:17 It came out of that time, and in particular, I got a passion for testing as a way of ensuring product quality in programming. Whereas before, the sort of style of programming that I was used to do, knowing that things worked was a real challenge. You have to try everything and we can automate so much of that.
00:11:40 I was passionate about Python and I became passionate about testing in that time.
00:11:46 I guess since then, do you still incorporate testing within all your development processes then?
00:11:55 Yeah, we did Test Driven Development rigorously. We did Extreme Programming rigorously for four years. So that was the first thing we do is write a functional test which exercises end to end the feature that we’re trying to add or demonstrates the bug. And then we start writing unit tests, and those are led. And we had a test to code ratio of at least three to one.
00:12:17 And we over tested in all sorts of places. We had testing couples to the implementation. So I learned a great deal about testing and the problems. If you over test, you couple your unit tests, you’re testing the implementation. So you’re testing your private methods. And then when you come to refactor and you want to change your code, change the way you call your code, your tests tell you nothing useful and they’re broken, because even when you fixed your code again, the test is not going to show you the right thing. They’re testing old code, functional tests, your end to end tests. If you’ve got good, reasonable end to end coverage, even just a set of smoke tests, then you can refactor and you can be reasonably confident that if you haven’t broken functionality, because with refactoring your end to end tests don’t change. So I think there’s a lot more value in end to end testing. I think there are dangers in over testing. I like to say unit testing is about testing to the unit of behavior, not the unit of implementation. Test through the public API if you can’t test through the public API, then your abstraction isn’t right. Typically these sorts of things help you to avoid over testing. Scripting to us, I won’t test.
00:13:27 Adding tests to big legacy projects is also very challenging, particularly in the face of ongoing feature work. And that’s something you have to sort of work to incorporate gradually and pragmatically, because businesses, you only get paid as a programmer if the business keeps going.
00:13:45 So you have to you have to incorporate building and test and testing to give you keep sanity with the work of maintaining and extending the legacy project.
00:13:59 I really like that. I’m going to steal that unit of behavior, not unit implementation. I like that a lot.
00:14:05 Because that’s what you want to test, right? You want to test. It does the right thing. You don’t want to test how it does it. Because in theory at least, it would be very much nicer if you could change the how. And your tests still tell you useful things. And if you’re only testing through the public API, as long as you’ve got the public API right, often you need to change that. Then you’re free to change the internals and your tests still tell you useful things.
00:14:31 Yeah.
00:14:32 And then, of course, as you know, there’s always cases where you really want to beat up some algorithm bit in the middle. And so there is going to be some tests around that.
00:14:46 There are bugs where you can’t replicate it without some timing issue or some file system quirk. And the easiest way of replicating is sticking some Invalid data into some internal state, because you know what triggers the bug that’s going to happen.
00:15:03 All generalizations are wrong, including this one.
00:15:08 Yeah.
00:15:08 Okay.
00:15:09 I’m definitely going to have to get this one transcripted because we got a lot of gems in here.
00:15:16 You got into Mocking and stuff is a natural part of I guess it depends on the year you were doing it, extreme programming and test driven development.
00:15:30 But Mocking is a little bit misunderstood by a lot of people, including probably myself.
00:15:36 I’m probably me.
00:15:42 And it’s partly your fault.
00:15:46 Alex game that blames me.
00:15:49 Well, when people start researching it, they get into things like, well, the article by I’m going to get it wrong called mocks art stubs typing.
00:16:05 He defines various categories and types of mock ways you can use mock, and he defines them as different objects, and it’s defining them as they are defined as categories, but they’re fairly rigid definitions. And I think by his definition, mock is all of his types of mock except a mock.
00:16:22 Okay, yeah, I was curious about that definition.
00:16:27 Okay, so in Python, we use the unit test mock library, or often wrappers around it. So like pytest, mock is a wrapper around the mock object thing so that you can use it’s. Basically context managers built around it, which is kind of cool.
00:16:46 Also really cool. You can use them as context managers. Anyway, right off the bat, I didn’t know that at first.
00:16:53 Yeah, that’s an interesting point. And that was an innovation in mock that actually it was from a guy at Resolve Systems. It wasn’t actually me, although I get the credit for the guy called Tom. I forget surname. Poor lad.
00:17:09 Good guy who discovered that patches are decorator, patch, monkey patches things. So you can inject a mock into all sorts of places. And the reason it blames me. And I think possibly what you’re about to say, I might steal your Thunder.
00:17:26 Patch makes it possible to test code that was essentially really hard to test before. So the combination of mock and patch make it possible to test code that’s really hard to test, which doesn’t give you an incentive to write code that’s easy to test, and code that’s easy to test is generally better code. So patch and mock let you disguise the fact that you’re writing terrible code. And that is definitely true.
00:17:55 But the innovative thing is that patch which puts mocks into place, the thing that it does that’s really powerful is it undoes it. It puts things back the way they were before, and monkey patching things is easy. Unmonkey patching them is hard. And there’s a lot of logic in patch that knows how to do that. So you can use it as a context manager with patch time dot time as mock time, and then anything inside the context manager sees time time as your mock object, which you can then configure the return values, but then outside the context manager, time dot time is restored to what it was normally. So the scope of the effects of the patch is limited by the context manager, which is what context managers are great for a visible scope of effect. And you can also use it as a decorator where during the function for the inside of the function that’s decorated the patch is in place there. This was actually done in Python 2.4 originally, back before we had context managers, and it was Tom who resolved the systems, who realized that the way that we do decorators is entirely compatible with context managers. So patch can do both. And patch is an interesting beast. And there’s actually in context Lib now there’s context decorator where you can write context managers that also work as decorators. And that came out of what first happened in mocking Resolver systems.
00:19:29 Bit of Python archeology.
00:19:31 Yeah. So if people have kind of missed it so far, how do you describe a mock to somebody that doesn’t even know what it is.
00:19:42 Okay. Right. So a mock object is an object that can pretend to be any other object, essentially. Now, the thing that Martin Fowler defined about mocks and what was common at the time. So he talks about mocks and stubs another one. But the essential point about a mock is that you can record your expectations and then replay those. So you create a mock object, you configure the mock object and say, how I think this is the Martin Fowler one and the Java mocking libraries and the Python mocking libraries at the time. You create a mock object, you say, I want you to have these methods and I expect this method to be called. It should return this. Then I expect this method to be called.
00:20:28 And then you run your code and you hit replay and it throws an exception if it was used in the wrong way.
00:20:35 Okay, so that’s the record replay style of mocking. And for me, that puts the you record your expectations on the mock, and then you call your code and that’s us about face. What I want to do is I want to inject a mock into the system, run some code, and then I want to be able to make assertions that it was used in the right way because not all of the things that happened to it might be relevant. It might just be one particular thing. I want to assert that you were called with this argument. I want to assert the mock might even just be going in there just for the purposes of returning a precar value stubbing out a system function that you don’t want called in a unit test. That’s a very good use of mocking, mocking, external dependencies to return deterministic results for the purposes of testing to avoid external calls in your unit test, that’s the classic and a great use of mocks.
00:21:26 You might mock out API calls, mock out network calls, knock out file calls. There’s some support in the mock library, particularly for files.
00:21:36 Yeah. Like for instance, I think of an example which probably isn’t very common. But if I’ve got a system on a logging system or something, and if I find something critical, it’s going to email a bunch of people during the test, I don’t want to actually email everybody, but I can make sure that the appropriate call to email the right people is called during the test with the right parameters.
00:22:02 Yeah, exactly. That’s the sort of thing that mock is. And just to finish off the thought previously. So instead of being recorded, replace style mock is AAA. What is it? Arrangeact assert you set up your code, you call act, which is arranged. Then you call your code, which is act, and then you make the asserts. So unit test mock. What it does is you can configure it so that methods will return values or method call will have a side effect raising an exception or calling another function. Or you can give a sequence of values for multiple calls, all sorts of ways. You can configure objects. They can pretend to be any object, they can pretend to be a dictionary. You can configure any of the magic methods or all of these kinds of stuff. You put it in place and then all of the calls are recorded on it, and there’s some convenience assert methods and convenient call methods. So the reason I created it was for two reasons. The first thing was that we were doing full test driven development and we were creating all of these little stub objects in scattered throughout our code base with like a mock workbook, a mock worksheet, a mock cell, a mock row, a mock column, and overall it added up to sort of hundreds, maybe thousands of lines of code. And initially I was like, I can replace all of this with a Python object with a done to get a response to every attribute look up. So the initial implementation was about 30 lines of code, and it was to replace all of these mock objects in the Resolver code base. But a requirement from Giles Thomas, who was the CTO at the time was that we had to have a way of limiting the API. So if we accessed an attribute that shouldn’t exist, it would still raise an attribute error. So that’s where all of the spec stuff in mock came from. So that was where it originally came from. It was motivated also by the desire of none of the existing mock frameworks, all of them in Python. They were all this record replace style which I didn’t like, and they’re the testing in Python community in Python was particularly close knit and fun, and we got the testing in Python boss at Pycons, which had a great run for a good number of years. It was a lovely community to be a part of, and mock really evolved very rapidly with users in the testing and Python mailing list and feedback from them and competition with other people writing libraries. I was full of passion and activity in those days. If any mock library came out with a feature I thought was good, I’d have a new version out with that feature in a few days.
00:24:31 I answered every email about mocks on the testing and Python email list, showing them how to do it with my library, I blitzed the competition.
00:24:43 Well, it’s interesting.
00:24:45 I share enthusiasm.
00:24:46 So the testing and Python mailing list still exists, has very little traffic on it. So a lot of people think maybe it’s dead, but we still get I mean, I still pay attention to it and replies, you get really good replies if you ask a question there. It’s pretty good. Now, I probably regret putting this in a podcast because suddenly people start using it.
00:25:09 Start using the testing and Python mailing list, folks.
00:25:13 Great.
00:25:14 But yeah, we’ve got some really great smart people paying attention to it.
00:25:20 Okay, so what is your relationship? If I’m thinking of this as somebody coming in and going, maybe I should use mocks in my testing.
00:25:29 What do we tell people that haven’t used it before?
00:25:32 Because there is this like you described, this test driven development model, which is, okay, there’s two huge classical Marcus, but obviously we’re talking about Mockus TDD, which means we try to test every function in isolation with everything around it.
00:25:51 Right.
00:25:52 The trouble then is you’re not testing the wiring between your parts. The great advantage of the great advantage of doing that is that your tests run nice and fast. But if you can have full coverage at the individual function and method level and still have no idea if your system works, because this sort of wiring between your components is not tested at all, and the same effort expended it just at the functional test level would give you confidence that the application actually works. The advantage of Test Driven Development and the reason it’s called test driven development. The other way of putting it is I have to talk about Test First. But the idea is that the test drives the design, and this is what I love about Test Driven development. And this is what I take from it, even if I’m not always religious about Test First these days. And it’s that step of thinking about the design.
00:26:44 If you do Test First, the first step has to be not what’s the solution? Let me bang out some code. It’s how do I call this? How ought this to work? What’s the best right API for me to be testing? And so there’s that it bakes in right at the start. The first bit of thinking is how should this look like? Because the default otherwise is you bang out some code, you think it works, I’ll add a test, and what you’re testing is whatever you happen to come up with, not the best way of doing it. And the other aspects of TDD, the simplest thing that could possibly work building up Tests incrementally is that by evolving a design like this, as long as you pay off the technical debt of doing the refactoring, incorporate the cost of refactoring into your estimates, do you actually come up with better solutions by basing your incremental approach on actual usage? So those are lovely reasons to do Test First. And as you say, if you’re doing Test First, if you’re trying to maintain some level of isolation, you’re going to need some mocks. But I think that’s the question. Then the question then becomes, what is my testing philosophy? And the specific question we’re asking is how do I get started with mocks?
00:27:56 So I think we can answer that much more simply. But we can say, look, the two things in Unit Test Mock Library are the Patch Decorator Patch Context Manager and the Mock classes the mock object. And actually, most of the time probably Patch is going to create your mocks for you. So first you need to use Patch. And this often confuses people. So we can talk a little about that, about it if you want. But basically you say with Patch and then the location, I’m going to patch out a method on a class It’s ModuleName class method as mock object. And then inside the context Manager, you can configure the value of the mock object. We probably want to say mock method. Return value equals three.
00:28:37 And then after we’ve executed our code, we simply say mock object assert called with and assert. It was called with the right parameters. So the very basics of using Patch to inject a mock and making the asserts are quite straightforward. But where should you do this? Well, an obvious place is replace file access with a mock object.
00:28:59 If you patch open the built in open, then you know it was called. If you’ve got your results and you’ll save time in your unit test, patch out caused a time with something that’s going to return you something deterministic system calls, network calls, database queries, anything where you want to return PreCan deterministic results. And you can avoid a real life network if you’re not testing your database access.
00:29:27 If you’re happy about the database access that’s covered, maybe the integration test level mock them out at the unit test level, make your test faster and less dependent on your underlying model. This kind of stuff is the place where mocking can give you a win.
00:29:41 Yeah. And these external parts of your system. Well, it’s really the system under test, and I think one of the things people don’t talk about a lot is the test architecture often mimics the people architecture.
00:29:56 Interesting. Well, I mean, like, let’s say I’m working with a database, but I’ve got a database layer that some other team is working on.
00:30:07 I think it would be.
00:30:09 And I’m not responsible for the user interface. I’m responsible for this middle layer of stuff. It’s completely reasonable, too. Then I think that there needs to be system level tests. But as a team, I’m going to probably feed my API and stub out my Dependencies or mock my Dependencies.
00:30:32 The principle is sometimes expressed as don’t test the browser, which really only applies to web application development. You test the code you own, not the code you don’t own unless you have to.
00:30:43 Right. And then the other really depends. Doesn’t matter what style of testing you’re doing, whether you’re doing a lot of test driven development or tiny unit tests or even functional tests, people are going to eventually need to mock out their external stuff like API calls to external services, like an example that I’m blanking his name. But Harry first of all brought up is the credit card processing. You’re definitely not going to hit the credit card API unless there’s a debug one, but why not just Mark that out or stub it out?
00:31:27 So the other thing that Harry brought up, which I thought was a cool idea, is any real third party system, not some other teams system within your own company, but like a third party, like a credit card processing or something that you really want to try to replace. Their API was wrapped in your own object or own module so that you have a limited set of API functions or entry points into that service. And then that’s a natural place to mock or stub out those functions to hit those.
00:32:05 Yes. And if your API system is making network calls, then you can test this abstraction by mocking out the network calls, the calls to request or whatever, or the calls to their client.
00:32:24 So you can be sure that’s work, that’s doing the right thing. And then you’ve got a nice abstraction layer where you can put your mocks and completely replace that bit of a system so you can test the two layers separately and be confident that it does work. Endtoend. I know that the system under test caused the API correct the API correctly that we’ve provided the abstraction provided, and I know that that makes the correct API calls because we’ve tested that as well. So yeah, I like that approach.
00:32:55 The other thing that I think is neat, and I wonder if this is one of the reasons why you get blamed for bad designs, is that I don’t really have to understand dependency injection to use mock.
00:33:11 Yeah, that was another motivation in Python. Dependency injection typically boils down to adding extra parameters to your function signatures, and I think function signatures are part of your API, and messing with those as a way of managing dependencies is not necessarily ideal.
00:33:29 I might have softened on that.
00:33:34 I sometimes say this, and I think it’s a useful thing to say is that every time you use patch, it’s an admission of failure.
00:33:45 Mox ought to be your last resort. It ought to be possible to test your system in the parts of your system it ought to be testable.
00:33:52 And if you have to replace a bit of it inside the live system in order to test it, then what you’re saying is I couldn’t design the system in such a way that I didn’t need to do this.
00:34:03 So working to minimize your use of mock and patch means that you’re going to get the best value as of I think only the situations where mocking is really clearly the best approach, rather than making it the tool that you turn to first, because there’s a definite I mean, I’ve written code.
00:34:23 First it does this function call, then it does that function call, then it does another function call, then it returns a result, and I’ve mucked out all my dependencies. I’ve mocked out this function call and the other function call, and that function call and now I’m testing my mock objects and not my code.
00:34:35 Right.
00:34:36 And that’s crazy.
00:34:38 You’ve really tightly coupled your test to your implementation and you’re completely testing your implementation details. That’s not a useful test, really.
00:34:48 So let’s take that a bit further. So let’s say in order to test my credit card processing part of my system, I’ve been using mocks or something with that. What’s the alternative?
00:35:00 How would I I think having cleanly, defined layers helps you have a single point.
00:35:09 I mean, the other thing I might do is do this by a conflict file, you know, or have another mechanism that loads a mock part of the machinery in place. When I’m running under a Dev conflict, I want to be able to guarantee that when I’m running my tests, I can never send a credit card request. So I don’t want to inject live things into the live system to make sure it doesn’t. I want the system I stand up to not be capable of doing it.
00:35:43 Okay.
00:35:43 So particularly for credit card or anything that’s Privacy or security critical, I would think about having machinery that does that for me. And it might put a mock API in place and maybe that would use mock objects, but you’d have a Shim layer.
00:36:00 You’D have some Shim machinery that kind of reminds me of the database stuff. So one of the things that people often do in testing is to replace the live database or a file based database with an in memory database, right?
00:36:17 Exactly. Yes.
00:36:19 But a lot of the databases like Postgres and others have a memory feature. So you can just use the live database and just have it be in memory if you want. For instance, any warning signs to give people.
00:36:41 I like the idea of look at patch first and possibly look at your external system, like you gave a list which is good, network system calls, things like that, and then the easiest Isolating those, making sure that those aren’t all over your system. I wouldn’t put like request calls in every file.
00:37:09 Have it done in a layer, which is much easier to stub out and test and have your calls go through that.
00:37:17 And here your sort of testing strategy starts to influence your design. And I think in a good way, it’s like putting the side effects into a separate function. So as much as possible, your functions are pure functions which are then really easy to test, and it’s really easy then to stub out, mock out the bits of your code with the side effects. It’s the same concept. If the file writing happens in its own function, then the rest of the function is much easier to test.
00:37:51 This kind of thing. Yeah.
00:37:52 When you’re teaching people, I see on your training site that you do teach testing training course.
00:37:59 How much of that is around mocking or is that just a small part of what you’re teaching people it’s a part.
00:38:07 It depends a little bit on the customer and what their priorities are. I maintained Unit test in the standard library for quite a while. That was my other involvement with testing. I added Test Discovery. I helped break it up into a package. Benjamin Peterson actually did the work on that one. It was controversial, so I committed it under my name as the main Chainer, and it happened.
00:38:30 But even so, I still recommend Pi Test for new projects. And I use Pi Test for new projects. And I will teach people to use unit tests. And there’s often the section in courses I teach in general on Python.
00:38:43 But when I’m starting a new project, it’s pytest.
00:38:52 If you’re using pytest, do you use the pytest Mark plugin or just use unit test Mark directly?
00:38:57 I tend to use Unit test Mark directly. That’s possible usually because I teach mock as a separate section just because you can spend a day or two days on mocking, depending on how much of the API you want to learn to use and how many different scenarios you want to cover.
00:39:15 And I’m quite fond of Unit test mock.
00:39:19 I’m not sure that’s necessarily best practice with pipe test.
00:39:23 Okay, so when I’m reaching for it, I usually use the Pytos mock plugin, but it has like this mocker object that you can pull in and stuff like that. But anyway, it’s a fixture, right?
00:39:40 Yeah, it’s a fixture that my one problem with pytest is how easy it is to get into fixture hell.
00:39:48 I even worked at one company where they had a culture or convention of providing stuff by fixtures. So you would use fixtures instead of imports.
00:39:58 So you’d look in your code and like your ID no longer has a clue where things come from or where they are and they’ve got fixtures taking fixtures, returning fixtures and you’re trying to work out where something happens and you’re following this graph of fixtures when you’re up to the 8th level and you’re like why my life has descended into hell.
00:40:21 Pictures are fantastic for limiting the scope of stuff. I love the scoping, but it’s like use them sparingly.
00:40:30 Imports are great. Use imports, not fixtures.
00:40:35 Well, okay, well I’ll disagree with you on that.
00:40:41 Fixtures are great as well. I’ve just been in fixed your health.
00:40:43 But also just a reminder that there is no framework or strategy that can prevent you from writing really crappy code.
00:40:55 Maybe Scala or Huskel.
00:40:58 Well, okay, so one of the things that you hinted at, I just want to inject this in here dependency inject this into the conversation with pytest. You can put fixtures either in your test file or in a comp test file and you can have one comp test file for every directory in your test structure.
00:41:18 That was the other thing, wasn’t it? Like this fixture. I’m using it. Where the hell does it come from?
00:41:23 Yeah, and then you scattered across your code base and your fixtures can depend on any other fixture that is anywhere in its parent to hierarchy. But I recommend people in a project to have one comfortest file at the top.
00:41:40 I love that feature. It’s like if I put a confidence up by the root of the project, pytest knows what the root of my project is now, and a whole bunch of other things just work.
00:41:50 It’s one of the nice things about pytest.
00:41:53 And that also helps people if they know if the fixture isn’t in the file. I’m looking at it’s over here. Just helps your whole project.
00:42:02 Yeah, Pi Test is really flexible.
00:42:09 When I started looking at all these things, I did look at Unit Test and I have used Unit Test some.
00:42:17 I actually was annoyed with a lot of the people’s complaints of Unit Test because the thing that people say is it’s too much boilerplate. And with Test Discovery added to Unit Test, you don’t have to do the name equals main thing in your file. There’s not a lot of boilerplate, I don’t think.
00:42:39 Yeah, I mean, it used to be if you used Unit Test, you had to write Test collection yourself and test running and there’s a test result, there’s the test collectors, there’s a whole load of machinery. And really Test Discovery was the minimum to kind of bring it up to this sort of a usable level for projects. A lot of the criticism I hear come from the it’s a Port of like J Unit version five or something, and J Unit evolved a huge amount and Unit Test didn’t. So we have the non Pep eight style naming, and pytest has tests as functions. Whereas with Unit Tests you inherit from Test Case and you write test methods and you use the assert methods instead of the assert statement. And honestly, that’s the boil of like people are talking about generally. And it’s like, yeah, I like writing test functions for pipe test, but I also like grouping tests together in classes.
00:43:40 So I don’t mind grouping tests together in classes. I don’t mind the assert methods. I was suspicious for years of the assert. The pytest Assert magic rewriting because the code that runs the test, they’ve rewritten the bytecode magically so they can tell the intermediate values and all the points, so they can tell you the values of the variables in the expression that failed. And that’s amazing. But it also means that you’re dependent on them having got it right. But after a bunch of years, I was convinced they got it right enough to let go of that.
00:44:15 Using the bearer search statement is great for Test is great, but Unit Test is fine. It works fine. It’s a common style of testing that everyone’s used to. And really the boilerplate boils down to what you need to inherit from Test Case and you call a certain method for doing your certs.
00:44:37 The main thing is, it doesn’t have pipe Test fixtures, right.
00:44:41 Fixtures, parameterized test and code have subtests in Unit tests, which are a nice addition that not everyone will be familiar with. And so that’s the context manager that allows you to have a bunch of tests in a sub test that each can have a separate name and tell you which condition failed with which parameters. So yeah, that’s nice in unit. Test, but there’s a bunch of stuff. pytest plugins are really pretty easy to write. I did some contracting for a firm called Guruk and they have a product called Test Rail, and I wrote a PY test plugin which reports for them, which reports pytest test results and records them in test trail. That was way easier than I expected it to be.
00:45:26 That was great to do. And Unit. Test is much harder to extend and really a consequence of using inheritance as an extension mechanism. Common for frameworks to provide something to sub class. But like test results, if you want a test result that spits out XML, J, unit. Xml, and then somebody else has written another test result sub class that reports orders your tests by time or whatever, you have to use one of the other unit tests. It’s composing. You can’t really compose the extensions.
00:46:03 So the way Pi test does plugins with plug in points. I actually had a version of unit. Test that did this unit Test too, and I didn’t merge that back into unit.
00:46:14 Test.
00:46:14 In the end, it became no two, which was popular for a while, but I think that was Jason Pellegrin. Is that his name? It was his project.
00:46:24 It was a one man project, and he couldn’t keep it going, unfortunately, which was a shame that made Unit. Test behave very similar. But I think the way Pipe. pytest does plugin is right. And it’s an interesting point about extension through inheritance as an extension mechanism from frameworks that it lacks flexibility.
00:46:44 It’s easier to understand at first, but hook functions do make it easier in the long run.
00:46:50 Hook functions with events. I think that’s probably something that we’re becoming more familiar with as more of us are doing async programming. And, you know, it is a different machinery, a different pattern.
00:47:01 Well, Michael, I’m just having a blast talking about testing with you.
00:47:06 I need to wrap it up.
00:47:08 But any calls to action for anybody they want calls to action or anything, drop your we already talked about your Agile obstruction site.
00:47:19 Yeah, give to the homeless.
00:47:23 The other thing I can say, which is I did write an article which is up on Opensource.com and I’ll drop you the URL. You may put it in the show notes.
00:47:32 I think it’s something like 30 things every developer wishes they didn’t have to learn the hard way, which is some pithy wisdom about testing and developing. My experience over the years, it’s quite religious and dogmatic about the testing, but there’s a lot of really good points and some of them we’ve talked about and a bunch of them we haven’t. So I’ll send you that URL that’s up on open source.com and I reckon that’s an article worth reading.
00:47:54 I love that article. Yeah, it’s excellent.
00:47:58 It’s a huge brain dump on people of like some decent practices right off the bat. It’s good.
00:48:04 Thanks a lot for your time and we’ll keep in touch.
00:48:08 Thanks, Brian. I really appreciate it, too.
00:48:14 Thank you, Michael. Great discussion. Can’t wait to have you on again. Thank you, Pie PyCharm, for sponsoring the show. Try PyCharm yourself at testing Code.com PyCharm. Thank you, Patreon supporters join them at testing co.com support show notes for this episode are at testing or five. That’s all for now. Now go out and test something.