I interview Harry Percival, author of “Test-Driven Development with Python”.
Transcript for episode 9 of the Test & Code Podcast
This transcript starts as an auto generated transcript.
PRs welcome if you want to help fix any errors.
00:00:00 If you work with web applications or test web applications, I think this is going to be an interesting conversation with you. I’ve got my very first interview I’m going to present to you. This is Harry Percival. I really want to thank Harry for putting up with web with the interesting quirks of being the first person to get interviewed by somebody. So I hope I nailed down and iron out some of the wrinkles that we’ve had through, but it was a fun conversation. So please sit back and enjoy this conversation with Harry Percival.
00:00:34 Hello, everyone. My name is Brian Akin, and welcome to the Python Test Podcast.
00:00:43 Today I’ve got Harry Percival joining me on the Test podcast.
00:00:49 I’ve been asked some questions about testing and using Python for testing the full stack, the full pyramid of testing, and in general, testing web frameworks. And I’m not a web guy, so I asked Harry to come on. So Harry is here if I learned about him because of his work on the book Test Driven Development with Python.
00:01:18 What is the normal introduction for you, Harry?
00:01:21 What do you mean, the normal introduction to me as a person? Hi, Brian.
00:01:25 No.
00:01:29 If somebody doesn’t know you already, who are you?
00:01:32 Yeah, all right, fine. Well, I guess I’m a relatively new programmer.
00:01:39 I think in 2008, I quit my previous job, which was mostly kind of wrangling Excel spreadsheets and PowerPoint. And I decided to go and do a new degree and do programming, which is something I loved as a kid. And as I thought, you know what? I don’t want to be a management consultant, which is what I was at the time. That’s a stupid job as far as I was concerned.
00:02:00 So I went off to become a programmer and I did that. I did a degree and I got my first client and I hacked away with Python and Django and got into a lot of trouble.
00:02:13 And then I got my first real job with an employer in a company which was called Resolver Systems. At the time. They made a sort of Pythonic Excel spreadsheet or a Pythonic version of Excel, if you like.
00:02:27 And they did sort of like full test driven Development. They did extreme programming, XP PDD, and they taught me that. And I went out there and decided to teach it to the world just initially little things like at conferences, I would do a little workshop or a tutorial, and I started a little online tutorial. And that kind of spiraled out of control into the book that you mentioned, which probably I mean, I originally had the title Test Driven Web Development with Python because it’s so focused on web development. So that’s probably an apt interview start.
00:02:59 Yeah. Okay. That’s good.
00:03:01 So you’ve given us that presentation or the tutorial and presentations around that.
00:03:08 How many times now have you kept track?
00:03:11 No, I haven’t kept track.
00:03:13 I guess I started maybe in 20, 11, 20, 12, something like that with a thing called Tddjango Tutorial. And the idea was to take the official Django tutorial and just run through it, but do TDD instead. So the Django tutorial says, oh, you know, you should open up the start some polls, models and stuff and open up the admin site and you’ll see it. And I said, well, no, let’s write a test first, that you can open the admin site.
00:03:36 And that was just a fun concept. And I turned that into a tutorial, and then it turned into a book. And that was 2012, three or four conferences a year. It must be 1520 times by now, maybe only a dozen. I don’t know if that’s exaggerating.
00:03:54 Anyway, it’s still fun, but it was just starting to maybe get a little stale. So I’ve been starting to think about whether I could do an intermediate level tutorial instead, and I’m starting to sketch out something about that, which would be about kind of more intermediate concept. Maybe we could talk about what is outside in TDD. How is that different from inside out as a methodological approach? And what’s this whole idea of purest sort of unit tests, like what you call isolated tests, where you make sure that each layer of test is isolated from the others, often using mocks. And what are the pros and cons of that with a worked example, which is basically the plot to the chapters 16 and 17 in the book.
00:04:36 Okay, great. I can’t wait till I get there, I guess. I haven’t presented this on my blog or podcast before, but I’ve read the book I’m on my second time, and I’m now learning Django because I’m not a Django developer. So are you familiar with other frameworks like Flask and stuff, or are you mostly Djangoish person?
00:05:02 Most of my hardcore experience, I guess, is the Django. I’m familiar with them because at Tyson Anywhere, which is the company that resolve the systems, eventually morphed into where we run a kind of a platform as a service.
00:05:14 A lot of our users, most of our users, I guess, are hosting web apps, and we host all sorts of Python web frameworks and that’s Django, but also things like Flask and Webtopie and Bottle and things like that. So we had to become familiar with them because so many of our customers were using them and so just get at least the basics of it. So I’ve built a few small apps in Flask, and I’ve helped some of our users build them as well. So I know some of the basics of it. Yeah.
00:05:39 So for the purpose of testing, is testing really any web framework similar to I mean, is testing Flasks similar to testing Django or Bottle and whatnot it is up to a point.
00:05:55 Although because Django has been around for quite a long time, they’ve built out quite a lot more tools around there.
00:06:02 Django is a macro framework. It’s a big framework. It tries to give you lots of tools to do stuff, and some of those are testing tools, whereas with Flask there are a few testing tools, but you probably have to roll your own a lot. So in Django, they’re only little things, they’re only little helpers. It’s nothing you can’t build on your own, but Django does have its kind of official solution, and it suggests you use it. So for testing, a Django app, there’s a version of Jenga will spin up a little Django server for you called Live Server Test Case. Whereas if you’re using Flask, maybe you’d want to in your test, set up a thing that runs Flask first and then runs your test or whatever.
00:06:39 So that’s the answer. I think there’s a few more built in tools in Django, and that’s like any framework, sometimes that’s great. And then sometimes actually, what you need is not exactly what the Django developers anticipated, so you end up rolling your own anyway.
00:06:53 So that’s not necessarily a real reason to use Django, but I found it worked pretty well as examples in the book.
00:06:59 Okay, well, I know that the book focuses on the Unit test portion, Unit Test. The tool, not unit testing, but I know that you’ve played with Pi Test a little bit.
00:07:16 Is one better or worse than the other or just different for testing web applications?
00:07:22 Yeah, I don’t know. It might be a little bit like anything like a fad or a novelty, but I feel like about a year and a half ago, maybe I heard more and more people talking about Pi Test, saying we’ve looked at Nos, we’ve looked at Unit test, we looked at Pi test, and the one we prefer is Pi Test. It’s the most fun, it’s the most.
00:07:43 Why did they say that? Some of the reasons are that it lets you write your test using just the assert keyword rather than in Unit test. Having to remember, self assert equals self assert. True self assert regex matches self assert greater than all of these things.
00:08:03 The Pytech philosophy is if you just write assert A is less than B, then Pyte should be smart enough to know, okay, well, look, you’re doing a less than comparison. So when I need to report your errors, I can inspect those two variables and tell you A was not less than B, so achieve the same result, which is that when you make your assertions, you can compare or tell you in what way the test fails, but it does so using a bit less boilerplate. So people like that, and then some of the arguments against it, particularly from my colleagues. Hi, Giles. Hi, Glenn.
00:08:37 Is the Pyte’s approach to fixtures by which pytest meet. And so there can be a confusion here, because in the Django world, the word fixtures tends to mean some data that you want to put in your database before you start your test. Whereas in the Pi test world, a fixture is a slightly more general word and it means anything that you need to set up before running your test.
00:08:59 And in the unit test world, what that means is stuff that you would normally do in the setup method.
00:09:05 So in the unit test world, we have your set up, your methods that run before and after each test, and then your test or anything. Any method begins with test underscore. Well, in pytest it’s slightly different. You write a function that’s going to be your test fixture and then you put that function somewhere special, and then in your test function, if one of your arguments names matches the function name, pytest sort of magically injects that fixture in as a dependency. And so a common complaint is that that’s too much like voodoo, that’s too much like magic. And I think that’s up to a point. That’s a fair criticism. But then again, if you’re used to unit test, which is based on the sort of Java unit testing frameworks, the junior philosophy, then you’re like, right, fine, obviously I have a set up and a tear down and test underscore methods in my class, but actually that’s magic too. You’ve got no idea how that works. If you ever have a look at the source code of unit test in the Python source tree, you’ll see that it’s incredibly complicated. Actually, getting a test runner to work is not straightforward. And so maybe it’s a different type of magic. Magic over here is set up runs, then your test runs, then tear down runs, and the magic over there is you name your tests and then the names of the arguments and your test functions match the name of these fixture functions, which is what’s going to get run, what’s going to get injected into your test at the time.
00:10:21 Yeah. Does that make any sense? I feel like I’m a little bit rambling this evening.
00:10:25 No, I think I’m letting you run with this because it was interesting to me to find out that the fixtures has a different meaning. I have run across that, people saying that, thinking that when I’m talking about fixtures, I’m just talking about data, which has always confused me because I come from an instrument test world where fixtures are actual physical fixtures, or I’m setting up a signal or something like that.
00:10:55 What do you mean, like physical fixtures?
00:10:57 Well, right. So if I’m in electronic testing, the setup methods might actually go out and set up a switch matrix and get a signal from one place to the antenna that I’m test for the device I’m testing on things like that.
00:11:20 And that does sound a lot more physical.
00:11:21 Yeah.
00:11:22 I wonder why it’s the word fixtures like that, whether it does sound a little bit like you’re going to Bolt something to a desk before you’re going to test it. Like it’s from a real sort of real life woodworking testing, physical engineering kind of lab thing, doesn’t it?
00:11:40 Yeah, well, I mean, in the electronic world, it translates fairly readily. The notion of just having data that confused me. I couldn’t figure out how that would translate to fixture.
00:11:53 Okay, yeah.
00:11:55 So I think maybe overall it’s Django. That is slightly misusing the term or giving the impression that the term is more specific than it actually is.
00:12:03 So for some of your colleagues that don’t like the fixture model, you could remind them that pytest supports X unit style fixtures just as well. It still supports setup and tear down.
00:12:15 I guess so, yeah, that’s true. But I guess what they’re saying is, hey, look, if the whole Pinterest philosophy is to use these magic fixtures, well, we think they’re too magic.
00:12:25 I think that’s partially effective, and then partially it’s from lack of familiarity. The more you use them, the more you sort of appreciate them when they feel natural. It’s like anything like any technological decision. We try and make these decisions based on sort of rational bases, but there’s a part of you that’s just jumping on the bandwagon and, you know, maybe there’s nothing wrong with that. I’m having quite a bit of fun jumping on the Pytech bandwagon.
00:12:51 Yeah, actually. And I also think it’s sort of funny hearing that argument coming from people that are already on the Python and the Django bandwagon anyway, and they’re kind of magical as well.
00:13:08 Yeah.
00:13:08 All right. So that’s the frameworks. So how about the different levels of testing? Can you use Python?
00:13:18 Is it just through Selenium, or is it full stack?
00:13:24 How much of the pyramid can you use unit test for?
00:13:40 I guess when you say the pyramid, can you use it for end to end tests, functional tests, as well as for sort of unit tests? And I think the answer is yeah, absolutely. Just like you can use the standard library unit test module for functional tests, you can use pytest for functional tests. And in the couple of blog posts I’ve written where we were talking about Pi test, they’ve both been for quite a high level test. Functional tests. The first one was about we were building a kind of Postgres service which is based on you spin up some little Docker containers and each user is going to have their own Docker container, and that Docker container is going to be running Postgres. And then we’re building a little actually Flask based API around that whose job it is to start and stop those Docker containers and provision new Postgres servers for users.
00:14:27 And the tests that we wrote for that were ultimately use the request library, Ping a URL where we expect Flask to be running. So the tests assume that Flask is actually running on the machine they’re running on.
00:14:41 And then the next step in the test, after calling Request Post or whatever is open up.
00:14:48 What’s the stupid name of the Python Postgres library. It’s really impossible to remember something silly. Anyway, open that up and actually make a database connection to the API has told you, try this Port to a host name and a Port and see if it’s actually running there. So that’s very much an end to end test. We make an actual Http request, and then we make an actual database connection and see if Postgres is waiting on the other side. And in between that sort of input and output.
00:15:21 I’ve talked to a Flask I’ve talked to actually. I think even NGINX, which is reverse proxy to Flask. Flask is called out through the Docker Python API through to Docker, which has started up an actual Docker container and then started up Postgres and Postgres bound to a Port. And then now listening.
00:15:40 And so you’ve got like, that is about as full of stack as I can imagine.
00:15:44 And yeah, you can absolutely write a test for that. And Pi test. And quite nicely, too.
00:15:51 Those are definitely full system tests.
00:15:57 I can’t remember the name of the term. So inside of a Django project, there’s like mini projects or something.
00:16:05 What do you call it? Projects and apps.
00:16:06 Apps. All right, so the test directory within an app, is that a unit test at that point?
00:16:16 Yeah. Well, I talk a little bit about that.
00:16:19 Where do you put your functional tests in a Django project structure? And I mentioned that I think maybe in chapter six in the book.
00:16:27 And so the idea is Django kind of encourages you to do testing, and it puts a file called test. Py inside each one of your apps. So you’ve got a project. A project contains many apps, and each one of the apps has a test file.
00:16:40 And the apps might be things like accounts forums, I don’t know, API, whatever, like different components of your whole project.
00:16:57 And my philosophy is that for unit tests, it makes a lot of sense to keep your tests inside the app, because the tests are about testing the code, and each app is reasonably self contained. So you probably have a self contained set of unit tests in there, and then for your functional tests, I think you could go either way. But overall, my feeling is that a functional test, a high level test, is really testing the whole project from the point of view of the user. And your user doesn’t care that you’ve split up your code into nice little self contained apps, because all they’re seeing is a website. So I feel like the functional tests are probably separate from that structure.
00:17:29 And one functional test might actually use several different apps. You might use the accounts module to log in, and then you might need to use the billing module to pay for something, and you might need to use whatever.
00:17:41 So that’s an argument for having at least some of your functional tests kind of separate from the Django apps structure that makes sense.
00:17:52 And then are there other levels of testing that you’re doing on an application or on a project?
00:18:02 Yeah, I think there are. Well, there can be, and I think it depends on the level of complexity of your project. So for web projects of the kind that involve a reasonably rich front end, I tend to recommend you have at least some tests that use Selenium that really test the kind of the full stack, I guess, that really tests that a real Web browser can come up, visit your site, click on things, and then there’s a web server at the other end that’s responding to those requests and showing the right information to the user.
00:18:35 And then at the other end, you kind of need some lower level tests. And Django gives you this thing called the Django test client, which is actually encouraging you to kind of write a halfway down the pyramid kind of test where it spins up a more or less realistic version of the Django framework, and you can test it with a thing called Self client that can dot getand post. And I think that’s quite good. That gives you quite a sort of reasonable kind of middle level of tests. And I think for a project that’s not too complex, that’s a pretty good balance of types of test and code. Django, you write your Selenium test and code. They are with a real Web server, real database, and you write your Djangoe tests, and they also use the database, but they’re going to maybe use the SQL in memory database. And so it’s a bit faster.
00:19:24 And you can sort of test your application in a way that’s reasonably pragmatic and lets you test your models and test your views and test the responses, come back all right. And that the site really works in the point of view of the user. I think after a while, if you really grow in complexity, you may start to want to take those kind of slightly mushy Django tests that test at multiple levels, and you want to start to think about actually having one layer of end to end tests, one layer of kind of integration tests that check that things are kind of glued together okay. At a sort of intermediate level, and then a lower level of unit tests that maybe don’t use quite so much of the Django middleware stack, or that maybe avoid using the database.
00:20:14 I’ve never worked on a project where that architecture is perfectly realized.
00:20:18 But I think that’s when you start to have an application that’s so complicated that you want to try and abstract a core of business logic actually out of Django. And so you want to have some core of your application that doesn’t depend on Django that only talks to Django at the very end when it wants to save from the database or retrieve from the database. And the business logic is reasonably self contained, and you could write some really focused unit tests in there that are very much what you’d call isolated tests.
00:20:48 Yeah.
00:20:49 Is it a good time to talk about kind of a test isolation and things like that?
00:20:54 As it turns out, it actually wasn’t that good of a time to continue our discussion about test isolation.
00:21:00 Harry had a pizza delivery. I needed to go back to work. It was my lunch hour when I was doing this interview, and the audio quality just started to crud out on me. So it turned out to be a good thing. We decided to pick up the conversation two days later. And so now here’s the rest of the conversation I think we covered like Pi test versus not no, Pi test versus unit test pretty good.
00:21:26 And the different levels of testing. And we were going to pick it up with test isolation. And then I’d like to talk a little bit about I’d like to talk about test isolation, but I was hoping to also talk about a couple of other topics, like whether or not you use external QA and to make sure we get to talk about the process of your book a little bit.
00:21:53 Oh, sure.
00:21:56 I haven’t got that far in your book, but I have, I guess, some personal thoughts. At least in my field, Isolating tests is both a good thing and a bad thing. So when you were mentioning test isolation, how much do you use and what is that?
00:22:18 Yeah, test isolation. I guess I think of it sometimes like a little bit of a Platonic ideal. So we have our functional tests that we’re pretty happy with, where we’re really testing the application from end to end. And when we have a functional test that passes, we’re pretty sure we’ve really got a working application.
00:22:37 And the kind of other end of the testing scale is that we want to have unit tests. And what does that mean? It means that you want to test a specific unit of code.
00:22:45 And typically that’s going to mean like one function or one method on a class if you’re keeping to sort of good coding practices. These are quite small. They’re three 4510, 15 lines.
00:23:01 And your unit tests are designed to kind of isolate down to just that function or just that method and help you get it right.
00:23:13 And so to have a really good unit test, you want to test just that function, and you don’t want that test to fail for reasons that aren’t to do with the logic of that function. And so that’s why we try and isolate it. So if that function happens to want to talk to a database or talk to the Internet or talk to a hard disk, we don’t want our test to fail. In the theory, we don’t want our test to fail because the database is misbehaving or because the hard disk is misbehaving. We want to sort of isolate it from that and just test the core logic.
00:23:41 And so that’s when you get into things like mocking or the alternative to mocking is to try and make it so that your functions don’t contain anything to do with the database or the file system or whatever, and abstract out the core of your business logic from your interfaces to the outside world, your interfaces to the Internet or to the disk or to the database.
00:24:06 And so those are your two approaches. Either your functions involve databases, whatever. So I knock it out, or I’m going to try and extract a kind of functional core of my application that’s not going to have any dependencies. And then I can test it with a very clean unit test that just goes sort of data in, data out.
00:24:22 And I think the thing about that ideal is it’s really good when you’re trying to develop some sort of complex algorithms.
00:24:31 If you’re trying to build the example that jumps to mind is a pause, because that’s what I’ve worked on when we were at Resolver One. So you’re trying to make, I think, the Translate Excel dialect into working Python code.
00:24:44 And it’s very much a tool that takes data in one end and puts data out the other end. And it doesn’t have any dependencies. And it’s just logic. It’s breaking down things into characters, identifying relationships between them. And it has a logic for building a tree structure, a pastry in this case, or if you need an algorithm for take jobs off a queue and take as many as you need, but only the high priority ones. And so you’re trying to build an algorithm that’s going to have particular behaviors. I think these ideas are very pure. Unit tests are the perfect way of testing those.
00:25:18 And so a lot of TDD literature out there will tell you about this kind of testing. And I think that sometimes clashes with the real experience of web developers, which is what I am, because we’re sort of looking around at our applications, and our applications are essentially just thin wrappers around a database. I mean, as often as not when you build a web application, it’s just a database with a web front end. And you’re going, you know what? I don’t have an awful lot of business logic to abstract out of this. All I really want to do is take the user’s input, save it to the database and tell them that’s done. Or when they ask me for something, I’m going to go and query the database and show them the results in a pretty way. And, you know, really there is no business logic for me to abstract out. So I don’t see where I’m going to write these unit tests. And in the meantime, putting in all the effort to do that, I just want to get work done. And so Django’s philosophy of the framework for perfectionists with deadlines is about saying, okay, well, look, the simplest way to build a thin, simple kind of database driven application is we have a view function which interfaces with the Internet by producing kind of HTML, and it interfaces with the database by using the sort of models, objects. And the best way to test that is to use an actual database and look at the actual HTML that comes out and maybe even do a real request to it so you can test your kind of URL routing and your database and your HTML all in one go.
00:26:42 And that’s the tool that the Django developers gave us. And they’re not stupid. They built the tool they figured that they wanted that they figured was the most useful for testing the kind of applications that get built with that.
00:26:54 It looks pretty my limited access to it so far. It is an interesting and very useful part place to slice the application in half.
00:27:06 I think that’s it. But the danger is that that’s fine at first for a simple application. But as it grows, you get more and more tests, they get slower and kidding yourself at some point that you haven’t gotten any business logic. And maybe if you did put in the effort to kind of try and abstract it out, you might be able to understand your code better and have a nicer architecture. But I think that’s still even in my mind, that’s still quite theoretical.
00:27:28 Yeah. I think of it as unrealistically theoretical.
00:27:34 I’m not a web developer. I do embedded systems. And the notion that you can really always have it’s an event driven system. You’ve got access to the hardware. I really want to test the whole thing, splitting things up and mocking stuff, the pure algorithm parts just not where my bugs usually are.
00:28:01 The bugs are usually in putting things together wrong or misunderstandings of interfaces.
00:28:11 There’s a lot of assumptions in the system. There’s a lot of assumptions, especially with teams that are large.
00:28:18 You don’t write everything down correctly or you use a third party library and you’re not quite using it.
00:28:26 Right.
00:28:26 Or you’re using it a little bit out of the normal workflow.
00:28:32 And those are the hard parts. Those are the real bugs.
00:28:37 There’s also one of the things about the ideal of isolated function test.
00:28:46 There’s this notion that if a test fails, you want that functionality to be the only if a functionality breaks, you only want that test to break, perhaps.
00:28:56 Yeah. The perfect world is like one test fails and you’re like, right, that points me exactly at one line of code to fix.
00:29:02 Yeah. And I think that sort of ignores the rest of the development our system. So if I have a bunch of tests that fail, I’m pretty sure that the code I just added might be the stuff or the code that was added since the last time everything passed. That’s probably where the bug is, right?
00:29:22 Yeah.
00:29:24 We also have revision control systems and continuous integration servers to help isolate where the problems are as well.
00:29:32 Yeah, I saw a tweet, I think, or just a little wittiness by someone saying the tests that give us green functional tests are what give us confidence their application works. But a red unit test is the most useful thing for debugging, which I think kind of captures that idea that a unit test is pointing at a very specific problem.
00:29:55 But you know what, I tend to find that my unit tests just don’t fail very much. Once I’ve written them, that’s pretty much there and it’s fine. The tests that fail are the functional tests and they’re the ones that are actually hard to fix.
00:30:08 And that’s because our software is difficult. And I also have to say that my perspective is my perspective and I work at a company called Python Anywhere and what we do is we build a web application. But that web application builds other web applications and it builds Docker containers and it talks to several databases and we spin up processes on the disk and we move them from server to server and our application is all boundaries and talking from system to system and it’s all integration points.
00:30:32 So obviously I’m going to be talking a lot about how the most useful tests for me are integration type tests and other people might have a different experience. It sounds like you’re saying, yeah, it’s similar for you, but the pure logic algorithm types of tests just aren’t that big of a part of what you have to do every day.
00:30:52 Yeah, that’s definitely true. And I only could think of one person I know or one team of people that have to deal with just pure algorithmic sort of stuff.
00:31:08 But these are like DSP signal processing where you really have like fixed, but then at the same time they can okay, I want to back up a little bit.
00:31:22 There is definitely a case for mocking and stubbing information so that you control the situation.
00:31:31 So you can say, well I need to know when if I add a user and that user ID is already there, well then I have to kind of mock up a database where that is going to be the case.
00:31:48 So the other thing I think that the arguments of the isolation fall down is just I don’t think people are utilizing fixtures enough or set up and tear down because if you push all that database stuff into your fixtures, your test isn’t going to fail, it’s going to error and that’ll be different.
00:32:09 Yeah, that’s maybe hard to debug too.
00:32:13 At this point I really wanted to just give a little name check to Gary Bernard who is a wise Sage in the TDD world and he’s looked into this stuff for much longer than I have and he’s got a couple of excellent talks on this subject. The talks are called boundaries and then there’s a follow up called Functional Core Imperative Shell. And so anyone that’s interested in this sort of stuff, I really encourage them to go and have a look at those two talks. I think they’re Python talks from around 2013 boundaries.
00:32:38 And what was the other one?
00:32:39 The other one is called Functional Call Imperative Shell.
00:32:42 I will have to check those out.
00:32:45 Now I’m going to try to get off this soapbox because I’m a little passionate about it.
00:32:50 So shift gears a little bit.
00:32:55 You have fairly complicated system. It sounds like. Do you have an external QA team or do your developers do all of the testing?
00:33:04 Yeah, no, it’s just us. I mean, there are four people that work at Python anywhere. And so we are the testers and we are the developers. So we follow extreme programming methodology. We now have short iterations.
00:33:19 We work on features one by one, and we do them in a test driven way. We write a functional test first, unit test second, and the code third, and that’s it. We have a continuous integration build telling us when we accidentally broke other stuff, we rotate around. We do everything in pair programming. So we’re the testers and the only other testers are the poor users. I’m afraid it will come along and tell us if we’ve accidentally coded any bugs or tell us when the performance is in up to scratch, which is another difficult thing.
00:33:49 So your addition to the normal TDD cycle of adding a functional test first, does that enable you to not.
00:34:01 So does all of your testing fall into that cycle, or do you have some tests that need to be written outside of the TDD cycle?
00:34:11 I don’t know if we have any tests. I mean, we tend to always go for that philosophy of functional test first and then unit tests to build the code that’s going to make that functional test pass. And then we call that double loop TDD, sort of a loop of functional tests and smaller loops of unit tests encoding inside outside of that. The thing we do do sometimes is we are trying to build new technologies. So we do go off and occasionally one of us will pick up an investigation of a new technology, and they might do that without tests. So my colleague Glenn went off and tried to figure out how we could integrate I Python notebooks onto the service.
00:34:50 And thankfully, so often in the world of open source, somebody had done all the hard work already. So huge, big up to the people behind Jupiter Hub, which we’ve integrated recently.
00:35:03 So they’ll go and learn that technology and do that sort of outside some tests, and that’s a sort of spike task. And then they’ll come back and go, okay, well, let’s Despike this. Let’s take my prototype and rebuild it as a kind of production ready thing. And that we will do once again with functional tests.
00:35:18 Unit tests but sometimes you don’t know what tests what are your tests going to look like until you’ve built that prototype, until you’ve understood that technology that you’re integrating. So that definitely does happen.
00:35:28 Well, cool. Well, we’ve covered a bunch of topics, and I know once I finish your book, I’m going to come up with probably several more questions along the way that hopefully I can throw at you.
00:35:45 And we also flew by a bunch of topics that maybe we glossed over a little too quickly.
00:35:55 I hope that anybody listening to this can let us know if we need to cover something in more depth and we’ll try to follow up with that.
00:36:04 But on a personal now, this is a personal note.
00:36:09 I do try to make a little money on the side from selling my own Python testing framework book.
00:36:17 So I do know that experience of doing it all by myself. But I asked you briefly the other day, but I wasn’t recording at the time about your process with O’Reilly and having an open source and not open source. But the book is just freely readable on the web.
00:36:41 Yes.
00:36:42 I wanted to just touch on that a little bit. How is that? Do you feel that’s a good model? Are you happy with that?
00:36:48 Yeah. I mean, that’s worked very well for me. It was important to me, I guess, because I wanted to we work in a world of free software. Python is free software, Linux is free software, open source, and we get all this stuff for free.
00:37:07 I think there’s an immoral imperative on us to try and give back, and different people do that in different ways. And one of the ways you can do it well, I felt that I could do it is if I’m going to write a book, let me make sure that it is also free if people need it.
00:37:23 And the great thing about that so I spoke to it was another publisher that approached me at first and said, hey, do you want to write this book? And I went, yes. And I went, listen, I’d write to release it on the Creative Commons. And they went, no, kind of give it away for free, which is a wonderful attitude from the 1980s. And you have this illusion of thinking I can’t give my book away for free, then people will be sitting there with this choice between a paid for version and a free version. So that’s right. You think, okay, hey, I can’t be giving people the choice between a paid version and a free version. I don’t want that. I want the only thing out there to be the paid version. But of course, that’s not the world we live in anymore. The moment you’ve released a book, it’s going to be outlined online and available in pirate copies as soon as you start, even if you’re like. You know what? I’m only going to release the physical book and that way people won’t be able to get a hold of a PDF, which is too easy to copy. If you’re so determined that you’re going to cut off all the ebook market and say, no, it’s just going to be a physical book. I mean, you can send books to be scanned. I saw a company that was doing it for $5 per 100 pages. You send them a book and for every 100 pages they charge you $5, they’ll send you back a PDF of it and they just do it by slicing down the back of the book binder and putting it through an industrial scanner.
00:38:47 It is so cheap to make digital copies of things and distribute them, but there is no such thing as a choice between having a paid edition and a free edition. It’s like either you have a paid edition and a free edition that you’ve authorized and that you’ve said that you’re happy for people to read, or it’s a paid edition, plus the pirate edition that you’re making people break the law if they want to read your book for free. And so honestly, I’d rather just say, you know what, if you really want to read the book for free, here it is for free. And then I think that then rather than having put yourself in a kind of adversarial relationship with your readers because there are some people who maybe they can’t afford it or maybe they really want to try before they buy.
00:39:25 They want to just have a look first. Those people, if you force them to break the law first, you’ve immediately got into a confrontational relationship with them. Whereas if you say instead, hey, okay, here’s the book for free. Read away and you’re just upfront about it and you say, hey, this is a tribe before you buy kind of scheme. If you like this book, I hope you’ll consider buying a copy. And I say pretty much this in the forward. I say if you like this book and you’re reading the free version, I hope you’ll buy a copy, even if it’s not for yourself. Maybe buy it for a friend or buy it for your local library or your local hacker space or whatever it is. And this is where you can buy it and whatnot. And I think that works really well. O’reilly are totally on board with it.
00:40:02 It is the modern way, I think, of evolving in this sort of space. And maybe we’re lucky, I guess, as tech publishers that it might be a different equation if I was Stephen King writing kind of beach reading novels or airport novels. But when someone’s reading a book like mine, they’re reading it for very specific reasons to do with their career. You don’t read a book called Test Driven Development with Python on the beach, or if you do it’s because you’re trying to learn something to go back and do differently when you finish your holiday. So it’s already a commercial transaction. People see a very clear link between reading that book and their job and getting paid and money. So I think if they’re reading the free version, they’re getting something out of it. It’s much easier for them to make that moral jump to, well, maybe I should pay for this. Whereas I don’t know, if you’re reading Stephen King’s book, you might be thinking, wow, it’s just a holiday book. This isn’t a commercial thing.
00:41:00 I don’t know.
00:41:02 Yes.
00:41:03 I also can’t expense report my Stephen King novel.
00:41:09 Right.
00:41:11 So that’s it.
00:41:12 I think people will pay for it. And the other thing that’s great about it is it means there’s all sorts of people who could never pay for it.
00:41:19 So there’s people who just like kids and students who are just too broke to buy this sort of thing. There’s people out in the developing world who like, there’s no local publisher making addition for them, and whatever it is, like $30 for a book might not be too much for us, but to some people, that’s like a month salary. So obviously they’re not going to pay that. Or I had people from Iran talk to me and say, well, hey, I’d love to buy this book, but the US export controls means that we can’t they’re not allowed to buy American books here.
00:41:52 Weird.
00:41:52 So for all those people. Yeah, that’s the real thing.
00:41:56 So for all those people, okay, I’d rather they were able to read the book than not.
00:42:01 And who knows? Maybe that the trade sanctions will be lifted or someday they’ll come back and buy somebody else’s books, they will pay it forward.
00:42:09 Okay, well, actually, I’m learning quite a bit from it, so I appreciate you writing it.
00:42:16 Personally, I started out reading the first read I read through it was on a Kindle, and then I’m reading the physical copy now, and I don’t know what’s different about it, but it is different. I can do the exercises and go through it quickly with the physical book.
00:42:39 Way easier than the Kindle, and it shouldn’t be.
00:42:42 Yeah, it’s much easier to flick through a physical book.
00:42:46 Flicking through the pages is much easier than trying to fiddle with a Kindle’s crappy UI to jump through it. And secondly, you have this real sort of like we’re monkeys.
00:42:57 So flicking through a book and having a kind of like three dimensional tactile feeling of like, how far through it you are so that you remember, oh, I saw that thing. It was about half an inch of the way through the book. You can flick to it in a much more instinctive way than Kind of going, oh, now what was that? About 18% Amazon Kindle Mark, 300 out of 5738. Like, forget it.
00:43:22 Yeah. So now I’m feeling a little guilty that my books only in ebook form, so I’ll have to do something about that.
00:43:31 Look, I don’t want to take up more of your time but I do want to really thank you a ton for being the first interview on the podcast and taking your time to talk with me.
00:43:44 Oh, no. Hey, you’re very welcome, Brian. It’s been my absolute pleasure. I’m flattered, to be honest.
00:43:48 Honestly and I hope everybody enjoys this.
00:43:54 Actually I’ve had a really good time.
00:43:57 It’s actually quite a bit easier to interview somebody than to come up with a half an hour’s worth of content on your own.
00:44:03 Oh boy.
00:44:04 Yeah.
00:44:05 Anyway, so hopefully I’ll talk to you later and we’ll stay in touch. Thanks a lot.
00:44:11 All right. Goodbye, Brian. Goodbye, everyone. Good night.
00:44:14 I’ll put a link to Harry’s book on the show notes that’s at pythontesting. Net nine. I think we’re at nine now and you can find all of these show notes at Python testing. Netpodcast so huge. Thank you to Harry for being my first interview and I think that wraps it up I’m way overtime. I hope you enjoyed it. Please let me know either through the comments or through Twitter, at Brian Hawkins or at test podcast. Let me know what you think of the interview and tell me who you would like me to interview more of. I’m not going to do all interviews. I still want to give you information and shorter episodes but I do like interviewing people. There’s quite a bit of quite a good time so give me some feedback let me know who I should interview next. Thanks and have a great day. Keep on testing.