IDEs can help people with automated testing.
In this episode, Paul Everitt and Brian discuss ways IDEs can encourage testing and make it easier for everyone, including beginners. We discuss features that exist and are great, as well as what is missing.
The conversation also includes topics around being welcoming to new contributors for both open source and professional projects.
Transcript for episode 106 of the Test & Code Podcast
This transcript starts as an auto generated transcript.
PRs welcome if you want to help fix any errors.
00:00:02 Ides can help people with automated testing. In this episode, Paul Everett and I discuss ways that IDs can encourage testing and make it easier for everyone, including beginners. We discuss features that exist in our are great, as well as those that are missing. The conversation also includes topics around being welcoming to new contributors for both opensource and professional projects.
00:00:39 Welcome to Test and Code.
00:00:41 A podcast about software development, software testing, and Python.
00:00:49 Welcome to Test and Code. I am excited to have Paul Everett on today. Again, I always am bad about looking at what the old number was for, but I think you’ve been on the upside before, right, or the podcast.
00:01:02 Well, actually, the big thing you and I did was that joint presentation at PyCon.
00:01:07 Right.
00:01:08 How about that?
00:01:09 So tell me about this joint presentation at Python. You mean the one where we did a presentation together?
00:01:17 We were going to do a presentation together. We worked on the material together. You wound up giving it solo just because of the format didn’t fit a splitting. And that was your first Python talk, right?
00:01:28 Yes, it was. So thanks for that twist half an hour before the talk anyway, but no hard feelings.
00:01:39 Well, who are you, Paul?
00:01:40 I’m a developer advocate at JetBrains, primarily Pie, Charm and Webstorm.
00:01:46 Okay.
00:01:46 I’ve been around since Moses was a kid.
00:01:49 Yeah. You’ve been involved with Python for a long time. I am.
00:01:51 I’m kind of old man.
00:01:53 I know I’ve had you on, so I’ll dig up to put it in the show. Notes later. Developer I forgot for Pyramid. I use Pipe Arm every day. Are there more than one developer advocate advocate?
00:02:04 Indeed. We have a team, some specialized in certain things. We’re growing the team, and it’s a great gig. Great company.
00:02:12 You stand in a booth and people come up and say, we love you and you get paid for that. How’s that?
00:02:19 That’s nice. Yeah.
00:02:20 But yeah, there’s a team of us, and our job is to have 1ft in the community and 1ft in the product and be the voice of the users and customers at the table when decisions are made, and also to be kind of an educator and a spokesperson.
00:02:38 Cool. Well, one of the things I want to educate people and educate is this visual testing.
00:02:45 Well, one of the reasons why I use PyCharm is because it’s got a VM integration, so I didn’t have to relearn all my key bindings. The other one was that it does testing well and it integrates with Pi test. It also does other test frameworks, too. But who uses other test frameworks?
00:03:06 You’re about to get a bunch of content.
00:03:09 Most people that were unit test advocates that I’ve talked to, there’s a few that really have good reasons, too, and I’m fine with that, of course. And unit test, I want to have improved as well. I think that it should not be a dead project it should live on as well. But I’m a fan of Pipes. It’s really beside the point, I guess. So one of the things we wanted to talk about today, actually the main thing was what you called visual testing or something.
00:03:35 Indeed, I got the idea. I think after reading your book, plug your book a little bit. Your book really got the idea. The joy, I’ll say the joy of testing planted in my brain.
00:03:46 I like that.
00:03:47 And as I was fulfilling my role being an explainer, there was kind of this feeling that Ide’s are big tools for professionals on big projects. And there was also a tool that there was kind of a concept that testing is a big workflow for high end people on projects. And in both of those cases, I kind of stepped back in partially reading your book, and I wanted to kind of revisit that and say to myself, well, maybe testing is what entry level people should do as part of their learning journey in the beginning, and maybe ideas can help that. And the thinking was, when I do sessions on Debugging, you know how many people out there still use print for Debugging?
00:04:38 I’m raising my hand right now.
00:04:39 Oh, my God. And so there’s kind of a similar thing about a visual debugger can make debugging more accessible to people because people might fall off the cliff with this Arcane command set that they have to do for Debugging. And it just feels like a different activity, different workflow. I’ll just stay in my editor and sprinkle a print command. What if you could stay in your integrated development environment and have a visual debugger? Same thing for testing. What if there was a visual testing that felt like part of what you were already doing gave you a visual representation of the aspects of testing, and maybe people wouldn’t fall off the edge? Maybe testing would become more of a first class beginning part of development. And the other part was the joy of testing. What if testing was fun? What if testing was what you did because it was the most productive way to do the simplest of tasks, simpler than running something, reloading a browser, or 57 other things. You just sit in the test tool and then hopefully the next thing we’ll talk about, auto run gets you into this mode that is more fun and more productive than the quote unquote normal way.
00:06:00 Wow. I don’t have to convince you of anything. This is awesome. I think that testing is fun. And I was actually surprised that I still have to convince people that it’s not or convince people that think it’s not. And I really probably hit you up for an introduction to them. My next book, because that was great.
00:06:18 Your book is the thing that started the thinking for me. Thank you.
00:06:21 I want to talk about a lot of the different things, the different reasons why I used why PyCharm is a good fit, and this isn’t necessarily PyCharm. I think all the things we’re going to talk about really kind of have to be supported by any IDE that does testing. Indeed, if I’m learning it is called test first, right? And it is surprising to me that we call it test first.
00:06:47 Tdd started out as test first, but we don’t teach testing to people until later, even though we really kind of do teach them testing because we teach people how to run their scripts on the command line or through the run command on the debugger or something like that. But why not teach testing? And one of the things I love to do is just that playing part to just play with some code. One of the great things about using ID and PyCharm for writing tests is I can write I don’t have to write a script to run a script. I can just write one function like a little test function and it can be like two or three lines of code to try out something and it’s got a little run button right next to it and I just hit run and that’s it. I can see the output and everything.
00:07:36 So before I came to Pi Charm, I was using various other Editors in the past, but there was a common thing of I was really shying away from huge Ides and I liked the command line because command line is really fairly useful. So I’d have like an editor open something that either vim or something that emulated them and then having another window open within the directory of just a command bash window or case shell or something where I was in the directory where my tests are so I could run the test and rerun them. So you’d save and then go back and run and save and run. So there’s whether good or bad. I know I’ve talked to people that like it or don’t like it. Pycharm you don’t have to save before you run. It’s just saving all the time I think. But also when you run you get like the output window pops up or the run window. And one of the things I like is if I’ve got multiple windows open I have to manage where they are. And the IDE settings, the function that I’m running is still visible, but it sort of shrinks everything up to make room for the run window. It just seems to work great.
00:08:47 The point for that is the same experience for me, being a man of a certain age. Let’s just say I don’t have a lot of empty registers left and that little bit of context switch looking, moving my eyeballs from the editor over to the command window with the output of the testing tool is kind of enough of a friction that it kind of throws me off a little bit, messes up my flow. And this idea of staying in the flow is really something that appeals to a class of people.
00:09:22 Yeah, like the class of people being programmers. Right?
00:09:25 Some people like having different tools for different things, the Unix philosophy, stuff like that, and the I and IDE is really a put off for them. But there are benefits to the I and IDE, especially when going into TV.
00:09:37 One of the other things is I am a command line kind of person, and all of the command line flags and being able to play with that, you haven’t hidden it too much.
00:09:47 Correct.
00:09:47 Once you know it’s there, you know how to get to it quickly, but there could be better, but it’s pretty darn good. I haven’t come across very many things that I can do on the command line that I cannot do by just adding those command line flags to the right thing.
00:10:04 What do you want to talk about next with this?
00:10:06 One of your favorite things is that auto run button.
00:10:09 I’m really glad the auto run is there, but I don’t really use it much.
00:10:13 All right. I think you pitch it to other people sometimes when I’ve been standing.
00:10:18 I think it’s neat. I think it’s neat that it’s there because it looks like magic. So there’s the auto run. You turn it on and you can set up, unlike the controls where you can say don’t run right away, but there’s a time delay or something. If you say something and things are silent for a little while, it’ll run. You rerun your test. And that’s pretty cool.
00:10:40 That’s a fun flow for me is to zero in on a specific test and then turn on the auto run and set the delay correctly. And then as I type, I don’t even have to remember to hit save and my test will rerun after delay. And that kind of mode is like one less thing for me to think about. But the bigger thing is there’s something in the back of my head constantly saying, well, you’ve Typed a whole lot, Paul. What are the odds that your tests are going to run? Yeah, because you’ve forgotten to run for about 2 hours, and when it fails, you’ve got a lot of stack frames to go back over to get back to what you were thinking when it was last working.
00:11:19 It’s interesting you bring that up, because that sort of workflow is not just test driven development. I mean, TDD promotes that idea of test bouncing back and forth, but also just running your tests a lot. But that doesn’t I mean, even if you’re doing tests at the same time or test after or something, writing a lot of code before you run the test again, even the older tests, the regression tests, once you get used to it. I have that feeling, too, of like, oh my gosh, I’ve been coding for a couple of hours or something, and I haven’t run the tests. So the stress is that might be too wasted hours because I want to have the ability to just roll back.
00:12:04 Right. And say, and your lizard brain knows you’re on the yes, definitely.
00:12:10 And nobody wants to waste time.
00:12:12 It’s a little bit like a branch that goes on too long and you’re like, this is never going to merge. I did too much. Yeah.
00:12:19 If everything breaks, you have the choice, do I roll back and start over or do I try to fix it and debug it and stuff? And so those are the choices. Those are either debugging or rolling back or essentially throwing away 2 hours of work. Of course it isn’t throwing it away because you learned a whole bunch in that 2 hours, and you can probably redo it in about 10 minutes.
00:12:44 But it does panic me sometimes. Anyway. I like that. The thing that I use probably the most that is one of the most frequently used command line flags is dash.
00:12:55 But for running the last stop on fail and. Well, I guess a combination of stop on fail and rerun last failures. And the last fail is something that is a cool feature of pytest. But I don’t even really use the flag that much because I’ve got to rerun the failures.
00:13:15 I use that a lot, and that’s a brilliant thing. So there’s a lot of these running the failures or just once you run it, you have the whole tree there. And then I can use that to rerun a particular test or something. I can just right click on it and rerun something. The thing I wish I could do is I wish I could Zoom back out again to like, okay, I reran the last failures. Now I want to go back to all of them. And yeah, that’d be cool.
00:13:42 Come talk to us at PyCon.
00:13:44 And then also because I’m a command line thing, and while I’m asking for stuff, I want sure. So the run controls are not hard to get. So my run windows at the bottom of the screen. I go to the top of the screen and I select my run configuration and I can edit it right there. It’s not terrible, but it would be kind of cool if there’s a lot of space on the top of the run window. Maybe we could put the flags right there so I can edit them or change them or see what they are.
00:14:15 All right.
00:14:16 Yeah. Because there are too many people. Unfortunately, you don’t know that those flags are actually flaggable and you make a good point about how to surface it. That would be a pretty cool way to do that.
00:14:29 And it’s different than other things.
00:14:31 I got to say. That was one of my things with Visual Studio code is the flags are hard to get to and I don’t even know if you can set them anywhere.
00:14:43 The last time I checked, they were recommending putting in your test and code.
00:14:48 I was going to say Pi test.
00:14:50 And yes, that’s what you should do for stuff you’re using all the time. But we have working on a project that any file is checked in, and it’s something that we’re all using and sharing with the team. So if I want to add some extra flags to Zoom in on a marker or use a keyword or something.
00:15:11 Everybody is going to get it.
00:15:12 Yeah. And then I have to remember to not check that stuff in, because then my larger test suite might not run the right stuff.
00:15:23 That’s not good. Yeah. What else?
00:15:24 Just some of the basics. Like a test fails, how quickly can I get to it? And IDs in general are pretty good at putting it somewhere in the visual UI to double click on something and jump straight to the line that failed.
00:15:44 Yeah, and I can’t do that with the Pi test output directly in another window. But watching the test failures, I can either. So there’s lots of ways to get to it, right? I can rerun the failure. I can double click on the little tree.
00:15:59 The test and code.
00:16:02 Or there’s links within the output window as well. So all sorts of places you can get to it.
00:16:09 That seems like a simple thing. But then again, for civilians, people who aren’t yet committed to TDD, that can be a very important point. It speeds up your resolution of test failures and reduces the friction and keeps you in the flow and all those kinds of things. And one that I use a lot. I wonder if you do or not, but I fiddle with when I’m trying to zero in on some concept and unload my brain. I don’t want to run all my tests, and we have different ways, like Gutter icons or context menus to run just this test. All the tests, if you’ve got a test class or in the module or in a directory or a double parent directory or whatever, and I wind up zooming in and sitting there while I’m focusing on something.
00:16:59 Yeah, definitely. I do too. Sometimes, if it’s the two levels I’m mostly at, I guess I’m all over the place is either just rerunning one function or one parameterization of a function.
00:17:11 Okay.
00:17:13 That’s actually kind of hard to get to. I have to be honest. I got to run the parameterizations first.
00:17:19 Okay. That’s what you were talking about. Yeah. Okay. Like if you could right click in the tuple and be able to run just this.
00:17:28 Yeah.
00:17:30 It doesn’t even need to be the whole thing, the whole suite.
00:17:34 I’m looking at my test right now or some tests. And in the gutter, there’s a little green run button with run test. When I click on it, I get a whole bunch of options. I can debug it, I can run with coverage. I love all of that. I can profile it. These are so easy. And we’ll actually talk about some of these a little later, but. Yeah, that’s cool. It’s kind of cool to be able to say run an individual parameterization or something.
00:18:00 I don’t know how to say that, but because sometimes we’ve got dozens, we’ve got a lot and they take a while.
00:18:08 You had an episode where there was a discussion about the word parameter drive, right?
00:18:12 I don’t know.
00:18:13 Have we? Probably, yeah.
00:18:16 So how do you say it?
00:18:17 Parameterize?
00:18:18 Do you say it the British way, then? Parameterize? Parameterize.
00:18:21 I try to leave out the tur.
00:18:23 I just say I do as well, but I didn’t before Pi test.
00:18:29 I never saw the damn word before.
00:18:33 So I had to train myself and I do it so that I try to spell it correctly and I’m annoyed that everything I write in will tell me that it’s spelled wrong.
00:18:43 Well, we ought to complete it for you, baby.
00:18:45 There you go. Yeah. Okay. So running I Zoom in and they’ll Zoom out. It’s really whatever I’m working on. If I’m working on a localized fixture, I’m going to be looking at maybe running a few tests within a module. Or you can Zoom out to just run a file or run a directory, because with a big test suite, you’re going to probably split them up into directories subsystems or something. So running a subdirectory is good and then running the whole thing. Zooming in and out. Whatever I’m working on and that ability to Zoom in and out is a cool thing about pipe test, but it’s also a cool thing. That pipe arm is almost just as easy. That’s pretty cool. I like it. While we’re on parameterizing, it would be really cool if I could put spaces back in my ID identifiers.
00:19:33 You know, I know some people.
00:19:35 Yeah. It’s an existing issue that somebody’s working on.
00:19:38 I think Ilia does just amazing.
00:19:42 Just in the past year and a half, some of the things that have appeared in the pytest support of parameterize, but also fixtures. And we’ll talk a little bit more about that later. Just things I wouldn’t even thought to ask that I use constantly and have made things so much easier for testing.
00:20:00 Yeah. So the space thing for me isn’t that I’m putting spaces because we have a style guy that worked to not put spaces and IDs in Identifiers. And by Identifiers, there’s pytest as a way to. I know you know this. I’m telling everybody else to be able to identify an individual, add an extra string to identify a parameterization so that string can be any string you want it to be for Pi test, but for Pi charm, the string cannot have a space in it, because if it does, you can run it just fine. It reports just fine. You just can’t right click on it to rerun it. If it fails, I want to rerun it.
00:20:43 It’s an important thing. And so this rerunning stuff is important enough to me that that’s why it’s part of our style guide, and I’ve convinced. But this zooming in and out is interesting that you bring it up because I thought, well, of course, everybody would figure it out. First time I tried Pi Charm and running tests with it, I’m trying all the different ways. I know there are different ways to run it. I’ll try everything, but I’ve worked with people that figure out one way, and then that’s what they use all the time. The way that I’ve seen is for people that’s very fairly common is to go up and to edit configurations and create a new configuration and run it that way. And that’s so much clicking and typing. Why are you doing that? Well, because I want to run it. Oh, man.
00:21:30 You’re making kind of a semi permanent commitment.
00:21:32 Well, yeah. And also one of the reasons was because people didn’t know about the global default configuration.
00:21:42 Yeah.
00:21:43 And I can’t blame them because it’s hidden.
00:21:45 Well, it’s my fault. I should be teaching that.
00:21:47 It shouldn’t be so hard to get to just saying that’s right next. I mean, maybe it could be like right next to edit configurations. Edit global configuration. I don’t know. But since we’re testing against instruments, you always have to tell pytest via command line or our system via command line which instrument we’re talking to. If you haven’t put your instrument name in the default, that makes sense. It won’t show up if you just right click on something. Even with that, it’s still faster to right click on it, run it, have it fail, and then edit that configuration for me.
00:22:25 But anyway, I wanted to go on to one that’s a little bit related to the thing we led with. You know, we talked about visual debugging to put a pretty face on debugging to make it more accessible to beginners visual testing. Same thing.
00:22:41 People beginners kind of get intimidated by testing. Let’s put a visual face on it. Coverage.
00:22:47 Yes.
00:22:47 Another thing. Which kind of gets put in the pro slot. I’m not a pro. I shouldn’t even have to think about this. But if you could put a visual face on coverage and make it feel more normal part of my workflow, then maybe more people would do coverage, and that would be a good thing for humanity to get kind of civilians to do coverage first. Do you buy that theory?
00:23:15 Yeah, I definitely think coverage should be understood very early on.
00:23:20 Again, the I and IDE, we bring it into something that feels like everything else. You mentioned the button, you click on it and you can run a particular test.
00:23:33 You can run it under debugger, but you can run it under the Coverage tool. And we will generate several different UIs without you having to understand what it all means and give coverage reports, including gutter decorations, showing you things that weren’t covered.
00:23:50 I personally find that I wouldn’t be doing coverage if it wasn’t visual coverage integrated in the tool.
00:23:58 Really?
00:23:58 Do you think there are more people like me? Probably as you followed up your book and talking to people and stuff like that. Is coverage an edge topic still?
00:24:08 I don’t think so. I never really thought about or I don’t think I’ve talked with people about how they’re using, how they’re running coverage.
00:24:15 One of the things that comes up a lot is whether or not you should care about 100%.
00:24:20 Sure.
00:24:20 And it’s valid. I used to be on the side of it’s an interesting number, and actually less than 100% should tell you which code you can delete, not which tests you’re missing, because you shouldn’t have any missing tests. But I dialed that back. I understand working with more and more projects.
00:24:42 One of the things I like to see, like, for instance, I’m kind of picky, and when I’m working on open source projects, I will reluctantly work on a project if it doesn’t already have 100% coverage because I want to be able to know when I add my code that I can use coverage to make sure I’ve written enough. That isn’t really true, but it’s an indicator that I’ve at least written some tests to cover the code that I’ve written, and coverage is useful that way. But I can’t really use that if the baseline is less than 100% for non open source projects, I think the line is different because it’s not somebody jumping into a project and jumping out, but it’s a group of people living with a project for a long time. Lower numbers are fine, but I do like the idea of being able to say, well, our coverage numbers right now are like 89%. So if our coverage goes below 89, the test suite will fail. And those are cool. Yeah.
00:25:43 What you’re talking about that idea of some of these things, test coverage, etc. If you’re working in open source and you want casual contributors, some of these things will give more confidence to the casual contributor and give more confidence to you in accepting the casual contributor. Alex Gainer this week wrote an article about scaling software development.
00:26:06 He was going into the hot topic about strongly Typed memory, safe languages, and things like that.
00:26:16 I’m a competency developer. I don’t need rust.
00:26:20 I don’t need a language that helps me on memory issues. Well, what if you’ve got any talks about operating systems or browsers with tens of millions of lines of code and thousands of contributors and someone walks up and wants to give you some code you’re going to trust, you’re going to trust that they know all the implications in those 10 million lines of code. So some of these things, some of these practices that we would like to put in the hands of either beginner programmers or in your case, a casual contributor to a project. If testing and coverage become more accessible and easier to do, then people will do them and it will help scale development.
00:27:02 Yeah. I’m going to pick apart even my own statement, the needs of an open source project where, like you said, it’s tons of code. You want to be able to have somebody jump in and from both sides. You want to be able to have somebody jump in, have confidence that they can make changes and they’re not going to break the world. And then you also want to have confidence that their code isn’t going to break the world. Actually, if it completely breaks everything, that’s not a problem. You’ll catch that right away. It’s the thing that when they clean up some code and throw away a workaround that was important for some corner case and broke 10% of the users. But your test suite doesn’t cover that’s one of the problems. Okay, so that’s true. But the individual contributor of an internal company project, not an open source project. You still have issues where somebody walks up, people leave there’s attrition people retire, people come in. Somebody new to a project is going to ramp up a lot easier if we have the same standards as an open source project.
00:28:03 How many companies out there have executives saying, oh my gosh, if this person, if she leaves, we’re screwed.
00:28:09 Yeah.
00:28:10 It’s the one developer who has everything.
00:28:13 If you like your job, you want that to be true about yourself. If you don’t like your job, you don’t want that to be true. I like having a coverage. I think actually integrating the coverage, even having more options with coverage would be cool. Is getting better with Pi charm. When I first interacted with it, the gutter colors. I don’t know if I just got a better laptop, which I did, but the gutter colors on some monitors are a little hard to read.
00:28:39 Time of contrast.
00:28:40 Maybe it’s a contrast thing, or maybe there are people with red, green trying to blindness. So being able to optimize those or just old eyes, it’d be cool to be able to make that thicker.
00:28:52 So I often run coverage outside of PyCharm because I personally like the red and the green to cover my entire screen.
00:29:03 Like, if you run coverage, the HTML appetite for coverage will show you got it. All of it. So having that option would be cool to be able to do that.
00:29:13 Okay, that’s a good point, because the other option we have is that kind of tree that you drill down. It shows you 87% files you keep drilling down, but having another visual way that shows you everything. Would you want that to be kind of clickable, too, to go, what would you click on if you had, like the HTML report and it said this sub package was 80% coverage and you click on that and you keep going down until you get to an actual milestone?
00:29:43 Yes, definitely. Because I don’t even consider that that might be an option within Pi PyCharm. I just use it outside of you have the tree though, right?
00:29:51 Yeah, the tree does that same thing that I was just talking about. But maybe you would want something that was a little more visual than the tree.
00:30:01 This would be definitely something that would be cool for a contrast demo instead of talking about it on a podcast. Yeah.
00:30:11 But that’s okay.
00:30:12 So in general, I’m interested in the theme of making some of these quote unquote professional topics and programming more accessible to beginners and civilians, whether it’s debugging, testing coverage, whatever.
00:30:28 Yeah. And then. So while we’re on, I guess the same one of the other tools that you guys have integrated really well is the profiling.
00:30:35 Right?
00:30:35 I should use that more than I do, honestly.
00:30:37 Frankly, I use it in combination with testing.
00:30:40 I right click and I run my test suites and it says, Would you like to run another profiler? Why sure, that’s exactly what I’d like to do.
00:30:49 But I think we’re definitely moving in the right direction with making it easier. So you brought up I was looking at our list of things that we possibly could talk about, the flipping back and forth between tests and code.
00:31:04 I don’t really do that because I don’t think of a function as a unit. I think of a behavior or a feature as a unit. So I’m not a London School of TDD person. So do you use that jumping back and forth between code and a test for function 100? Really? Okay.
00:31:28 Yeah, certainly. When I’m creating whatever it is that I’m working on is the thing I put on the left. And whenever I’m testing, I put the test code on the right and then I put the test output at the bottom and that kind of left, right, bottom thing is the way to hack my brain into TDD mode. When I see it, I kind of get a warm feeling that I’m in. I’m entering managed code zone and I’m working correctly and staying in the flow and all that kind of stuff. So just the visualization of it helps me.
00:32:03 So if I look at it like I just have a function, if I got a function somewhere and I right click on it, I can go to generate and I can generate a test and luckily it asks me where I want to put it. It doesn’t just say it’s going to go anywhere. This is a leftover for something else, right?
00:32:21 No, actually, we don’t promote it as much as we should. But you can be sitting in a code and say to PyCharm navigate to the test for this code and pyramid will say, yes, found it, or it will say didn’t find it. Do you want us to generate a skeleton test for you? And that’s what you’re talking about.
00:32:45 Now, which test is for it? Is there a naming thing are you supposed to name a test the same as the function with prefix?
00:32:53 You go into Slack and you ask Ilia, what are the magic rules?
00:32:57 No, the name. It’s all about the name.
00:32:59 Okay, so if I’ve got like Foo the test where it would be test.
00:33:02 Foo and module and stuff like that. It’s actually a place where organizing into classes helps, so you don’t have to keep repeating that junk over and over and over. You can just jump to a class for that unit, and then the tests in that class are all the variations that you’re trying to do on that.
00:33:26 Yeah. Okay.
00:33:30 You’re never going to do that.
00:33:33 You’re going to come.
00:33:33 No, that’s fine.
00:33:35 I like different styles.
00:33:37 I am from the behavior side, so my behaviors are not they are implemented by my functions, but they aren’t my functions.
00:33:47 Also, certainly the pytest community is embraced. Organize your tests as functions and modules, and if you need to group them, group them in a module. Yeah, if you need another grouping, group those modules into a directory.
00:33:59 In my personal projects, I have no problem with throwing testing classes. I put tests within classes.
00:34:06 Mostly if I need to group a set of tests with their fixtures, I don’t know. Mostly if it’s just easier for some reason to group things together. Sometimes that’s an easier way to call things. You can just say I want to run all these tests together. So throwing them in a class is trivial, so why not? So that’s good. But then I have to put self in front of everything, self as a parameter. Luckily, within Python it just does that for you. The other thing that I’m having trouble being convinced of, and I’m probably never going to get there is the BDD cucumber stuff. Yeah. So Automation Panda is a proponent of BDD, and I actually really love the concept of behavior driven development and the notion of thinking in terms of behaviors and in thinking of terms of given when then that’s a model that I use. I just don’t use the syntax.
00:35:02 But Pytorm does support the BDD, right?
00:35:05 Indeed, in a pretty cool way that whether it’s Cucumber or Python, BDD what you’re describing. I think the thing that you bounce off of is there’s going to be this semantic abstraction between me and my tests. There’s going to be these scenario things that you write in an englishlike language with special symbols, and the structure of that kind of outline thing matches the structure of the tests that get implemented from it.
00:35:35 And so Pi Charm does a pretty interesting job of helping that structure that maps between the scenarios you’re writing and the actual tests and navigating back and forth between them and refactor, renaming, and giving you squiggly when things are out of sync between those two artifacts.
00:35:53 Really? Okay, so that’s kind of cool.
00:35:55 Yeah, it’s been around for quite a while, too. It wouldn’t surprise me if there might be a few bugs on it. It doesn’t get as much usage as the normal stuff, but the idea is pretty sound. Let the IDE almost put a UI on the BDD side.
00:36:10 Yeah, I’m not opposed to the cucumber syntax.
00:36:15 It isn’t going to help me at all. So anything for me. I also think it’s sort of one of the ideas was it’s human readable. Therefore, stakeholders and managers and stuff can read it and write it, and it’ll be easier. But whenever I’ve seen it in practice, it’s still a bunch of test engineers writing it and reading it.
00:36:38 It’s one of those things that should be true.
00:36:39 But yeah. So at that point, if you’ve got the same test engineers reading and writing it and writing the scaffolding underneath, why not just write tests? It does help you think about the structure a little bit. Now, one of the things on my to do list is to do more weblike stuff like Flask and Django and Pyramid and things like that. When I’m testing something like that. It’s a different world than just testing pure Python stuff, because I’ve got probably JavaScript stuff to think about. So what’s the story with that and PyCharm and testing and stuff?
00:37:21 Is it a similar sort of thing? Can I run my JavaScript tests and other stuff that way?
00:37:28 Pycharm Professional bundlestorm or ID for web development.
00:37:33 Okay.
00:37:33 And the Webstorm team, my goodness, they’re great.
00:37:37 They’re keeping up with the insanity of JavaScript, so you don’t have to. And the neat part is Pi Charms, Pi test support unit test support, et cetera, nos, whatever. It all shares this machinery from the IntelliJ Idea platform, where the test running is done by a component, and all of these different things feed information to that component. And so that component is shared on the JavaScript side as well. So whether it’s Karma or which is the Angular test runner, or whether it’s Jest, which is kind of the React test runner, or Mocha, any of the support of test runners, we’ll use that back in. But more importantly, from your perspective, Brian, it’s the same UI. It will look exactly familiar, the same shortcuts. You will think you’re sitting in a Python test when you’re running.
00:38:29 Wonderful. Okay, great.
00:38:30 You can walk up to a single Just test and right click on it and run just this.
00:38:36 Okay.
00:38:37 How is Just?
00:38:40 I haven’t learned it yet. Are you comfortable with it?
00:38:43 Crazy comfortable.
00:38:45 I’m saying that it’s a look of remorse.
00:38:49 Okay.
00:38:49 I do a course on React and Type Script and TDD where I teach people doing React, but I get them to throw away their browser and learn all these concepts using testing.
00:39:03 Oh, nice.
00:39:03 Cool.
00:39:04 And testing the output so that you’re not always switching over to the browser and hitting Reload and trying to remember where you were.
00:39:10 I’ll have to check that out. One of the things we sort of skipped because it was an issue and then it wasn’t an issue. And everybody’s just happy with it. Is the fixture support within Pi PyCharm is excellent. So people that say the only thing that I’ve heard before from people, from why they don’t like Pi test is that they’ve probably faced a test suite that uses a lot of fixtures and they don’t really know how to find where they are. Now. Pipe Test itself has ways to display where the fixtures live. There’s command line flags. And if you want to know more about those, I’ve got a book for you. But there’s PyCharm added that a long time ago, and you could just right click on a fixture. It takes you to the code, the navigation is built in, the typing is built in the autocomplete. It just sort of works. Just like another function now. And thank you. I guess whoever fixed that. Thank you.
00:40:11 That was also Ilia. And you and I have talked about this before. You know what, Brian? Things are hard. You know, it’s the hardest stuff. The simple stuff. The simple stuff is the hardest stuff. And people file the tickets like, oh, could you do this? It’s so simple. And that’s like the red flag where you have no idea how much iceberg is under the water on that thing. And the fixture stuff is a really powerful part of Pi test.
00:40:36 And it can get pretty fiddly. And he did a really good job of thinking rethinking, re, rethinking shipping it, finding some of the breakage, fixing that, finding the next kinds of things. So it’s pretty mature. And I can’t I mean, I really rely on that. It’s so comforting to get something from a fixture and just being able to command Hover on it and see the type inference and say, oh, yeah, that’s what I got.
00:41:05 Yeah.
00:41:06 It’s so great. I’m telling people all the time how to use it, and then they forget that it’s a problem because it just works.
00:41:12 Yeah. And even on some things where you’re like, okay, smartypants PyCharm, I’ll bet you can’t. Oh, wait, you can do that, too. Like the fixture fixtures where you can ask for something and there’s kind of the chain of its dependencies might come from something else and stuff like that. And you’re like, how did you follow that path to get all the right things?
00:41:34 Yeah, I really appreciate it. And I love it. And most of the things we talked about today are just in the free version as well.
00:41:42 Yeah. The things that aren’t would be the coverage and profiling stuff. And we haven’t talked about multi processing yet, but the multiprocessing part is in professional.
00:41:51 Oh, yeah. Let’s just bring it up quickly. What is the multiprocessing thing?
00:41:54 Sometimes you want to speed up your tests by running them across multiple cores.
00:42:01 It’s not even a plugin anymore. Right. It’s in pytest.
00:42:03 Right. Well, is it extist? Is that what you’re using.
00:42:07 No, you’re right. It is still a plug in, but we put a friendly UI on that thing in the run config. And then again, another fiddly thing in the visualization of your tests as they’re running and how much their lapse time is, you’re going to want to let the developer know. Well, she’s got three quarters that are being used on this, and this one took that amount of time, and that one took the other amount of time, and the aggregate amount of time was this, and this is the one that failed and all that.
00:42:36 We put a pretty good UI.
00:42:38 I didn’t even know it was there. So if I look at my run configuration up in the right hand corner, there’s an allow parallel run checkbox. Is that what you’re talking about?
00:42:46 Yes. You and I were talking at the top about stuff, then run configurations. It’s not always visible.
00:42:52 Okay, well, cool. I’ll have to try that out more. This probably isn’t easy. Then when it’s filling out the tree and things are running in parallel, does it populate the tree in parallel then? Yeah.
00:43:06 As things come in?
00:43:07 Yeah. Okay, that’s neat. I’ll have to try that out.
00:43:10 Not while we’re talking. I say as I’m trying to do it right now, but I’m excited about that.
00:43:15 Let me close out one point while you’re clicking.
00:43:19 Let me close out one point on the fixture stuff. It’s also true on the parameterize is refactor rename. What you refactor rename to rename something and find everything in the project that uses it and gives it the new symbol name. So if you want to change the name of a fixture, which is a pretty common thing to do as you’re writing your tests and as you’re abstracting things out, you learn more about the problem. But then you’re like, I don’t know, all the tests that use it and all the other fixtures that might depend on it. Wouldn’t it be nice if there was a tool that knew where all the bodies were buried and fixed everything for me? So being able to walk up to a fixture, either usage of a fixture or definition of a fixture and changing its name and having everything get reflected is a pretty cool thing.
00:44:04 Yeah.
00:44:05 And then if you change your mind, you hit undo and it rolls back all of those changes.
00:44:09 That’s cool. Well, so Pipe test also allows you to have a different name for a fixture than is the name of the function.
00:44:17 Does pyramid deal with that?
00:44:18 Okay, yeah, it does.
00:44:20 That must have been annoying for any implementation.
00:44:24 The simplest thing.
00:44:26 Oh, yeah, we forgot about name fixtures.
00:44:28 I want to tell everybody PyCharm does sponsor test and code.
00:44:33 I’m not using PyCharm because they are a sponsor of the show. They’re a sponsor because I asked for it after using it for a long time.
00:44:41 In a larger sense, this discussion, maybe it is a lot about PyCharm. But it is about Ides.
00:44:48 And how can your tool do more for you rather than less? And there are people who want less, and there are people who want very lean environments, and I get that. But there are also people who want assistance and people who want if I spend a little time learning this, I’ll get ten X productivity game. Okay.
00:45:10 And I’m just going to be honest about it there’s. Vs Code is a pretty cool editor, and I really like a lot of the people on the team. There’s going to be people that come to testing that are already embedded in the Vs Code world or another editor. And so having the test capabilities of other Editors, I’d like them to be up there with as good as you guys.
00:45:36 What you’re saying is true is it’s not just about PyCharm, it’s about IDs.
00:45:40 Indeed. And it would be great to see new thinking, new ways to make these practices more accessible.
00:45:49 And I’ve known a lot of those guys for like two decades.
00:45:53 I consider them good friends, exceptionally smart people, and they’re going to do some brilliant things that move the state of the art forward. So this a lot of the talk that I’m about is Ides in general as a way to give a visual palette to put things within reach that previously were out of reach.
00:46:16 Yeah. And bringing it, if we make it all easier, brings more new people to testing sooner in their programming.
00:46:24 I’ve worked with people that do training courses and stuff, and it’s been helpful to have testing as early as possible. And one of the things I like is this model of not teaching people to write tests right away but having tests in place to help them. They’re like the guide, one of those things on the gutter walls. And when you go bowling, they’re like those to keep people on the right track while they’re learning how to code. Once they’ve kind of figured out their code, they can look at the implementation of the test and go, hey, wait a second. These things really help me a lot. Now, how do I write these?
00:47:03 Right? You’re giving them confidence, right?
00:47:06 Yes.
00:47:07 It’s cool. Hey, we’re running long. Like everybody that I talked to, I could probably talk about this testing for hours, but we should wrap it up. Any closing thoughts?
00:47:17 You’re going to be doing any podcasts at PyCon?
00:47:19 Yeah. Well, I’m going to do lots of interviews. And last year I did the thing where I just walked around on opening night and just talked with people. I want to do at least one of those. I might do more because it was just so fun. So, yeah, we’ll see.
00:47:33 So I’ll send out an encouragement. If you’re going to Python, look for Brian. If he’s got his microphone, chase him down, get your voice heard and share some ideas on this topic of bringing testing to the masses. The world of Python is getting huge influx of new programmers. Let’s find ways to get them to work with confidence from the beginning.
00:47:56 Yes, definitely. Thanks. Good place to leave it. Thanks a lot.
00:48:00 Thank you very much Brian. Thank you for writing that book. I know you probably hated it wished that you would have never said that you were going to do it until it was done and then you were thrilled but it made a big difference with me.
00:48:12 I’m glad you liked it. I loved writing it so yeah, no negatives there so thanks.
00:48:20 Thank you Paul for that interview.
00:48:22 As you can tell.
00:48:23 We recorded that interview before the news that Python was not going to happen as I was doing the final edits to this episode I had just found out that I knew that Python was not happening but I had just got the email that it really was not postponed but canceled and I considered cutting that discussion out of the end but I decided to leave it in.
00:48:47 It does make me sad but the Python community has been amazing and I cherish the memories of Python’s past and meeting so many amazing people and building genuine friendships. I know we will get back to in person conferences in the future but in the meantime we have to do what we can to keep this community alive. I hope that you will stay safe and practice in real life social distancing but don’t practice online social distancing. We need each other. I need you definitely. I am working from home now so I have saved commute time and have more time spots open for recording so I’ll be here if you want to talk reach out, go to test and code.com there’s a menu item for be a guest, check that out and get in touch with me and if you’re just missing the water cooler or break room or Kombucha Keg conversations, if you’re at a hipster company check out the slack channel and join the community there. That’s all for now. Now go out and test something.