We talk with Adam Johnson about his new book, “Boost Your Django DX”.
Developer experience includes tools and practices to make developers more effective and efficient, and just plain make software development more fun and satisfying. One of the things I love about this book is that it’s not just for Django devs.
I’d guess that about half the book is about topics that all Python developers would find useful, from virtual environments to linters to testing. But of course, also tons of tips and tools for working with Django.
Transcript for episode 181 of the Test & Code Podcast
This transcript starts as an auto generated transcript.
PRs welcome if you want to help fix any errors.
00:00:01 In this episode, we talk to Adam Johnson about his new book, Boost Your Django. Dx developer Experience includes tools and practices to make developers more effective and efficient and just plain make development more fun and satisfy is fine. One of the things I love about this book is that it’s not just for Django devs. I guess about half the book is about topics that all Python developers would find useful, from virtual environments to Linters to testing, but of course also tons of tips and tools for working with Django.
00:00:32 This episode is sponsored by Rollbar. Rollbar is the leading platform that enables developers to proactively, discover and resolve issues in their code, allowing them to work on continuous code improvement throughout the software development lifecycle. Rollbar has plans for all situations, from free to large enterprise. With Robar, developers deploy better software faster and can quickly recover from critical errors as they happen. Learn More at rollbar.com Thanks, Rollbar.
00:01:14 Welcome to Test and Code.
00:01:25 Welcome, Adam Johnson. It’s been a couple of years since we’ve talked. What have you been up to since then?
00:01:29 Yeah, I’ve been doing some client work, surviving a pandemic, and last year I wrote another book, Boost to Django.
00:01:39 Yes, client work. When you say client work, what does that mean?
00:01:45 I kind of work as a consultant on Django projects and jump around between a few handling things like improving testing developer experience as this kind of broad umbrella term, things that can help and doing things that I might have a bit more leverage over, like upgrading Django. Upgrading Python.
00:02:08 That’s interesting.
00:02:09 So on a lot of the projects, it’s probably a mix. But do you develop new sites with Django for people, or is it mostly coming in and helping with existing Django?
00:02:22 Mostly people with existing projects, perhaps a few years old, who either want a short audit. So I’ll do like even a few days or a week just looking through for the most obvious issues that can crop up in projects that have lasted that long. Or I often work on a part time basis with them over a longer period.
00:02:44 That actually sounds fine.
00:02:46 Yeah, definitely. I like having a bit more of a feel for what people struggle with with the framework. I think it helps inform me building more third party packages or helping maintain Dangle Call.
00:03:01 If I was working on a single project, I’m sure I could get that kind of like, tolerance, right? Something works a certain way and everyone is like tolerant of it working that way and just don’t touch that thing again. Whereas when you see the same, oh, like two or three different teams struggle with the way static assets work or something that informs me to make open source contributions.
00:03:27 Nice. Okay, I could go down this rabbit hole a lot, but I want to switch gears. So just to remind people, if they weren’t back here listening in 2020, you are on episode 128 and we talked about. I test randomly and how this should be order independent, very important.
00:03:46 Now I’m telling people to always use there’s a couple of different plugins to do randomized stuff, and randomly is definitely one of the better ones.
00:03:56 Whoever works for you, I guess.
00:03:59 And then Episode 135, we talked about your first book Speeding up Django Tests or speaking. I forget what it was called.
00:04:07 Speed up your January Test.
00:04:09 Yeah. Okay.
00:04:11 We talked about the contents of that and the book. And now your second book is Boost your Django DX. Dx. Is developer experience, right?
00:04:20 Yes.
00:04:21 Okay.
00:04:27 It should have been obvious to me.
00:04:30 I kind of put it together because of UX’s user experience. So I figured the developer. But I love the idea of focusing on developer experience. So what does developer experience mean to you?
00:04:43 Yeah, it’s definitely a very broad term. That could mean a lot of things. From how comfortable is your chair? Is your screen at the right height or are your colleagues pleasant to work with? But I think for most people it’s like the experience of writing or debugging code. And is it easy to make changes? Is it easy to avoid common pitfalls and bugs? And is it fast and pleasant to get your changes from the first time you like type a key to production?
00:05:19 And there’s a lot here, not just in your book, there’s a lot that you’re covering, but there’s a lot to developer experience that I don’t think.
00:05:28 One of the things I like about focusing on this is because I don’t think a lot of management chains think about this too much. They just hire coders and just have them code. And maybe hopefully that’s not the case.
00:05:45 I think this makes a big difference.
00:05:48 It’s hard to put a number on it. But do you have like a gauge for if people don’t focus on it at all versus spending some time focusing on improving tools and stuff?
00:06:00 Yeah, it’s one of those things that’s very hard to put a number on you’re right. That most management chains probably don’t think of it, especially the further away you are from the call face of coding. I think that it’s something of a responsibility for us as technical people to be thinking about our own kind of working environment. We have the most control over it. To put a number on it. I don’t know. I mean, there’s that myth of the ten X developer, and perhaps the ten X developer isn’t ten times smarter but makes ten times fewer mistakes because they have all the tools and stuff in place.
00:06:35 Maybe.
00:06:38 I’m sorry.
00:06:39 No, I was thinking what number would I put on it? And I don’t know if a bit ten X, but that’s not out of the realm. I think watching people that are comfortable with their tools work quickly, like not even around Django. But I was remembering the first time that I the first time I noticed that my tool game was weak was in grad school and I was using nothing against Emax people, but I was using Emacs at the time and I had one of my office mates had an office because I was a teaching assistant, but I shared it with two people, but one of them, she was an avid Vem user and with them with Tags. And it wasn’t like Windows Vim either. She was command line vim all the way or Bi at the time with Tags and being able to jump around through the code. And she was like thinking through the code flow without having to care what files she was in. And that was the first time I saw that sort of magic. And I’m like, oh my God, she really is working probably 20 times faster than I do because I just take a look, try to find the file in the header file. Guess where it is. Look there. It’s not there, look somewhere else where she was just going, popping right to the tag. So that’s just one section and that’s one that we actually hopefully don’t have to teach developers to go out and pay attention to because I think they know that they should understand their editor.
00:08:26 It’s not even VI versus anything else.
00:08:29 It should be a coding editor and all the good coding Editors can have tricks like this. But I guess what sort of things do you cover in the book or what do you think is important?
00:08:42 Yeah, well, Editors are not what I cover in the book. Definitely a key area, but there’s just too many of them. There’s individual guides for every single editor out there that you should definitely pick up and find all the relevant Python and whatever plugins that make the workflow faster. The kind of things that I cover are the more universal that probably everyone or most developers can make use of.
00:09:11 So that’s things like documentation tools make it faster to get to the right docs code quality tools, which is like Umbrella for Linters and formatters and other kinds of checkers, including building your own.
00:09:28 And then that kind of stuff is more general Python. And then the other half of the book is more Django specific.
00:09:37 Like the best way to write your Django settings files and common pitfalls that you might encounter there.
00:09:44 Best way to populate your database in development so you can always have a good working set of data that makes it very fast to get started.
00:09:54 That’s one of the things I did notice was the like almost half the book seemed like everybody should read this. Whether you’re using Django or not.
00:10:05 If you’re using Python, these are good things like really understand how virtual environments and dependencies will go together.
00:10:14 This seems obvious to me, but it isn’t obvious to a lot of people.
00:10:19 I remember a team where I had to I did have to start convincing people that virtual environments were a good thing when you first start out. Maybe not everybody, but some people’s introduction to Python is similar to their introduction to Perl, whereas I think Pearl has some of the similar problems. But most of my Perl writing was not applications or big things with lots of dependencies. It was just straight Pearl stuff. It was like a search or a play script and things like that. So you just stick it somewhere and I don’t have to bundle it with dependencies or anything like that.
00:10:56 Of course, later when I started doing advanced things with Pearl, we would get into packages as well. But the Python now everywhere, even a little build script. I tell everybody we’ve got to use virtual environments.
00:11:12 It’s not optional anymore, I don’t think, and that’s just scratching the surface.
00:11:23 One of the second topics you get into that I was surprised by is I Python.
00:11:28 Does I Python really like do you use that much with Django development?
00:11:34 Oh yeah.
00:11:36 Django has a shell command that you end up using quite frequently. It sets up Django and opens a Python shell, and if I Python is installed, it will use IPython in preference to the normal Python shell I found it invaluable over the years. In fact, I was lucky enough to be shown it on one of my first few weeks using Django, so I never really used the Python shell. And then when I started finding I needed to use it on other projects because Python wasn’t there. It’s like hey, this is so much harder. I don’t have like a real history.
00:12:11 It’s not easy to paste multiple lines of code. The order complete is worse. These things add up.
00:12:17 Yeah.
00:12:19 So I had kind of knew the IPython shell was around, but sort of ignored it for a long time. But I was reading a data science book which I can’t remember the name of right now, but one of the during part of the introduction is a description and brief fast forward introduction to Python. I’m like, why am I not using this all the time? This is great. There’s a lot of cool features jumping ahead a bit.
00:12:56 One of the things when developing a web application, Django or anything else is understanding a development server, so I was glad that you covered that.
00:13:08 Is Django different than somebody else that you use development server differently?
00:13:17 I don’t know if it’s that different. It does the same kind of Http, but it has some extra features that are pretty useful actually that maybe you would miss because most people would just run the development server and never read the docs, so I hope to expose those.
00:13:36 The key one there is Watchman, so I guess most of our service work with a fairly inefficient file watching system where it’s universal, but it takes a lot of CPU. It’ll run in a loop and check every single file hasn’t been changed. And Django does this every second by default. So once your project has grown, especially by installing a few packages, which could be a few thousand files, it can take a notable amount of CPU, and then it can take a notable amount of time to actually discover its time to reload because it’s just busy all the time checking files.
00:14:13 So Watchmen is something that lets you hook into the kernel and listen for Firewatch notifications from the kernel. And that’s different on every operating system. So we need a separate pool to do it for us. Otherwise we’d have to maintain like six different versions of different OS.
00:14:32 Okay, so that’s something you install extra.
00:14:37 Does that drastically reduce the CPU load then?
00:14:41 Yeah. On even a medium sized project with 100 dependencies, I found it would take 25% of a CPU call to run. And that adds up when you’re using a laptop on battery power as well, because that means the CPU can never scale down to its low frequency and reduce the power consumption. So you can take your eight hour battery to 4 hours or something like that. You install Watch when it becomes 0% because nothing happens until a file actually changes.
00:15:14 That’s cool.
00:15:17 Well, now I’m thinking that trying to think about incorporating Watchmen in a better way to watch for tests to rerun.
00:15:26 I’ll have to look at that. That’s got more uses than just Django then, right?
00:15:31 Definitely, yeah.
00:15:32 Wow.
00:15:33 There’s alternatives to Watch them as well.
00:15:37 Like, Django used to use one that was hooked mostly into Linux. There’s also something called Watchdog and a fork. I think it’s called Watch God.
00:15:46 For whatever reason, Jenga picks Watchman, which is maintained by Facebook.
00:15:53 Watch God. That’s funny.
00:15:56 Good. Nice.
00:15:57 I kind of love the dumb jokes in programming. It’s good having a little bit of focus on Linters as well.
00:16:10 I’m actually surprised. I guess I’m curious about this because I’m surprised frequently entering and talking to different teams how many development teams don’t use any sort of lending as a standard role. I mean, there might be individual developers that use it, but a lot of people, like a lot of groups, don’t have a standardized way. Is that something common?
00:16:34 If you work as a consultant, coming into groups? Is that something you often have to teach people about?
00:16:42 Yeah, it varies. And teams might have tried setting up Linters and they’re like opt in. So if a developer wants to run them locally, but that’s not maintained on CI or something like that.
00:16:58 But twice, actually, in the past couple of months I’ve been looking at a new project. I’ve run Flake with the Flakate bugbear plugin, which is a fantastic one for picking up on extra common issues that tend to be like real sources of bugs. And I found the same bug.
00:17:16 Once you’ve found like a real bug that could affect production, then it’s normally taken as a strong argument, like, oh, we should have had this all along.
00:17:27 So how do you recommend people use things? Let’s just take that like Flake Aid and Flake a bug there. Should it be run locally or should it be run on the CI server or both?
00:17:40 Yes, both. And in your editor as well, if you can get that going, I like to use pre commit.
00:17:48 You’ve had Anthony on the podcast many times. Yeah, we commit made by Anthony Satill. And that’s a good tool for setting up all these hooks, running them when you run get commit or before if you run on the command line or through an editor and then also in CI, it even has a CI service.
00:18:10 I was actually slow to come around to enjoying Black, and I think I’m a convert now. Black is great, and I can’t remember who said it, but most people have. I think a lot of people have this experience with Black and that they may not agree with all the decisions, but they agree that it’s not productive to argue about this stuff anymore.
00:18:41 My particular one was whether or not the default should be single quotes or double quotes. And I had lots of arguments on my favorite I thought why single quotes should be better, but it’s not really a Hill I’m worth dying on. So I’ll let you go.
00:19:01 There’s a universal things that I’ve been focusing on because I’m not a Django developer.
00:19:07 And we did talk about some of the Django server and some of the fun Django things like Django and I Python.
00:19:18 But you do cover a lot of Django specifics in here, I think.
00:19:24 Let’s pick one.
00:19:27 Settings. You have a chapter on settings. Is settings that big of a deal that I have to focus on that or what’s going on there?
00:19:35 Well, yeah, surprisingly, you think Django settings file is simply just a module with constant that you might pull from an environment variable. So it’s just some uppercase things like what is the settings for the database that you want to connect to?
00:19:55 But there’s actually a lot of small things that I guess there’s like a lot of tutorials that people copy them from or something like there’s the tiny anti patterns that add up to setting spires, becoming a bit of a mess or really problematic once you go down the road.
00:20:15 So I hope to provide a template that gives a lot of good guidance all in one place in a much more declarative format than the docs can give you.
00:20:26 Nice.
00:20:27 Okay. I was glad that you this, and I was like thrilled to find out that actually most of the stuff that I’m learning from this book applies to all Python projects, not just Tango.
00:20:43 However, it’s pretty cool you wrote it, and I do notice it shows up a couple of times in here, which is cool.
00:20:52 I’m curious, and I think that’s completely nice and it’s completely valid to cover both, because I think a lot of people are still using the built in testing for Django is unit test based. Right.
00:21:09 But a lot of people are using Pytestjango, I think. Is that the plugin that most people are using, or is there some other way to run PITest and Django together?
00:21:18 I’ve seen simple projects just to roll their own. Like you just need a Pytest fixture to configure Django settings and run the database migrations. But yeah, most projects would need by test Django. I would guess that three quarters of projects, at least those that are being maintained over a year or with someone working full time on them, are using Pi Test at this point.
00:21:44 Really? Three quarters?
00:21:46 That would be my guess, I think.
00:21:50 Yeah. There’s a tipping point. Once you’ve written so many tests with Django’s Test runner, you’re like, well, I’m getting bored of output being so minimal or hard to debug, missing these features from Pytest just over there.
00:22:05 And how about your personal stuff? Do you prefer to be working with Pytest? With Django apps?
00:22:13 I prefer Pytest. Yeah. All my open source projects use Pytest.
00:22:18 Interestingly. One pull request I’m working on right now with someone else from the Django world is an extension to the Django test runner. But I’ve picked Pi test as the test runner. So the Pytest test runner is invoking Django’s test runner to check that it does the right stuff.
00:22:36 That’s awesome.
00:22:40 It’s all test runners all the way down.
00:22:44 Well, cool.
00:22:47 This is pretty quick.
00:22:50 I guess from the outside world it looks quick.
00:22:53 How long were you working on this?
00:22:56 Yeah, I think it took me about three or four months from first commit to publish.
00:23:04 Four months I’m going to go through right now. I wasn’t doing much else at that time. Okay.
00:23:10 I took a break from work due to family reasons last year.
00:23:16 It was all I did, and it’s really the culmination of everything I’ve been working on, trying to block about since I started being a consultant in 2019.
00:23:27 Okay.
00:23:34 I’m glad you put that little caveat that you were doing this most of the time, that makes me feel a little better because I spent like bloody 18 months on my book or more, but I wasn’t counting. But I just know from the dates that’s about how long it was.
00:23:52 And my first guess that I told the publisher was when I proposed a second edition. I said, I think it’ll take somewhere around six months, give or take six months. So I was off by a year, give.
00:24:10 Or be late. Right. Like it’s very hard to be early.
00:24:13 All right. Unless it was already done, I wouldn’t have been early.
00:24:18 Now you self published this. Is that correct?
00:24:22 Yes.
00:24:24 Any final topics you want to cover about before we sign off?
00:24:30 I was thinking before the podcast about how Linters type checkers formatters they all fit against testing, because we often hear that testing is the thing to do to make sure your code is high quality, but there are definitely things that you cannot test for and you need all of the different bits of the puzzle. I think they will cover different gaps.
00:25:01 Yeah. So Lenters and what else did you cover? Renters and what else?
00:25:08 Formatters, type, checkers.
00:25:10 Yeah, and then I would probably add monitoring afterwards.
00:25:16 Oh, definitely.
00:25:17 Yeah.
00:25:18 So you’re not done.
00:25:21 When you push, you’ve got to monitor as well. So one of the things that I like to have in place is to find out what URL somebody was trying to use when they hit a 404 because sometimes it’s surprising. I’m like, I really thought that was there what’s going on and there happens to be like a misspelling or something like that. It’s worth putting a redirect in little things, but that’s not really it. Anyway, this is kind of a neat thing that you put together because now when you’re a consultant, you can say, could you just read this before I show up or is it something that after you come in.
00:26:03 It will only be after you show up.
00:26:05 After you show up. Okay. Well, yeah, because you don’t want them to read it and then not need you. Right.
00:26:12 I think there’s like a division. There’s the people who have the time so they’ll go read a book and the people who don’t have the time so they hire me and then I’m like, well, I’ll make this change and here’s the citation myself.
00:26:31 Nice.
00:26:32 Yeah, that links on to everything else.
00:26:36 It’s convenient. I have to repeat myself. Hopefully.
00:26:40 Well, always a pleasure to talk with you, Adam. I wish you luck with this book and with your contracting work and everything else and we’ll keep in touch.
00:26:50 Thank you very much, Brian. It’s been great to be back.
00:26:57 Thank you, Adam. Always fun to talk with you. Thank you, Robar for sponsoring Robar Enables Developers to proactively, discover and resolve issues in their code so they can work on continuous code improvement throughout the software lifecycle. Learn more at rollbar.com. Thank you, Patreon. Supporters join them at testandcode.com support all of those links are in the show notes at testandcode.com. That’s all for now. Go out and test something.