What is the difference between a unit test, an integration test, and a system test? Mahmoud Hashemi helps me to define these terms, as well as discuss the role of all testing variants in software development.
Transcript for episode 27 of the Test & Code Podcast
This transcript starts as an auto generated transcript.
PRs welcome if you want to help fix any errors.
What is the difference between a unit test, an integration test and a system test?
Those terms mean different things to different people. So instead of answering them myself and giving you just my perspective, I thought it would be fun to invite a guest on the show and help me discuss it. Today we have Mahmud Hashemi to help me answer this question and also catch up on what he’s been doing lately. We also talk about test driven development and a lot of philosophy about testing. Today’s episode is sponsored by Talk Python courses Nerd Lettering and Test and Code Patreon supporters.
The courses at Talk Python are at Training talk. Python. Fm.
The Python for Entrepreneurs course is still in early bird pricing and includes a $50 credit from Digital Ocean and a three month license to Pi PyCharm Professional Edition. He just put up a new course called Consuming Http Services in Python, which looks fun. I’ll have to check that out, and I always recommend that everyone coming from different languages that think they know Python but aren’t quite writing code that looks like Python to take the right Pythonic code like a seasoned developer.
So that’s Training talkpython. Fm. Thanks, Michael.
Do you love Python? Show it with some Python swag. I’ve been sent a beautiful black coffee mug that says Python. It’s good for you from the folks at Nerdlettering. That’s Nerdlettering.com. I’ve also got the same design as a mousepad.
There are a few other designs, including some gorgeous lettering for the pie ladies unite. And there’s a new design they just put up that says Pythonista. I think I’m going to have to get that one, too. Start a collection at work. Nerd lettering is done by Dan and Anja. If you also listen to Python Bytes, you’ve heard us mention a couple of Dan Bathers Python articles. That’s the same Dan. So get some custom made mugs and accessories for Python Easter by Python Easter. Check out Nerd Lettering.com and use promo code test code all one word to get 15% off the last. Thank you. Needs to go to Test and Code Patreonsupporters. That’s at Patreon.com testpodcast at just a buck per month or per episode, you can prod me into making more episodes.
Welcome to Test and Code, a podcast about software development and software testing.
Today I’ve got Mahmud Hashemi Mamu. I think I got that right.
Yeah.
It’s close enough.
Yeah.
And we’ve never talked before, but I kind of feel like I know you already, I think because you’ve been on top Python a couple of times and you’ve been a supporter for what I’m doing a test and code for a while, too, and I appreciate that.
Yeah. I think I’ve been there since the beginning, at least as soon as I sort of clicked that it existed, I went on Patreon and did my part and I appreciate it.
And just the encouragement is kind of cool. The encouragement I get from people all over. It’s incredible.
Yeah.
But we’re not here to talk about me. Everybody knows about me.
The tireless publisher that you are. It’s true.
Tireless? Yeah. About once a month, maybe. If people are not familiar with who you are, tell me about you. Sure.
Yeah. Like you said, my name is Matthew Shemi. I’m a developer primarily specializing in Python. I’m probably best known for my work at PayPal on the Python infrastructure there. But I do a lot of other things where I volunteer for Wikipedia in terms of developing code in the periphery using Wikipedia APIs. And I’m involved in various types of open source and education. I have an O’Reilly course called Enterprise Software with Python based on my experiences at PayPal. And yeah, I sort of like to keep really busy with software.
So how long have you been at PayPal?
So let’s see. I was continuously employed at PayPal from 2008 until just recently. I’m still writing up the full account, I guess, because it is so long. But yeah, me and my team actually moved kind of wholesale to another company, and we do kind of the same thing we did at PayPal. But for them as the first principal engineer at a company called Shop Kick, it’s a little bit less famous than PayPal, but it’s a decent sized company, 40 engineers or so, and we’re happy to be sort of tech leaders there now.
Okay. Have you announced this and I just missed it?
No, that’s the thing.
It’s been very busy lately. Basically what happened was I sort of wrapped up things with PayPal. I did some Capstone projects for myself, wrote a bunch of blog posts.
I did the O’Reilly course and open source some stuff. And once it was all said and done, I had actually met these folks at the conference. They read the blog post, they had seen what we had open source, and they really wanted to have that sort of architecture at their company. And they made a pretty good offer for the whole team. So I decided to move over.
Kind of incredible how many people moved over.
Yeah, it’s just a few of us right now. We’re still working on, let’s see, consolidating the team, if you would. But it’s a little bit complicated, I guess.
How exciting. Yeah. But does this mean a move for you or you?
No, it’s still in the Bay Area. It’s all still in the same locale. I guess the thing is that I did take a bit of time off between the jobs to sort of like write it up and announce it and do it right. And then I got sucked into this thing called WikiLeaks Monuments, which is the largest photography competition in the world. And they were really lacking in tooling and weren’t sure how they were going to manage to judge this giant competition with the new team that was running it. And so me and a few other volunteers got together and developed a new judging platform for them, which was a really fun project and it was on a deadline, so that made it all the more fun.
But that did really take away my time to sort of like write up an announcement on this and I’m just getting back to it as we speak.
So.
Yeah, that’s what I’m up to these days.
I was just actually down in San Jose last weekend.
Oh, wow. You should stop by.
Yeah, I guess I missed the big torrential rainstorm.
That’s true.
It’s a rainy one right now. Not usual.
Yeah, I was in Los Cabos. I’d never been there before. It was kind of a fun trip. I want to start with something light. Sure. Hat Note is the thing that you do.
Oh yeah. So Hat Note is sort of the imprint that I created for all of these Wikipedia projects that we’re constantly putting together. It just sort of made sense to have there be a destination for it and have to be an organization so that if other people wanted to join up and maybe put on their resume, they could. At this point, I think we have effectively like six Sudo staff.
Okay. Yeah.
I mean it’s just people with different specializations who have developed different software projects that maintain it under this umbrella and we provide like hosting and this sort of stuff.
I run it like an organization, but it’s really quite loose.
And then there’s like this listen to Wikipedia thing. Yeah.
So listen to Wikipedia is probably our most popular project. It’s been on NPR and BBC and French television and all over the place. It sort of took off and I think that will probably stay my most popular project for a while. It’s going to take a bit to outdo myself there.
I always get it wrong when I try to tell people about it. So what’s the quick elevator pitch of what is it? It’s playing music based on something. Sure.
So I mean, originally the catch description I came up with was was it sort of like a wind chime for the internet or specifically Wikipedia? But it was sort of inspired by a project that we saw by a guy named Maximilian. And what it does is for every Wikipedia edit that happens, it creates a visualization and sonification in real time. So you can listen to other people edit Wikipedia. It ends up being a really nice writing tool. I mean, you were just talking about being so busy with writing and that’s definitely something I’ve experienced. And one aspect of this that I like is that whereas writing is normally a pretty lonely activity, I mean, I’m way more productive when I have a quiet room to myself. But at the same time I like to know that there are other people out there who are doing that and this really brings that to life.
So each note is somebody clicking publish or something.
Right? So every tone that you hear, which is either a string plug for a net removal or a Bell for a. Net addition, every sound that you hear is somebody editing an article on Wikipedia.
Okay. So it averages for English Wikipedia between 60 and 160 edits per minute.
Wow.
Okay.
Well, it’s very pleasant to listen to. Yeah, I like it.
That’s what we’re aiming for.
And I can’t think of anything useful about it other than being interesting. Sure.
So I can tell you a user, too. That’s come up. So for other Wikipedias Besides English Wikipedia, the edit velocity is much lower. It’s not 60 to 160 /minute. And so it sort of acts as a kind of notification service for things that may need review on quieter Wikipedias of other languages. Yeah.
Okay.
People find uses for these things. We sort of like theorized, but we didn’t expect it to take off this much. People use it for yoga classes and, yeah, it’s really taken off life its own.
I like the idea of playing it while I’m writing, just to know that there’s other people out there trying to produce content for the world. That’s pretty cool. Yeah, exactly. I like it. So one of the things that came up that one of the reasons why I wanted to have you on and because you volunteered was a long time ago, maybe three or four months ago, I had a question that I’ve been reluctant to answer. And the question was basically, could I cover the different levels of testing, like unit testing and system testing? What did all those things mean? And instead of trying to answer that myself, I thought I’d bring somebody on to help. And you said you would.
Sure. So, I mean, hopefully it’s going to be a group project.
Having listened to your past podcast, your past episodes, I think that we may have slightly skewed views on this sort of thing, but I think we’re kind of of the same mind in a way. But anyways, I think the usual test levels that people describe are units tests and integration tests and regression tests and functional tests and system tests. And I kind of go over these in my O’Reilly course, but I give it a pretty light touch. I’m not very, as I would say, religious about what these definitions are. So for one, it’s like, what is a unit? There’s all sorts of different levels of units, really. And it kind of varies by your abstraction.
Let’s just start there for you. When you’re thinking of your code, what do you think of as a unit? What do you think of as a unit test?
If I’m back in school and I want to get a good grade on the test, I’m going to say a unit test is a test that focuses on a public class or a public static function. Right. Some aspect of a public API just writes a test just for that. That is what gets tested in a unit test.
If that function calls some other function in a different library or a different class, are those part of the unit test?
No, they shouldn’t be. And that’s where mocks come in.
I’m sure we had a lot of fun discussions on the podcast about those.
So do you do that? Do you practice tight unit testing and mocking?
Yes and no. So historically, not so much, right.
In my own libraries, yes, I do unit tests at the function level, because that’s what makes sense. However, I’m a big fan of consistency. Right. And if there are conventions that exist in the organization or team, then it’s really important to maintain those unless you have a meeting of the minds and decide to go a different direction. So our current organization, they have a long history of doing unit tests. So, yes, we do unit tests now. And the way that we achieve that is that mocks are injected, they’re passed in as opposed to being sort of changed or monkey patched at the module level. So they have a pretty good rigorous and practice that they followed for many years. And it’s like, okay, this is something you can do. And we’ll do it too. Now, that said, we’re trying to inspire a lot more around integration testing. So basically right now most things have nothing but unit tests. And then there’s a QA team that does manual testing, and that’s sort of the big integration test, aka maybe a system test right now. Instead, what we want to do is we want to create automated integration test and code kind of going trying to find what convention is going to work given the legacy code base, which is quite large.
Is this Python, I assume, and the integration test, is that just something somewhere between system and unit, or you have a particular definition?
Yeah, exactly.
It’s something somewhere between system and unit. Exactly.
I don’t have a really great specific definition that always works for this. And in fact, I don’t want to be too contrary in here, but I would say that there shouldn’t be one. So one thing that I find a little bit appalling about tests, I don’t know exactly how to phrase this. This is a little bit off the cuff, but for me, my first introduction to testing wasn’t formal in any way. It was kind of indirect. Basically what it was was someone was saying like, oh, I don’t really feel like tackling this feature or this assignment today. Maybe I’ll just go write some tests.
Right.
Somehow the tests are a smaller thing. They don’t want a full meal, they just want to snack. Right. At the time I just accepted it. And at the time I was also writing unit tests like crazy, like very soon afterwards. So it was kind of a different time these days. My view is that it’s very strange that application development and infrastructure development, it doesn’t have these levels, these labels assigned to it. And yet, in my experience, a test is always necessarily more dynamic than the code that it’s testing. It has to really flex around this code in order to test it from every angle. And so the test is actually much harder. And I think that the reason why unit tests are so popular and so well defined and rigorous is because they’re kind of easy. Right. They are these bite sized snacks. You just isolate a very small piece, pack, it full of mocks, and then it’s almost something that can be a little bit tedious at times. And I think that a good test is never tedious. And integration tests are challenging things to do. And the fact that it’s challenging is a sign that it actually does need to be tested because otherwise someone’s just going to have to manually do it at a system level. Yeah, that’s fine. Sorry to rant there, but I think that’s basically where I’m at right now with my testing philosophy.
I like it the one thing that. Well, I’m going to rant a little bit as well. Sure. Because at least this part of the conversation hopefully wasn’t just an interview to get your view, but I want to spout my view as well. Before I get there, though, I have one intermediate question. Have you always worked with QA teams? Yeah.
From the very first day of my professional full time employment, there was a QA team, and that sort of faded over time as what we did got sort of harder to test, if you would.
Yeah.
Okay.
In my career, let’s see if it worked at lots of different groups. And usually the norm is not having a QA. The norm that I’ve been with is the development team does all of the testing. And so in a philosophy like test driven development or something, where the idea is you spend I don’t know if anybody’s ever said this, but approximately half your time writing tests and approximately half your time writing source code.
Maybe that sounds great.
But if you’re doing that and half your time is doing unit tests, then I don’t have another half of my time left over to write system tests.
Yeah.
And so every time I’ve been on a project without a QA team, then the development team needs to decide where they’re spending their testing time. And the promises that we’re giving to the customer, to me, are more important than the promises that I’m giving to myself for individual function. So I actually am not opposed to unit tests. I write quite a few, but I think a balance of trying to find out, writing tests where I can find the most information about the system I’m trying to design.
Exactly.
And doing that as the level that makes sense.
It was off the cuff. I can’t remember who I heard it from. But instead of things like unit test integration, and system. I heard a reference to unit tests, package level, subsystem level, or service level, and then system tests. I think even system tests could even be broken into like a sub feature or something.
But when I first learned about this, I was developing a product with there was absolutely no way you could run a unit test.
It was a VxWorks embedded system, and I guess I could have tried to write unit test, but it was just most of our testing was system level. And so I tried to morph the TDD idea into what I thought of as a functional unit, trying to assume the rest of the world works and test one function at a time or one functionality. Yeah.
I mean, that’s definitely an approach. And when I look at code bases that I admire, the only thing that I find really consistently between them isn’t like the plethora of unit tests or something. But the thing that I find in common is that even if I can mutually understand all of their different code bases, their testing strategies are so wildly different.
And I think it has to do with the type of software that’s being written. So I think about SQLite. I really admire SQLlite code base, and it has at least three different testing strategies, everything from unit tests to test specifically for the query, parser, and planner. And then it has like, even I mentioned Fuzzing, basically, and it has much more than that. And it’s a component that really needs it. And I don’t know how much you can take from that approach and just go cross apply it to an arbitrary application. It’s a piece of infrastructure of a certain sort that needs its own kinds of tests.
Yeah. And there are certain things that are going to be difficult to do at a small level.
I’ve looked at the test code for pytest itself, and a lot of that is a little tight.
It’s actually pretty decent documentation. I’ve never been one for believing that tests or documentation for a system, but for Pipe test, it works pretty good because the tests are they write little tests and say for this feature to work, then this kind of a test should result in something like this. And that’s definitely a functional system level.
I got to think that’s really hard for that team to discuss when your tests are themselves like their test cases. But the test case can itself also be part of a bigger test case. It’s just going to be really tough to discuss.
I really applaud those framework offers.
Yeah, there’s some pretty smart guys for sure, and most of it volunteer.
But anyway, definitely I donated to the recent Sprint, and I got some stickers. They’re pretty nice.
I didn’t donate, but I begged for stickers and I got them anyway.
Well, I mean, I think that you’re contributing plenty to the Pipe community as well, in your own way. So I was also going to go off a little bit more on unit testing, please. Yeah. Basically, I think the TVD can be useful for a certain type of programmer in a certain place. But I think that your approach as using tests to discover, like the application that you need and the architecture that you need is really critical. And the reason is I talked about this, Mike course, that when you add tests to something or documentation or really any sort of software engineering periphery to a piece of code, it’s a little bit like pouring cement on it. You’re kind of solidifying this piece of code and it becomes something that’s harder to refactor. If you go unit tests first, you’re so zoomed into the design that if a refactor comes, you’re going to really weigh yourself down. So unless there’s something like a spec, a standard and RFC that you’re implementing to, or if you’re doing a Port right, then I think that this bottom up testing isn’t great. You talked about I can’t remember how long ago, like, basically people are obsessed with the testing pyramid that you have this heavy base of tons of unit tests, then you have some integration tests, then you have this tiny little triangle of system tests at top. And this is, I guess, to try to minimize manual testing. But if that manual testing is something that would have helped you discover a better design, a better API, you can think of it as like a UX test. There’s no way to really turn that into a unit test.
If you try to prioritize that base and work your way up, you can end up with some pretty bad software. Tdd isn’t necessarily going to give you great software.
Well, if you do it from the unit test level, you’re exactly right.
Yeah, exactly.
But if you read Kent Back’s early writing about it and his recent discussions about it, he never intended it to be unit test only.
I should temper that for sure. It’s just every time I’ve seen a demo, to me it’s like, oh, we’re going to implement a test for this addition operation.
Yeah, well, I’m guilty of that, too. But it’s an easy example to show.
I guess, if I have any innovation and testing space, because everything I’m saying now, I think has been said sort of here and there by you and others. If I have any innovation to add to the testing space. It was when I did a Port of the dust templating language for Python. It’s called Ashes. It’s on GitHub. It’s sort of a single file templating language that is intended for HTML, web use and so forth. Is sort of basically enables you to have server side rendering of something that you can also render on the front end. Kind of an unrealized dream. It’s never quite perfect, but it works really well as a Python template language. And the way that I got it to that point was I was porting it from JavaScript, and because it was a Port, I was able to start with the unit test and code are all these different stages. You have to parse the template, you have to tokenize the template, you have to parse the template, you have to optimize the template, you have to compile it, you have to compile it to Python code and compile that Python code and run it, and then check that the output is the same as the JavaScript. I called it cumulative testing. At the time, each of these stages was like a unit test for a given template input. And if all six stages or whatever passed, then that was kind of my integration test. And if a whole set of these templates from the docs of the dust. Js templating language were to pass, and that was sort of like a system test, if you would. And so it had a nice grid that it output that showed that it was compatible with the spec of dust. Js. And I had to write my own test runner for that. I tried to figure out a way to get to work in Python, and it just wasn’t a great fit. But I know because it was a simple Port. I know that that was the best testing strategy, though.
Well, you tested at different levels to give yourself confidence at different levels of the software, right.
I think that if you’re going to write tests early, then they shouldn’t just weigh you down. They shouldn’t just end up being crushed. That delays a refactor. They should really be helpful during the development process.
So that’s sort of going with what you’re saying, discovering more about the application you want.
That’s what I focus mostly on, the APIs when I’m doing testing. And the API can be an internal API, like a package that I’m intending to reuse later, but it also could be the end user API or something.
Right.
And like a project I’m working on right now, I don’t know what the user interface is exactly going to look like. So I’m focusing on one layer down. That is something that’s easy to access, an API level, but that’s really easy to run from Python.
Yeah. I mean, anything that I’m doing that I haven’t done before, any sort of creative software that I’m doing, I’ll take any angle on it that I can get. So I’ll start writing the code, and then I’ll say, oh, should I do this or should do that? And I try to answer that question through either writing a test or writing some docs or telling someone about it in writing like a higher level.
Actually, any level test, but a higher level, sort of like API test or something. Do you learn about the software that you’re designing while you’re writing tests?
No, absolutely. I’m telling you, that’s the whole point that I write tests that early on, unless I have a specific standard report that I’m going against. The reason I write tests while I’m developing is to feel what it would be like to use a library or an application like this if that learning is so important.
And I think it is.
It absolutely is. Yeah.
Why would you let a separate team like a QA team get to do that learning?
Oh, arrangement that would have a QA team do that, I think is robbing the developer of fulfilling development process. The QA teams that I’ve seen, it actually goes the other way actually often end up feeling kind of bad for QA teams because oftentimes they include very capable developers like yourself. And they aren’t allowed to be QA engineers. They aren’t given the proper amount of time to automate the stuff that they should just have done manually. And on my desk yesterday, they just verify a build. And that’s okay for some folks, but I’ve really, in my experience, seeing QA people who want to expand beyond that role and start using their programming skills again. But Alas, that’s not always the case.
Yeah. I haven’t worked with a QA team that does mostly manual for a very long time these days.
Things are happening in closed systems. It’s harder to automate something then, especially if you’re working with consumer software, then the test really is to be the consumer of the software. If you’re dealing with an app. And so if you’re dealing in server code, infrastructure code, then automation is really within your grasp. And that was a place where I felt like the most frustrated that they weren’t being encouraged to automate being given the time to automate these days, I really don’t know. Your hands are kind of tied by Apple and Google in many cases. So what are you going to do?
I guess I’m fortunate for always. I have for my whole career been worked in systems that have a complete API that is actually usually more functional than the manual user interface.
That’s is nice. Yeah.
But then I don’t get to have fun with things like microservices and stuff like that.
Don’t feel too left out.
We talked about unit tests, kind of talked about integration testing somewhere between system and unit system.
Hopefully that we agree that that’s just some test that tests the whole thing.
Right.
And hopefully as close to a production environment as possible and a functional test. What’s a functional test?
Like I said, I think that all these labels are just sort of created so that people can move around tests that need to exist into these various buckets and it can be useful to discuss them, to use them as sort of placeholders for degrees of granularity. But I think that you won’t find moving from team to team. Right. You’re going to find a wide variation of what is a functional test versus what is an integration test. And so personally, I would place if you want an ordering. Right. By my expectation it would be like unit functional integration system is my guess. Maybe integration and functional are reversed. Okay. The one thing that deserves a shout out regardless is regression testing, which isn’t a level because regression can happen at any level. But I think that ends up being maybe one of the most important testing practices that you need to have an affordance. Even if you’re not using a standard test run like Pi test, you need to have some affordance for feeding buggy inputs into the software and making sure that they stay fixed.
Okay, so what is a regression test?
So regression test is when you discover a bug and you create a minimum viable or maybe not minimum viable, but like if you obtain some input that will trigger that failure and then you codify that into your testing automation system. That is an automated regression test.
Basically, once you fix that bug, that test should never fail again. It should never cause that bug to occur again.
No. After you’ve written that test, do you put that somewhere else than the other test? Do you have just a regression suite or do you have just put it in with the rest of the tests?
In our server framework, we had a lot of custom serialization code and there were a lot of corner cases that could occur with that. So we had a custom regression test directory, a whole tree for these strange and exotic serialized formats that would cause errors, and we go and we’d fix our serialization code and then we’d never hear from them again. I’m not sure where I read it, but someone called that directory. It’s like a pattern that emerges in a lot of code basis. Someone called that directory the Zoo because that’s sort of your menagerie of all these very intriguing, interesting wild beasts because it’s just called it Portland.
It is your hometown. You can say that sort of thing.
I was curious what you guys did or what you’d like to do.
That sort of a test we just try to put in with wherever it makes sense at the same testing level as everything else. But I do like to test the red also just like every other code. And I do like to put in for a particular test like that a reference back to the defect tracking number or whatever you’re using.
Absolutely. Yeah.
So that somebody can not necessarily in the test name, but at least in the dock string or something to say, this test is here because this bug happened and we’re making sure that it doesn’t happen again.
Yeah. I’ve never actually worked at a place that had really good documentation sort of standards and practices.
That said, I’m pushing it really hard. Like when you write an integration test or frankly, any piece of code, I always tell them, don’t tell me how this works. I can read the code that’s Python and tell me why this code exists. Why you think this code exists? Why this test exists that is going to really help me make decisions down the line. Much more than a rough, shod description of what the code did when it was first written. Tell me, like a change log of why changes were made, and that’s going to save me a lot of code having to be read.
Yeah, definitely. I think it’s even more important, but I guess it’s equally important to test to say why a test is there.
Definitely.
Because some tests are there. Just because this is the functionality that works as is, we’re just making sure that nothing changes or this is the functionality that really important Customer X wanted. And so we need to make sure that’s in place. Something like that. Anyway, if I was going to change gears a bit.
Absolutely.
Unless you want to keep on talking about test levels.
No, man.
Well, I want to ask you a little bit more about your enterprise Python program that you have.
So tell me what it is.
Sure, absolutely. So once the Python came to PayPal reached a certain place where we were maintaining our own framework and we had lots of users and so forth inside the company, we had a little bit of spare time to write about our learnings. And so I wrote a blog post about the myths of enterprise Python. Basically the kind of misconceptions I had to dispel meeting after meeting. And I was like, this is just going to be way more efficient if I write it up. And what better place to write it up than the public company tech blog, of course. So having written that and a couple of other posts, O’Reilly reached out to me. They were curious if I could put together a video course for enterprise Python usage. And I agree. I was a little bit hesitant about it, but I ended up agreeing put much time into it and it’s available for purchase and viewing. But the TLDR of it is that there’s not really a formula for it. There’s no silver bullet. There are a bunch of practices that you learn, there are a lot of things to avoid, but you have to kind of just hunker down and write code that actually gets the job done. Basically, when I’m interacting with people in enterprise environment, this is the reminder that I constantly have to give them is that if your code works, if it does its job, if it provides value, then we can get out of all these meetings. Right. Like, meetings are not going to get the code written. And so this is kind of the power of Python. And our experience is that you can sweep in, you can get the job done, get something that works and is reliable, and then you can move on to more projects.
Faster code, less meetings. I like it.
Yeah. It’s a nonstandard approach to enterprise software, but it’s trying to kind of clean up the act, if you would.
Okay, so by enterprise software, what is the enterprise is that stuff like tools that are written for internal enterprise usage or.
It’s a great question. I think that may be one of the preview segments of the course is I sort of outline what enterprise software is. I get kind of nine hallmarks of enterprise software, and some of the keys are like, well, unlike consumer software, it’s not being shipped out to the market as a whole. Right. You have a target market, maybe as few as just five or six people. I wrote the pricing tools at PayPal, and I don’t think more than six people could change the prices of PayPal, but it was still a pretty complex piece of code because the pricing structures were so advanced. So that’s something that made that enterprise software. It was written on Django. So it wasn’t necessarily like something crazy under the hood, but it was pretty niche. So that’s an example of a hallmark of enterprise software.
I’d say, I guess. Does it make a difference if you’re going to write a little tool that has a Rest API or a user interface on a web app, does that make a difference to which framework you’re going to use if you’re going to use Flask or Django or something?
Absolutely. So at PayPal, like I said, we started with Django and we went with that for a while when we were doing these admin type front ends and that ended up being an architecture that was very popular. It turned kind of viral. I wrote really good docs for it, and then many other teams kind of just adopted it wholesale so that you’ll find a lot of Django for admin tools at PayPal. Then when we switched to more service oriented architecture, we ended up dropping Django. We didn’t use Django, Rest framework. And even when looking at Flask, we found that to be a little bit too focused on very expansive APIs. Like, very featureful, not very performance focused. And we knew what our target consumers would be accessing us with, like what kind of clients they had. So we actually switched to a web framework that I wrote as part of Hat Note for Wikipedia project that uses work swag under the hood. So it’s like Flask, but it is much lighter weight and has less global state.
So that made it really good for our high concurrency use cases.
And is that classic?
So that’s classic. And yeah, you can see kind of how we integrate it with the support reference framework that we released on the PayPal public GitHub.
And we still use these tools at our new organization as well as Shopkick, I guess because it was kind of focused in nature. It was suitable for our particular use cases and we’re pretty satisfied with it.
And you would still grab Django if you’re going to use something with the front end, then.
So one of the main reasons I recommend Django for folks is like, I look at the big picture, right?
If they are someone who enjoys web development and they want to specialize in this, I’m not going to recommend a tiny little micro framework. If they want to have a career developing Django sites or developing websites, then Django is a great way to go. It’s a huge ecosystem, great community, tons of documentation, and those are really important parts. It’s not just about the code underneath, because Django has some ugly bits internally.
That sort of thing would affect our specific use case, but that doesn’t disqualify as a whole.
So can I use Classic for something as a front end then?
Yeah, hopefully it’s there. I use it. The Montage project that is the WikiLeaks Monuments judging platform is probably the largest external to PayPal usage of plastic that there is, so it makes a pretty good reference.
Can normal humans access that, or is that an internal thing?
No, that’s on GitHub. Comhatnotesmontage.
Okay.
If you want to actually access the live site, you would need to be invited as a jury member to one of the competitions that are being held on Wikipedia Commons. So in a way, Montage was a lot like this enterprise software again, where they had a very specific purpose and it had a pretty small audience. I think that even though this is the largest photography competition in the world and had like 275,000 photos entered into it from 50 different countries, only 200 people have logged in ever.
Really?
Okay, yeah, it’s pretty niche, but as you’ll see from the code base, it’s far from trivial. I think that is one of the key hallmarks of enterprise software.
Cool. Yeah, we’re wrapping up nearly an hour. Sure. But do you have anything else you want to plug or anything?
No. I mean, not really. These days I’m sort of putting together some more writing, tying up some loose ends. I’m focused a lot on application instrumentation right now that WikiLeaks Monuments has done. So if people are interested in performance measurements, robust, structured logging, I’m always looking for collaborators, I guess. And so, yeah, you can hit me up on Twitter. I’m M-H-A-S-H-E-M-I.
I’m sure that you’ll have links on the site as well, or I’m just math moved on. Github, too.
I’ll put links up. Thanks to time for coming on. Yeah, absolutely.
Thank you for having me.
Thanks again to Talk Python courses, Nerd Lettering and testing Code Patreon supporters for making this show possible.
The only promo code that I had to give to you was the Nerdlettering. So that’s testcode all one word at Nerd Lettering.com. Talk Pythoncourses are at training. Talk Python. Fm.
And you can support the show by becoming a Patreon supporter.
And there’s information on that in the supportlink at testandcode.com show notes are at testandcode.com. Yes, that’s correct.
You can still find me at Python testing.net. But testing Code.com is now a thing. Past episodes are there, but they might not be pretty yet. I’m still working on it. I’ve got some credible people lined up for future episodes and a couple of interviews already recorded and in the pipeline talk to you next time. Now go code testimcode.