Twisted has been supporting asynchronous / event driven applications way before asyncio. Twisted, and Glyph, have also been encouraging automated tests for a very long time.
Twisted uses a technique that should be usable by other applications, even those using asyncio or other event driven architectures.
Transcript for episode 184 of the Test & Code Podcast
This transcript starts as an auto generated transcript.
PRs welcome if you want to help fix any errors.
00:00:02 [Brian] Twisted has been supporting Asynchronous and event driven applications way before Async IO came on the scene. Twisted and Cliff have also been encouraging automated tests for a very long time. For testing Asynchronous or event driven systems, Twisted uses a technique that should be usable by other applications, even those using Async, IO or other event driven architectures.
00:00:26 [Brian] This episode is sponsored by Rollbar. Rollbar is the leading platform that enables developers to proactively, discover and resolve issues in their code, allowing them to work on continuous code improvement throughout the software development lifecycle. Rollbar has plans for all situations, from free to large enterprise. With Rollbar, developers deploy better software faster and can quickly recover from critical errors as they happen. Learn More at rollbar.com Thanks, Rollbar.
00:01:09 [Brian] Welcome to Test and Code.
00:01:19 [Brian] Welcome to Test and Code. I’m super excited to have Gliff on the episode.
00:01:24 [Brian] Welcome to the show. How long you been doing Python?
00:01:27 [Glyph] I’ve been doing Python for probably about 22 years.
00:01:32 [Brian] Wow. I should have dug it up, but I read a bio view of like that, talked about both you and Twisted kind of together in the same context. Has Twisted always been part of your Python experience?
00:01:45 [Glyph] I had about a year of professional Python before Twisted started in earnest. It was an outgrowth of a project called Twisted Reality, which was originally in Java.
00:01:56 [Glyph] I won’t rehash that story here. It’s kind of long, but you can certainly find some podcasts where I’ve talked about it before.
00:02:03 [Brian] Okay.
00:02:04 [Brian] And public speaking, has that always been part of your professional life or just something you enjoy doing?
00:02:12 [Glyph] I started pretty early and I guess it’s been part of my professional persona for many years now. But yeah, it was really just a personal interest thing. I could sometimes convince my employer like, oh, it’d be good to have a slide at the beginning of the deck that says where I work. But it was never really never my boss saying, hey, it would be good to give a talk.
00:02:38 [Brian] Okay, so what I wanted to have you on the show for was to talk about Twisted. And since Twisted has a lot of Asynchronous in it testing of Asynchronous code, which both of these things seem confusing to me. And I have to admit, I know Twisted is a thing, but I don’t really know what it is. So can you introduce us to Twisted a little bit?
00:03:00 [Glyph] Sure.
00:03:02 [Glyph] The tagline for Twisted is that it is an event driven networking engine.
00:03:06 [Glyph] I actually kind of prefer event driven to Asynchronous because Asynchronous sounds very jargony.
00:03:12 [Glyph] I know that with the Asynch keyword, it’s kind of embedded itself in the language pretty hard, but if you think about it as event driven, it’s a way to write programs that respond to things happening as opposed to necessarily like just doing things sequentially.
00:03:29 [Glyph] Twisted is a 21 year old engine that has support for a variety of protocols. It’s got IRC, SMTP, IMAP.
00:03:41 [Glyph] Obviously it’s a web server. It’s got Http and Http two. It’s got third party support for a bunch of different protocols, including things like WebSockets, which you need for modern web apps. It’s the original flavor of Async Python. There were Async things before it. There was Medusa, which it was largely based on. So it wasn’t coming into a total vacuum. But it also includes compatibility with and support for Async IO in the standard library.
00:04:12 [Glyph] So, yeah, it’s a little bit difficult to give comprehensive overview of it. It’s kind of a way of programming in Python. It’s like a dialect almost.
00:04:21 [Brian] Okay, well, I guess I’ll jump in and ask some pointed questions around it.
00:04:28 [Brian] What would I use it with? It sounds like it’s web centric. So would I use it with another web front end like Flaskango or does it have that sort of stuff built into it already that I don’t need Flask or Django if I’m using Twisted.
00:04:42 [Glyph] Twisted is a little bit more low level than something like Flask or Django.
00:04:47 [Glyph] It does have a couple of projects within the Twisted.org on GitHub, you can find Trek, which is our sort of requests alike that uses Twisted and Cline, which is our Flask alike, which uses Twisted. They’re both kind of a little bit different than the libraries that they are patterned on, but if you were going to use Client and Trek and Twisted together, that would be a lot like writing a Flaskrequest type web application.
00:05:16 [Glyph] It’s actually a little bit less web centric than most other things in this category.
00:05:23 [Glyph] It’s not just for writing web apps. As I said, it supports like SMTP and I Maps. You can use it to write an email server. You can use it to write a DNS server in most practical projects, because the ecosystem around Klein and Track is much less mature and certainly much smaller than around something like Django. Most people who use Twisted end up using it as a kind of sidecar that’s doing something event driven in the back end of a site that’s using Django or Flask or something. For example, ASGI like the reference implementation of Django’s.
00:05:59 [Glyph] Event based Red socket support is a server called Daphne, which uses Twisted. So quite often, if you’re using it, you don’t actually know that it’s index. It’s also a part of scraping, so if you don’t really scraping, you might find it there.
00:06:12 [Brian] Yeah.
00:06:14 [Brian] Okay, so it’s definitely a networking thing, but it’s not necessarily the immediate front end stuff that you’re looking at. It’s handling a lot of other work.
00:06:25 [Glyph] Yeah, it can be. It’s also a WSGI container, and I think being a pure Python WSGI container does give it some advantages. But again, it doesn’t have the community size behind something like G, Unicorn, or Uisgy, I would say, like, I’ve certainly deployed lots of production stuff on Twisted as the network facing thing, and I generally recommend it. And kind of when people are saying, oh, we have NGINX talking to you, talking to album, and they have this huge stack of stuff that’s in the path of every Http request. I often begin the conversation when I’m talking to folks who do that saying why not just put Twisted at the front and like if you want to use Django, Django goes into WSGI container so you can do that. And it’s often, oddly enough, for Twisted has this reputation for being somewhat complex because a lot of people aren’t super familiar with eventually programming. It actually is a thing you can use to simplify your architecture quite a bit because you can get your whole site or your whole app onto Python and just Python and then kind of have most of the functionality that you would get from some of these bigger, more complex like C Plus plus low level servers.
00:07:38 [Brian] Yeah.
00:07:40 [Brian] So the article I was reading about Twisted had a quote and I think it was from you that Twisted doesn’t have anything or one of your design ideas is to not put any new ideas into it. Yeah.
00:07:56 [Glyph] Twisted is very much not innovative.
00:08:00 [Glyph] As we mentioned before we started the show, my dad is a programmer, he’s retired now, but he has been doing this since the 70s. And so I was exposed to a lot of ideas when I was growing up and I believe I might be mixing this up a little bit. But many of the ideas in Twisted about how to structure programs are directly lifted from some stuff he was doing in C on Stratus minicomputers in the 80s.
00:08:33 [Glyph] So they’re not super new ideas.
00:08:36 [Brian] Yeah, well, I mean, I love the idea that you said instead of using Async that’s a buzzword or asynchronous event driven.
00:08:49 [Brian] We’ve been doing that for years, a really long time. When I think about it like that, most of my programming is event driven programming, most of, you know, user interface stuff.
00:09:02 [Brian] On my end, with embedded systems, it’s usually database change events that trigger actions to happen and things like that.
00:09:13 [Brian] Yeah.
00:09:13 [Glyph] And you could definitely hook up Twisted to something like Postgres. Listen, notify like it doesn’t have to be pulling stuff off of a socket.
00:09:19 [Brian] Wow. Okay.
00:09:21 [Brian] I could probably even have buttons attached to it and have buttons drive it and stuff like that.
00:09:28 [Glyph] Funny you should mention that the thing that I was going to say next was given that we brought up public speaking, the talk that I got interrupted with the flashing sign that I didn’t know what it meant that said five on it. I actually did know about the like five minutes left thing that they do at Python, but my session runner got the timings wrong and interrupted me way too early. So unfortunately this talk is a little bit rushed at the end. But I have a talk called So Easy you can even do it in JavaScript.
00:09:58 [Glyph] And it is event driven programming for regular programmers. And the gist of that talk. Like, I actually used PIJs to make the point.
00:10:08 [Glyph] But one of the core abstractions in Twisted is a deferred, which is nowadays, like in Async I O, there’s this thing called a future that’s actually in standard Python, it’s just part of the standard library, and Deferred was a precursor in a parallel to that.
00:10:25 [Glyph] And I show what deferreds are like, why they exist, and what you need them for by this process. In that talk of building up a dialog box with a yes and no button in it and showing like, really the way that you think about a deferred. Or like nowadays, this would be like a future that you would have wait on.
00:10:46 [Glyph] You think of it as like, I’m coding on coding. I’ve got this process, I’m going down linearly, and then I need to stop and wait for something, and then I keep going.
00:10:54 [Glyph] But really, when you get to that point where you stop, you’re just waiting for some event to occur, and you want to know what the result of the event that you were expecting to happen was. And so in the case of popping up a visual thing like a dialog box to the user, other things can be going on behind that.
00:11:16 [Glyph] And that dialog box just has event handlers for yes and no. And those are the things that unblock your program and allow it to proceed.
00:11:26 [Glyph] But you don’t need to structure it that way. Right. The reason I don’t love Asynchronous as a word is because Asynchronous is that style of code where it’s sequential and you stop and then something happens elsewhere. Whereas Event Driven is like anything could happen lots of different. You could get a function called from some event occurring somewhere else. Like, your code may not have been started yet, there may not be an ongoing coroutine or process, and unblocking a coroutine and starting a new coroutine are both instances of an event happened? What do you do next?
00:12:03 [Brian] I have to go watch this then.
00:12:05 [Brian] Okay, never mind that everybody listen. Just stop and go.
00:12:12 [Brian] So let’s just jump ahead.
00:12:15 [Brian] One of the difficulties with Event Driven or Asynchronous code is figuring out if it works because sequential code is easier or still hard to test, but it’s easier. Is there some tricks and things you’ve learned when testing event Driven code?
00:12:34 [Glyph] Well, there’s a whole story behind that. So first of all, testing has been part of Twisted for a very long time. Nowadays, in the culture of programming, we all make public shows of saying, oh, I wish I tested my code more. I wish that I had better coverage. Of course, we all it’s sort of a universally recognized virtue to have tests. Twisted started Mandating testing as part of its development process in 2005, and it was not a popular thing back then. Like, definitely not everybody thought you should write unit tests for everything.
00:13:07 [Glyph] So we have routes that go back quite a long ways in testing. And the process of doing that has really been an evolution from kind of what you just described. Like, oh, test and code is hard. Like you need special tools, it’s difficult.
00:13:29 [Glyph] And that’s where we started was we have our own test runner. It’s called trial, and the way that we originally wrote it to support asynchronous testing was spin up the whole event loop, get all the sockets and all the infrastructure for actually talking to the network ready, then hook up a loopback connection. So you have both the server socket and the client socket, and then you send real network traffic over local host and you kind of see what happens.
00:13:56 [Glyph] You can still do this today. It’s useful for certain kinds of integration testing, but what you’re doing there is that’s full system integration testing. And this is what happens if you return a deferred from a trial test case. So if you write it looks like a regular unit test, but you can stick in a weight in the middle and, well, there’s some other stuff you have to do too. But like, you can basically write a coaching or a deferred returning function and it’ll run the reactor for you in the background and actually wait for that event to finish.
00:14:29 [Glyph] Unfortunately, that style of testing is actually not really appropriate for most event driven things. Okay, so the reason being that involves your whole system. It’s a full blown integration test, including the whole operating system.
00:14:44 [Glyph] And it’s very slow because you have to wait for real network traffic, you have to contact switch out of your process, wait for the OS to do something. There may be other things happening in the background which slow that down.
00:14:57 [Glyph] It’s very error prone.
00:15:00 [Glyph] If your test fails to fire the event at the time that you expect, if you didn’t send the request that you wanted, the only failure mode you get is like Oops, you’re hung test is producing no output. It’s not very useful. So while that feature is hard to build yourself and absolutely required for some very high level kinds of testing, the thing that we eventually converged on was don’t test anything that way. At first. The way that you test stuff asynchronously is, it depends which way you’re going. If you’re writing a coroutine or a thing that returns a deferred, some process which is like this is a request handler that does something asynchronous and then comes back.
00:15:44 [Glyph] You implement that by calling it and then using a couple of functions. We’ve got actually implemented within Twisted, a certain result success result of and failure result of. And those three methods on Twisted Zone test case don’t require a reactor, they don’t require an event loop. You can write them completely with no networking involved at all, just a deferred object and your logic. And so you’ll typically make a mock back end that returns like the lowest level deferred, the Http request or the Open socket or whatever the thing is at the bottom of your tower of abstractions.
00:16:22 [Glyph] And then you control what’s happening by firing that mock, or like putting some data into a fake network connection, just like pushing some bytes into the data received method of your lowest level protocol, and then saying at this point in the network stream, I expect this deferred that I got back from my application to have not fired yet.
00:16:42 [Glyph] Do some stuff. Okay, we shouldn’t be like nothing should have happened. A certain no result. Okay? Trigger an error condition failure result of and that will fail if the deferred has succeeded. So you have to get an error and then you get the error out.
00:16:55 [Glyph] It’s sort of like assert raises. You say what types of exceptions you expect and then success result of, which is like, okay, I’m going to send a request, I’m going to assert the transport received the bytes for that request, then I’m going to push the bytes of the response in, and then there should be a success result. And then you could say like, okay, I’ve got a success result. And this is actually one of the reasons why, despite the fact that the standard library has event driven stuff in it, now you can use Async IO. This is one of the things that makes Twisted kind of still relevant. Is this type of all in memory? No. Actual I O testing is like really well supported by Twisted, and there’s some features of deferreds that still make them a little bit nicer than Futures to work with in this way. You can do all of this with Futures, but futures expect to be attached to an event loop, whereas deferreds live totally on their own. So you can just have application code which has no idea if there’s even an event loop running. You can just say like this is async in some abstract sense. I expect this deferred to be fired when it fires. Like, I’m going to have a test assert about it.
00:18:10 [Brian] Okay, there’s a couple of questions in there.
00:18:17 [Brian] If I’m using Async I O, then I need to actually set up an event loop to test it.
00:18:22 [Glyph] Then yeah, and you can fake it. You know, the event loop is pretty abstract. You can go and implement all over 17 methods or something, and you can write your own fake in memory one.
00:18:33 [Brian] Maybe some exist.
00:18:36 [Glyph] I’m sure that some projects exist to at least do this partially. I personally haven’t actually seen any. I don’t code with Async IO in anger a lot, so like, it could easily have escaped me. I don’t want to say that that community has not got anything at all, but it is definitely something which I would be very surprised if it was as well supported as it isn’t twisted, largely because we made all the mistakes already. We had a huge super flaky test suite for years, which we grabbed some parts, which was its own test suite, still do a lot of real network IO where they don’t need to, just because it’s a venerable code base with a lot of history. But we’ve been gradually migrating to this strategy of doing everything in this really high level way.
00:19:19 [Brian] Okay.
00:19:20 [Glyph] For many years.
00:19:22 [Brian] Yeah. I think it was an article on the Twitter engineering blog that I read a long time ago talking about feature tests, where it just reminded me of that is that you’ve got event driven code, but you want to test it in their idea of a feature test is a test that tests a high level like most of the system, but does it in a context of a single thread so that it’s more predictable, stuff like that.
00:19:58 [Glyph] Yeah. The idea with Twisted is you not only test in a single thread, you test with a fake network transport. Okay. And there’s actually some great stuff that’s been built up at a little bit of a higher level for that. Like if you were going to test a web application, for example, let’s say you were writing full Twisted stack. Everything’s been twisted. You’ve got a track network client and a client like Flask style server.
00:20:26 [Glyph] You don’t have to go all the way to writing your own Http requests and bytes. You can use Trek testing to make requests of your client back end, but there’s no actual sockets. There’s no actual back end. You just handed an instance to your client application and you say, do all of the Http stuff, and it still tests the whole system. It still tests request parsing and cookie handling and everything, as if there’s a real network request, because it’s building it all up in memory and like all the actual bytes and invoking the whole Twisted protocol parser, but there’s no socket. So although you burn a little bit of resources doing all of that nominally unnecessary integration code, you don’t end up context switching. You don’t end up using real sockets. You don’t have to worry about binding an unused Port for each test or anything like that. And so you get a lot of the benefits of a big system integration test with a lot of the benefits of a nice tightly scoped unit test, because it’s still all in memory, it’s still got no side effects. Everything gets garbage collected at the end of the test, it doesn’t leak state across.
00:21:36 [Glyph] And so it’s kind of a nice sweet spot to use some of those tools to just test your whole application. At my previous job, we had our whole application tested that way, and it was really easy to drop it into route and start dropping a couple of tests that would exercise the whole thing.
00:21:53 [Brian] That’s pretty cool.
00:21:55 [Brian] One of the things you said a few minutes ago was you were talking about being able to separate deferred.
00:22:05 [Brian] You don’t need an event loop to test the deferred.
00:22:09 [Brian] And you said that’s one of the reasons why Twisted is still relevant. So hopefully this is a rude question. But is Twisted still relevant because of all the things that have come up in the rest of the Python community to support event driven code?
00:22:28 [Brian] Is Twisted still a good choice for some applications if they’re starting today?
00:22:33 [Glyph] Yeah, I think that it definitely is.
00:22:36 [Glyph] It is certainly less relevant. Like, if your goal is to write Async or event driven Python, Twisted is less relevant than it was five years ago because there’s a huge ecosystem now that uses just the standard library.
00:22:53 [Glyph] And sometimes people because they’re familiar with kind of old examples or they find some old search results on Google, they think, oh, Twisted is like the Python two way of doing it.
00:23:03 [Glyph] It’s the past. But that’s definitely not true. Twisted is definitely still relevant in the sense that if you need to do stuff with DNS, IRC, email any of these protocols that Twisted has support for, it’s just a good implementation of those things. And if you’re writing an Async I O project that uses an Async I O web framework and doesn’t care about Twisted at all, and suddenly you need to handle a DNS request, you can just drop Twisted in there, use the Async IO reactor, and call back and forth from your Twisted and Async IO code, and they integrate perfectly well. So you can just use Twisted for the suite of protocol support that it provides.
00:23:39 [Brian] Okay, how about future proofing? Is the Twisted maintenance going to go on for at least five years in the future? Do we know?
00:23:48 [Glyph] Well, I mean, there’s always the question of, like, are we going to have enough volunteer effort? That’s a perpetual problem with open source sustainability. I can’t promise anything.
00:23:57 [Brian] But there’s no plans on, like, whatever they call it, fading it out or something?
00:24:04 [Glyph] No, not at all. There’s lots of code running in production with it. And also, again, that impression that it’s outdated is largely just because we have 20 years of search results that may come up that show you something that is outdated.
00:24:19 [Brian] But the curse of old blog posts being popular ten years ago.
00:24:25 [Glyph] But we have kept pace with modern Python. It has support for type. Ins. Deferred is a generic that tells you what its result type is. If you’re using mypy, it has support for Async def Con routines. There’s probably a half a dozen other things that I’ve forgotten that we’ve got support for.
00:24:44 [Glyph] But yeah, Twisted is definitely a modern Python project. It’s got a lot of legacy and history. But coding with Twisted is not like writing Python two code. It is like writing very modern up to the minute Python 310 code, if your project supports that.
00:25:04 [Brian] Okay, I have a confession to make. When I started my current role in 2010, the test framework that I was used to was a Python based thing, but it was proprietary. So I had to learn the new company’s proprietary test runner, but I wasn’t happy with it, so I came up with one of my own things. But then I started looking out at the Python community to say, Why are we doing these bespoke things? Let’s look at the rest of and of course notice right away. Notice unit test, nose by test. Those are some of the things that I saw at the time that people are using, and I wrote a blog post to say, okay, I’m looking at Testing Frameworks, looking at these three. Are there any I forgot that I really should take a look at. And one of the things that came back right away was Twisted trial. Of course, I didn’t do a lot of research on it because I just looked at the whole Twisted ecosystem and the Twisted trial and I thought, I’m not using Twisted. So I think Twisted trial is tied with it. Did I get the right impression?
00:26:12 [Brian] Twisted trial, something I can use on its own? Or is it really go hand in hand with Twisted?
00:26:18 [Glyph] It’s a part of Twisted and it is pretty deeply integrated, and I have mixed feelings here.
00:26:27 [Glyph] There’s a lot that I like about trial, but it’s also one of the parts of Twisted that does show its age.
00:26:34 [Glyph] I am a little bit of a contrarian in that I don’t love Pytest.
00:26:38 [Glyph] Every major project I’ve had to interact with, it used Pytest. I ended up looking at a function and being like, okay, this takes six parameters and nothing ever passes any of them. What is happening?
00:26:54 [Glyph] I’m aware of the fixture configuration vaguely. I know that there are ways that those get populated, but I find that a very distasteful way of writing tests because it’s very confusing to me and even Python experts. I will sometimes ask, where does this come from? And they’re like, oh, let me get back to you on that.
00:27:14 [Glyph] Trial is definitely errs in the other direction in the sense that it’s just unit test, like everything is just unit test with these little sprinklings of extra async support.
00:27:26 [Glyph] The one thing that I will say so there’s two things, I guess. First is the and this is the thing that I think most of your listeners should probably take away from this conversation is those methods that I mentioned before, a certain result, success, results of failure, result of, and the kind of companion on the networking side, which is, I believe called the I O pump.
00:27:47 [Glyph] These are all things that allow you to test your code as just you make an object and you call methods on it, but you do it in such a way that you can tell that it’s plugged into the rest of the IO system without being slow and flaky and unreliable. And those techniques can easily be ported over to any other system, even ones that are not necessarily event driven.
00:28:11 [Glyph] And I would encourage people to go look at that philosophy and see those very small pieces of Trial as a system. If you were just like starting a new Python project today and you were like, I’m going to use Trial for some reason, why would I do that? What’s the ideal project for using Trial on? And the thing that Trial lets you do better than pretty much any other test suite that I’ve seen is it lets you have sidecars that are things that your tests might need to communicate with externally. So, for example, I have worked on multiple projects where you need a database. And of course, one way you can do that is you use SQL Lite. But for a lot of production sites, if you’re using Postgres, you need a Postgres to test things for real. And how do you get that? Well, you could start one up. You could have like a service that’s running in the cloud which lets you get a new database when you need one. For testing.
00:29:08 [Glyph] You could run in Homebrew or something. But Trial lets you spawn a process in the background and kind of keep it in the real reactor and then have tests which do do that, like full on Real IO back and forth which allow you to communicate with database or a caching demon or something else that you can even write synchronous tests that do this. Like the process is running in the background and all Trial is doing is kind of consuming its logs and managing it.
00:29:40 [Glyph] It’s not trivial to do this, but it does because it’s running with a real reactor in every process where tests are running.
00:29:48 [Glyph] Because you’re doing this with rather, Trial itself supports parallel testing and can start up multiple processes for you.
00:30:00 [Glyph] It lets you do this type of coordination in a way which is a little more production quality, I guess, is how I would describe it. Whereas things like even pytest Twisted tends to really want to clean everything up for you, and Trial will also try to clean stuff up for you, but you can use the appropriate reactor APIs to say like, okay, here’s my continuously running background service that I’m using for all of my test and code. Each test, I will remove it from the reactor, check that everything else is clean, and then put it back.
00:30:27 [Glyph] That’s not quite as trivially straightforward as using those test methods that interact with Deferred that I mentioned. You definitely do need to know a little bit about Twisted, but I have worked on projects that use Trial in that way that test the code with Trial and that use Trial and Twisted facilities from managing aspects of the test process that don’t actually even use Twisted internally at all. Okay, it’s a qualified recommendation. This is mostly for people who like, if you know Twisted already, this is a thing you can use it for.
00:31:04 [Glyph] It’s probably not worth it to fully spin up on Twisted and Trial, which is a little bit of a learning curve just so that you can test something that needs like one background process.
00:31:17 [Glyph] But nevertheless there’s a lot of other reasons to learn Twisted and once you have that’s a cool thing you can do. So hopefully Trial is going to be getting some attention in the coming year. Its support for coverage is very outdated and most people just use like coverage start subprocessor some of the other Pi packages for dealing with parallelism.
00:31:38 [Glyph] Hopefully we’re going to make that more native. So maybe Trial will be one of the pieces of Twisted to see more maintenance than most in the upcoming year.
00:31:45 [Brian] Okay, but there’s a pipe test Twisted, so I can just.
00:31:50 [Glyph] Yeah, if you like Pytest. Twisted is very well maintained. I’m a weirdo in the Twisted community for not liking Pytest. I think probably most applications that use Twisted do get tested with Pytest Twisted and that uses components of Trial internally.
00:32:06 [Glyph] It is a different philosophical approach to testing, but it’s not like there’s no communication between the communities. There’s quite a lot of back and forth.
00:32:13 [Brian] So Twist is not your whole life?
00:32:20 [Glyph] Oh, very much, no.
00:32:21 [Brian] So what other sort of projects are you excited about that you’re working on more actively?
00:32:30 [Glyph] I have to be a little bit mysterious about what I’m working on right now. I’m kind of on a sabbatical. Okay, I’m on a break right now. I am doing a lot of writing. I’m trying to write more about code on my blog.
00:32:44 [Glyph] Unfortunately, in the world in which we live, there are a number of hot takes on there as well. So that’s not all Python. But yeah, I’m really interested though, just speaking broadly in getting Python into more places. That how best to put this that empower people and then empower kids.
00:33:12 [Glyph] There was this vision when I started in Python Low these many years ago.
00:33:19 [Glyph] The big project that Guitar was always talking about was CP for E, the computer program for everyone project. And that kind of fell by the wayside, largely because it was actually like a funded research effort that he no longer works on.
00:33:33 [Glyph] But I find that as a very interesting thing to try to push forward. And I find that the community in general is not always pushing it. Like we are often concerned with production quality enterprise code that runs on the back end and does like big, serious data science or infrastructure with Python.
00:34:01 [Glyph] And that stuff is all great. That’s my career largely speaking, but I wouldn’t want to say that I would diminish that at all. But there is a surprising dearth of popularity of like programming your own computer with Python.
00:34:20 [Glyph] Like most people don’t realize a lot of us use Macs. Mac OS is very popular in the developer community.
00:34:29 [Glyph] Most people don’t realize the number of things that you can automate with Python on a Mac. And so I’m doing some stuff trying to I have a lot of scripting projects which I’ve been using for years, kind of just for personal automations that I’m trying to clean up to publish.
00:34:47 [Glyph] Pi game, I think is also very interesting.
00:34:51 [Glyph] I’m working on a game, among other things right now, kind of in the background, and hopefully that will see the light of day at some point. But I also feel like that’s somewhat neglected as well. Like, there’s a lot of work that’s going on in the Pie game community to help teach kids just how to make a very basic game. And this is what I’m talking about with empowering kids, not just teaching them to code, but teaching them to say like, oh, you made a game, let’s get that on Steam. Why can’t you publish that and sell it and get it to your friends on Windows and Linux and Mac?
00:35:24 [Glyph] So I’m also doing some work partially in facilitating my own game development, but also with an eye towards I’m not just going to make an artifact that I ship. I really want to work on some of the build system stuff that makes it easy for people to go from. I took a Pie game tutorial course. I learned just enough to get something working. But how do I go from there to production and not in a way which like, here’s your 27 step Python tutorial that last four and a half hours. That’s like for enterprise software developers. But what if I was like eleven and I wanted to do that? Exactly.
00:36:06 [Brian] That’s a good example because we think about event driven systems as these complex enterprise things. But games, we want to teach kids how to play games and they’re totally event driven.
00:36:18 [Brian] So interesting.
00:36:20 [Glyph] Yeah. I feel like another thing that I’m interested in right now is browser Python.
00:36:25 [Glyph] There’s some very exciting developments in that space.
00:36:28 [Glyph] Pyodide the thing that lets you have a Python, a fully functional C Python Jupiter kernel running inside your browser in the back end, Python, which actually has a clone, like an implementation of Pi Games API. It’s very incomplete, but lets you hypothetically run a Pie game game in the browser. So one of the things that I’ve probably looked at the most and I have yet to manage to transform this into a project, but I would really like to figure out how to get one of the browser Python things to work with.
00:37:07 [Glyph] An easy to articulate tool chain that lets you get from I wrote a Python program to I deploy it in a browser because right now that stuff like there’s tons of really cool technology.
00:37:16 [Glyph] But if you want to get a Python back end up into a Docker container, it’s like write a requirements text. Here’s a standard Docker file. It’s like three lines long. We’ll get you 95% of the way there. And there are tutorials that will help you with whatever else you need. And the Docker run and you’re good to go. The question of like, okay, I have a Python program. I want to put it in a web browser.
00:37:37 [Glyph] Do I need web pack? How do I get the packages installed? How do I host this? There’s a whole universe of questions which are possible to answer, but there’s nothing that really brings them together. There’s nothing remotely like poetry for a front end Python thing.
00:37:50 [Brian] I think those are great things to talk about. And so for people that want to keep track of what you’re talking about, your blog again is where Glyph Twistedmatrix.com. Okay, I’m looking at the right one.
00:38:04 [Glyph] Then you can also follow me on Twitter. I’m Glyph.
00:38:09 [Glyph] Hopefully it’ll be more on the blog than on the Twitter feed at some point.
00:38:15 [Glyph] It’s a constant psychic struggle to make that happen.
00:38:21 [Glyph] And I’ve done a little bit of blogging about some techniques that I’m experimenting with in Pi game.
00:38:27 [Glyph] The build process question is a big one, and there’s probably going to be a bit of time before I have anything public to share in that area. But it’s definitely something that is going to require work for more than just me. Like, we’re going to need to have a community wide effort to make that work.
00:38:47 [Brian] I’m going to get tomatoes for asking this, but any inklings in the by game community to maybe upgrade the logo?
00:38:59 [Glyph] That is well outside of my control.
00:39:01 [Brian] Okay.
00:39:03 [Glyph] I would definitely endorse any effort to do so.
00:39:07 [Brian] Okay.
00:39:09 [Brian] Anyway, I’ll leave that aside. Well, I really enjoyed talking to you today, Glyph, and thanks for coming on the show and just think.
00:39:20 [Glyph] Thank you very much for the opportunity.
00:39:22 [Glyph] And yeah, I really hope that I think that Twist is cool. You should definitely check it out if it’s something that sounds like it might be useful to you, but I really do feel like that some of those techniques with the way that Twisted does testing of modeling a system as just like an object that you just call your methods on and that all lives in memory, but is relatively complete, is really a testing strategy that I think applies across the board that I’d love to see more people investigate.
00:39:55 [Brian] Yeah. And I’m sure other Asynchronous methods outside of Twisted could do something like that too. And if not now, hopefully somebody will listen to this and think, let’s make that happen.
00:40:07 [Glyph] Yeah, it definitely would apply. It would map pretty cleanly onto Async. Io, and I think there may even have been some efforts in that direction.
00:40:13 [Brian] Really.
00:40:14 [Glyph] The thing that I think I owe is you would just need a full in memory event loop to hook up to your features in order to make it happen.
00:40:21 [Brian] Yeah.
00:40:22 [Brian] Cool. Okay.
00:40:24 [Brian] We talked about a whole bunch of stuff. I’ll try to make sure I put links in show notes and everything, so we’ll talk to you later. All right.
00:40:31 [Glyph] Thank you very much.
00:40:37 [Brian] Thank you, Glyph. It was a very interesting conversation. Thank you, Rollbar for sponsoring Rollbar enables developers to proactively discover and resolve issues in their code so they can work on continuous code improvement throughout the software lifecycle. Learn more at rollbar.com thank you, Patreon supporters join them at testandcode.com support all of those links and lots of links from Glyph are in the show notes at testandcode.com. That’s all for now. Now go out and test something you.