Interview with Thea Flowers, a Pythonista and open source advocate. We start the conversation talking about a custom synthesizer she’s building from Sega Genesis chips. We also talk about nox, urllib3, and PyCascades.

Transcript for episode 59 of the Test & Code Podcast

This transcript starts as an auto generated transcript.
PRs welcome if you want to help fix any errors.

Welcome to Test and Code, a podcast about software development and software testing.

On today’s episode, we have Thea Flowers. She’s the creator of Knox, the command line tool that automates testing in multiple Python environments. Similar to Talk.

She is co chair of Pikascade 2019.

She’s the lead maintainer of Urlb Three, the most downloaded package on Pi.

She’s a member of the Python Packaging Authority and Packaging Working Group.

She works on API client libraries and community outreach for Google cloud platform.

And all of that is very cool, and some of it will talk about in this interview. But first, we’re going to geek out on a cool project she’s working on to make a synthesizer module based on Sega Genesis chips. I think you’ll really enjoy it. Thank you to Digital Ocean for sponsoring this episode. They’ve been so awesome in sponsoring this last bit of 2018, and I really appreciate it. Check them out at Test And Code. Comdigitalocean.

So today on Test and Code have invited Thea Flowers and Thea thank you for coming on the show.

I’m super happy to be here. Thank you for having me.

And there’s a ton of stuff we want to talk about, but before we get into it, I really want to nerd out about Sega Genesis and stuff. So introduce yourself. Who are you? Yeah.

So I’m Thea Flowers. I am a self proclaimed type initia. I work at Google on their developer relations team. I do all kinds of stuff. As you mentioned earlier, I’m a little bit everywhere in the Python community, from making things like Knocks to maintaining your Lip Three to working with the Python Packaging Authority. I’m kind of all over the place.

And we’ll touch on this at the end. But you’re also involved in the Pycascades, right?

Yeah, I’m the co chair for 2019.

Okay. I’m used to seeing your name, like, all over the place recently with a lot of the stuff that was going on recently with the Python packaging and all of the and all sorts of stuff like that. However, what really intrigued me and wanted me to have you on the show is you’ve been playing around with Sega Genesis.

I just have briefly looked at some of your there’s, like three blog posts so far on this, but tell me what’s going on with you and Sega.

Yeah. So my blog posts are hilariously behind on where I’m actually at with this synthesizer.

Okay.

But I recently got into, like, hardware hacking about a year ago, and I got into it because I love to tinker with music. I used to be a professional musician, and I was never very good at it, but what I was good at was being the guitar tech for my band and making sure everything was running smoothly. And that sort of tinkering attitude sort of transferred over later in life when I decided to start playing with synthesizers. So I always wanted to build one, but I never found the inspiration, right? The ones that I had or enough fun to play with that I never felt like I needed to actually build one, but I wanted to. And then I had this crazy idea of, oh, I love the way that music sounds in the Psychogenesis. Maybe I could build a synthesizer using either a Genesis directly or using the sound chips from the Psychogenesis itself. And so that’s what I ended up doing was using the sound chips directly and talking to them from a microcontroller and being able to create a Midi controlled synthesizer using those sound chips.

Where did you get Genesis sound chips?

It’s actually pretty interesting. So Yamaha so the sound chip inside the Genesis is the Yamaha YM 2000 and 612. And Yamaha produced, like a huge broad range of these chips, like, from everything to really small toy keyboards that you would get at Walmart to sophisticated production ready synthesizers like the Yamaha TX Seven, which you can hear in hundreds of tracks in the 80s, including Michael Jackson and stuff. So Yamaha produced this huge range of chips, and they were using all different kinds of places. The thing is, they definitely don’t produce this chip anymore. They don’t produce the Y in 20, 612 anymore, but they produce enough of them back in the 80s, and all of our jump gets shipped over to China and they recycle it. They extract what they can that’s valuable, and then they resell the parts, essentially.

Okay.

So it’s actually not hard at all to get your hands on a Ym 2000 and 612 that’s either harvested for Mega Genesis or one of the Yamaha keyboards or one of the arcade cabinets that it lived in.

Since I’m not a music nerd, this chip, how is that affecting the sound when you make a synthesize of this? Does it sound like an old Genesis game or it does.

So what’s super cool about this chip in particular and about the second Genesis sound is that it’s kind of like the last of its kind and also one of the first of its kind.

Way back in the day, CPUs were too slow to do real time audio playback. I’m sure people listening to this podcast are enjoying real time audio playback as they listen to it. Old consoles couldn’t do that. It just took too much memory, too much CPU power. So they had dedicated chips that basically the CPU just told it a note and how long to play, and the chip would go and play that note, and it would have like one or more sound channels. So you could play multiple notes. But the ones in the entertainment system and the Game Boy and stuff like that were very simple. They could do square waves, which, if you can remember the Super Mario theme in your head, it sounds very much like that. However, the Genesis kind of took us to the next level. It was like, okay, we can do something like that where we have a ship that you tell it notes and duration, but let’s also make it where you can do all different kinds of tombrows at the same time. And so the idea here is to use FM or frequency modulation synthesis, which is exactly what you would hear in, like, the ads and things like that with those synthesizers and stuff. A tiny FM synthesizer essentially into the console so that the console can essentially synthesize the sound in real time and play it back. So you kind of, like, have this amazing six channel synthesizer living inside of your Sega Genesis.

That’s amazing. I didn’t know that.

And then the PlayStation came out, and then we had audio playback and all this stuff, and we didn’t need it anymore because then you could just record whatever instruments you wanted to and throw it on the audio track. But the Genesis was like this awesome limitations, which tend to breed creativity. And so it was really cool to have that sort of chip. So that’s why I’m very excited to build a synthesizer using that distinctive voice of that chip.

This seems like all very hardware and stuff, and you’ve got, like, circuit diagrams and stuff like this.

Like you said, not up to date blog posts, but did you do hardware engineering at some point, or is this stuff you just learned on your own?

No, I never went to College. So this is all just me guessing? Essentially, yeah.

The hardware here is not excessively complicated. It’s a mix of a digital circuit, which can be relatively easy to work with, and an analog circuit for the audio amplifier, which is actually the most difficult part for me. So, yes, no background in electrical engineering. What I did was essentially, like, just stitch things together from the Internet. I found a Sega Genesis hardware mod for better audio, like clearer audio, and I took the schematic for that and just recreated it in keycad and then had, like, protected on a little proto board and then send it out for manufacturing. When I had a PCB ready and actually learned how to make PCBs for this project, which is really interesting and fun, but the rest of it, like talking between the microcontroller and the sound chip. Like, the hardware part of that’s really easy. You basically just wire up eight lines from one thing to the other. The really tricky part is talking to it via software because there’s almost no documentation for this chip. Okay, yeah, a lot of that was reverse engineering. And looking at old Sega Genesis emulators to see how things worked.

And probably a lot of trial and error.

Oh, my God. Yes. Holy crap. So much trial and error. I’m still not 100% certain about my timing calculations on the thing, but it works well enough, and it seems to work in real time when I’m pressing notes.

So I guess it’s okay now what’s the end goal here just to have a full keyboard based on this or what?

Yeah, it’s one of those things where I got, like, halfway through, and I’m like, what am I actually going to do with this when I’m finished, other than just have a cool story to tell? Yeah, I don’t know.

I do want to have a synthesizer that has that unique voice and be able to basically render some songs using this specific hardware. But also I wanted to see about needing new life into Sega Genesis tracks, because when you have a separate synthesizer and you can kind of give it all the attention of one instrument, like a lot of stakeholders tracks, they all had to play on the synthesizer. All the instruments had to share the synthesizer, essentially. So you have six channels. So you can basically have six instruments playing that have one note, or you can have one instrument with two notes and then four with one note. But when you have the full six voices available to you for one instrument and you can do multitracking like you can today, then you can add effects and you can do really interesting stuff with that synthesizer sound that they couldn’t do on the hardware itself, but still using the original hardware. So one of the things I thought about doing is like rendering some of these old tracks that I love using the synthesizer, but also with modern production and effects and things like that.

That would be cool. Okay. We’ll keep an eye on it. So where is it at now? What’s the state of things?

Yeah, so hardware wise, I’m basically done I have printed circuit boards for every major component that work, and I’ve assembled, so they’re working great. I have an enclosure that I’ve designed, and I’ve cut out of cardboard at this point, so it’s sitting in cardboard, but I’m going to actually laser cut it out of Acrylic, so it has a nice, beautiful case to sit in. So hardware is done. The software is where I’m not quite done yet. It’s actually completely playable and actually pretty fun to play with right now, which is one of the problems. Every time I go to work on it, I get distracted and playing with it for an hour. But I need to add a few more features, like some, like switching between monophonic and polyphonic mode, saving and Loading patches, which you can load patches right now, you can’t modify them and then save them back.

Okay.

But a few other things I need to do as well. But, yeah, I’m really closing into the end here, which, again, as I get closer to the end, it gets harder and harder for me to work on it because I just want to play with it. So, yeah, it’s super fun, and I’m very close to being done.

Okay, cool. I guess maybe I’ll just ask you on the air here. Is it going to show up at a Python anytime so we can hear it in person.

I would love to. It’s very portable, so I could definitely bring it along. I thought about how to do a talk on it, and it’s actually really hard to do, like a 30 minutes or 1 hour talk on something like this because it’s so complicated. So I need to find a way to break it down into something that’s, like easily comprehensible in 30 minutes or 45 minutes and give a talk on it somewhere, in which case I’ll bring it and actually demo it for people because it is actually really cool to see this thing work.

Okay, well, I’m going to put in just a plug or a request for even just an open session where we could just sit around, ask questions and listen to it and stuff like that.

Yeah.

So how do you play it right now? Is it just all software based or is it hooked up to a keyboard?

It’s a Midi module. So lots of synthesizers, especially like Irn synthesizers, will come in with a keyboard format. So they have a big keyboard and they have all the little knobs at the top. A lot of them will also come in what they call module format, which is just a standalone thing. It just has all the knob parts, but none of the keyboard parts. These are great because if you’re in a big studio, you would have, like, one MASTERMINDI keyboard, and you would have all these other modules hooked up and wired in. And essentially you could use software to select which one you want to actually play. So I’ve designed this as a module so you can get any keyboard up to it then. Yeah, exactly. And I’ve also thought about at some point actually embedding it into a keyboard to make it even more portable.

Okay.

Yeah. So we’ll see. I haven’t decided what I’m going to do with that yet, but I can move it either way at this point.

Well, now I’m jealous, and I kind of want to get into music and keyboards and stuff like that because that just sounds like fun and a complete waste of time, but not in a bad way. Just a fun hobby.

It’s definitely an intense hobby. And be prepared to just set fire to a stack of cash because instruments are expensive.

Well, I guess probably sending out for printed circuit boards and stuff like that probably isn’t super cheap either.

Oh, my God, it’s so cheap.

So I had three boards made for my synthesizer. Each one costs $5 for five boards through China. There’s like PCV Way, and some others have just prototyping services like this.

Okay.

There’s limitations, but it’s $5 and then like $12 shipping, so it comes out to about $24 or something like that. I can’t remember exactly with taxes and everything. And then they show up about a week later, and then you’re good to go.

I thought it’d be like hundreds or something.

No, it’s amazingly cheap. I was really surprised myself. I was like, what?

Wow. Okay.

Thank you to Digital Ocean for sponsoring this episode. Digital Ocean is the preferred cloud platform of hundreds of thousands of innovative companies. Digital Ocean makes it easy to deploy, manage and scale applications with an intuitive control panel and API designed for developers. Get started with a free $100 credit towards your first project on Digital Ocean and experience everything the platform has to offer such as cloud firewalls, real time monitoring and alerts, global data centers, object storage and the best support anywhere. Join over 150,000 businesses already creating amazing things on Digital Ocean. Claim your credit today at testandcode.com, DigitalOcean.

Okay, so I’m going to just harshly apologize to everybody listening, change gears completely, and ask you about some other stuff. So one of the things that you’ve done is Knocks.

No X. So Knox, I’m assuming is something kind of like Talks. So tell me more. What is Knocks? Yeah.

So Knox is very much kind of like Tox is like this direct spiritual ancestor to Knox.

Knox is a test automation tool. Just like Tox, it creates virtual Ms and lets you install things into it and run commands. The big difference between Talks and Knox is that Tox uses this any file for configuration, which is totally cool. Nox uses a Python file for configuration, sort of like fabric or Pi invoke, and that gives you a lot more flexibility. So you kind of imagine a continuum, right, of a project complexity where on one end you can use Tox out of the box, which is fine for you. And then in the middle somewhere is Knocks because you need a little bit more advanced things and then all the way on the other end of the spectrum is things like pie and folk and make and things like that, where you basically are creating a bespoke automation tool for your project. So like Knock sort of sits in the middle of that little specs from there and it’s actually really awesome. It’s been amazing for me at work. So Google manages 50 or so client libraries for our products, and we also manage around 150 samples for things. And having a tool like Knox that can help test that Hydra of clients and samples, it’s really useful. So that’s actually where it came from. We were using Tox for a long time and we ended up with this talks any file that was absolutely crazy. I think it was like 200 lines of any code and then a bunch of helper scripts and we were like, this is wild. Let’s take a step back and see what we can do. And we ended up with Knox and then we open sourced it, or rather I open source it and then the community sort of jumped in on that and it’s got several contributors now and has been adopted by several projects. So it’s really neat to see that grow.

Okay, so that’s cool.

One of the things that I’ve done with Docs is like a matrix style test set up. So can knock do matrix testing as well?

Yeah, NOx is actually really awesome at this. We basically stole pipes’parameterization thing if you’re used to that, which is like kind of an advanced feature of pytest. But basically because each NOC session is a function, you can add the parameterization thing on top of it and basically create a test matrix out of all the parameters you specify. And Knox will create sessions for all of them. And just like with talks, you can individually address sessions in the matrix, but you can also just run the top level session and run every session in that matrix, which is really cool.

Okay. One of the things I’m looking at, just at the home page of the Knox documentation, it shows a couple of decorators use like knocks session and then a function inside there. Do each of these have, I guess the output of this? Can I use it in continuous integration? Does it have a past fails sort of thing then?

Yeah, absolutely. And that’s the thing that’s super important to us when we were creating Docs is like not only do we need this to run under continuous integration and say pass fail based on sessions, did all the sessions pass or did any of them fail? That’s sort of the default mode, but we also have it where it can output adjacent file that has a report of every session, including its output and its status and whether or not it passed or failed. And we actually take that output in RCI systems and transform it into an XML format that’s ingested by Google’s internal systems that can give us a dashboard and stuff like that. So it’s like really powerful in that way in terms of reporting and things like that.

Is this fairly stable and done, or is there a future roadmap to take it further, do you know? Yes.

So I think the core parts of it are pretty done. We had a big major breaking change earlier this year when I swapped around how sort of Knox’s internal engine worked from imperative to declarative, or rather the other way around, Declarative to imperative. I can talk more about that later, but essentially now the core features are done, right? The basics are never going to change. Like you have session functions you call session run or session install.

All that is never going anywhere. We have some ideas for new features such as breakpointing and things like that. Breakpointing actually would let you set a Breakpoint in your knock session so you can use the Python 3637 Breakpoint function and it would actually drop you into a shell in that virtual and you can then debug and do whatever commands you want to within your virtual and then you can exit and that session would continue to run that’s something we’re planning on doing, and I’ve been experimenting with, which is really interesting and challenging, but fun. But yes, for the most part, we’re mostly just going to see new features. Not breaking noxious interface is in my best interest because when I do, I have to go and update like 100 different projects that I own.

Okay.

So that’s a huge motivating factor for me not to break things.

Okay. It’s definitely being used regularly now, at the very least by you and other people. So it’s going to live for a while.

Okay. Yeah. Okay. The other thing now, I don’t even know how to do this since you do so much other than doing harsh transitions. You told me that you are the lead maintainer for URL Three.

Yeah.

What does that mean to be the lead maintainer?

Good question.

With any project, there’s like different kinds of maintainers, right? And you’re the Three is an interesting project. It is, by the way, the most downloaded Python package, which is scary to think about me being the lead maintainer or something like that. But effectively, your Three has been around for a long time. It’s had several lead maintainers, which is a huge testament to Andrew, the original creator’s ability to find more people to take over his project. Like that is such a rare skill to see an open source. Most of the time he’s just got abandoned. So that’s awesome. But effectively, my job as lead maintainer is to because there are other contributors and there’s like Steph Michael Larson, who is effectively the current maintainer of the project.

My job is basically just to handle administration, like making sure that we have everything turned on that we need turned on and settling any debates or disputes or anything like that, and helping out with code reviews and things like that. In terms of actual code on your Three, I do very little. But in terms of administration, like accepting sponsorships or financial contributions or helping onboard new contributors or code review and things like that, I help out a lot.

Okay, now what is your Lib Three?

Is that something I have to install then? It doesn’t come with Python, right?

Yeah. So if you probably use your library and don’t even realize it because it’s the underlying Http library that Request uses. So Request is like this beautiful porcelain on top of your Lip Three. Not that your Lip Three has a bad interface, but Request makes it that much easier. And so Pip uses your Lip Three through Request, and then almost every project that uses Request uses your Lip Three as well. But yeah, you’d have to Pip install it as a third party library, so to speak.

Okay. Would I want to use it directly or is it best to use through Request?

Like most users would want to use it through Request? However, there are certain cases where you want to use your Live Three directly, or at least like drop down in Request to the year of three layer and configure some things. So when you think of things like using Proxies, sometimes you have to drop down to the year of three layer if you want to use different types of protocols and things like that, Yearly Three can help with that because it’s not just an Http client. If you want to interact with things like weird environments like things that have limited access or has to access Http through a weird interface like microcontrollers or things like that, like Yearly Three has a pluggable interface to add an extra connection benefit for that. If you want to change how connection management works, like if you want to make the connection pool 100 connections or one connection, you do that through configuring your loop three.

Okay.

Even if you’re using requests now.

What is Y three?

Yeah. So it’s actually kind of a funny thing about the sort of naming conventions of Python HD libraries. And I think this is maybe one of the reasons why Request is so successful is because Request went with a really easy non comedic name for it. So it’s very obvious thing like, oh, what do I use to make HP requests? I use requests. Your Lib Three was kind of born out of this thing where we had in the standard library Http Lib and we had your Lib and we had your Lib Two and all these lived in the standard library and it’s kind of like these are all building on top of each other. And then at some point someone released Http Lib Two, which was a third party library. And I guess the creator of your live three, Andrea was like, okay, let’s create one that can do thread safety and session and pool management and we’ll call it your Lib Three and it just stuck and then it’s too late to change it.

Now is it built underneath it and how it’s implemented? Is it using Urlb two then underneath?

No, actually, you’re actually testing my knowledge of how deeply I understand. I believe it uses the socket interface directly.

Okay.

It very possibly uses things like some of the standard library components to do serialization and things like that. But I think for the most part it’s a complete replacement and doesn’t depend on those things because some of those libraries are actually removed in Python Three.

Okay, it’s very interesting.

Yeah. Actually I’ve admitted on Twitter just recently that even though as far as being a web developer, I’m not a web developer, but I touched on it since College. So way back when dinosaurs roamed the Earth and Mozilla was an actual thing, I guess it still is. But when it was the browser, that was when I started and we did things like Perl scripts in the back end and hand coded HTML, I love it. I wrongly assumed that web development was a bad thing to get into because eventually that was going to be like a minimum wage job. And so I chased hardware and stuff like that and got into C Plus Plus, but I’m getting back into it now, and I kept track and learned PHP for a while and WordPress development and stuff. But in the Python world, I’ve mostly used Flask, and now I’m learning Django. So all of this web stuff just sounds interesting to me. And it’s fun.

Yeah, there’s a lot of things to learn. It’s really interesting, and it’s a huge part of the Python community. Yeah. There’s so many resources and so many things to learn. I kind of love it.

Speaking of Python community, you’re also involved with podcastcades.

Yeah. So I’m the co chair for 2019, along with Alan Zina, and we are so excited for podcast AIDS. It’s going to be super awesome.

No, it’s coming up here just in February, right? Yeah.

February. Oh, my God. I don’t want to say the date wrong because I always say the date wrong. I think it’s the 23rd and 24th. Hold on, I should know the date of my own conference, right? Yes, February 23 and 24th in Seattle.

Okay. I have tickets. I don’t have a hotel yet, but I have tickets. So I’ll go even though you didn’t accept my talk. But I do like, it’s interesting, actually. It’s kind of cool to have, like, a whole bunch of different types of venues because one of the things I thought was interesting about the Pike Cascades talk proposal process was, is it a single track?

Yeah. So that was part of the whole difficulty with selecting talks is that we had 250 submissions. We’re a single track conference that happens over two days. So we have 20 talk slots.

So it was very hard for us to make those final decisions on what talks were actually going to get in because we had so many amazing talks and just 20 slots.

At least for part of the selection process.

You don’t look at who’s presenting, right? You just look at the proposal text.

Yes. The first half is totally blind and I think totally blind, but there are some cases where people actually put in identifiable information in their comments, and we’re just like.

And then there was a couple where we definitely, like, we could tell who it was just from their tone of voice, because it’s people we know and work with, and we’re like, oh, I know this person, and we actually have a special process for selecting out and saying, I have a conflict of interest on this talk because I know the person.

So.

Yeah. So we were very serious about, like, the first half of the selection process must be blind so that we can do a better job of selecting the diverse set of speakers and talks.

I’m super looking forward to it because I didn’t go last year, but I heard tons of good things about it.

Yeah. Last year. I was a sponsor. I wasn’t even part of the team or the volunteers. I was just there representing Google, and I had such an amazing time. And I was like, so excited when they invited me to be the co chair this year because I was like, this is like a dream come true because I love this conference and it was super basic. I can’t wait to put my own spin on it in some ways.

But being a coach, it’s extra work, right?

Oh, yeah, it’s extra work. I’m a bit lucky here, and I will admit to being not as big of a contributor as I would have hoped to this project. Like Alan and the other organizers are absolutely incredible, and they have done the majority of the work. I’ve helped out where I can with the selection process. I’ve helped out with content and marketing and all that stuff, which is what I’m good at. But in terms of the actual business side of this thing, making sure everything is running well, the other organizers are so amazing. Nobody could do this sort of thing by themselves. And it’s really amazing to be working with these people.

Okay, that’s cool.

Well, I’m really looking forward to doing that. It’d be fun not doing that. Meaning going to the conference, I’m not going to be participating in helping put on a conference. That just seems like just so much work.

Well, you are welcome to volunteer to get a little taste of it.

So what’s going to be in Portland next year? I think, right.

That’s the plan. The idea is to be in Portland and to rotate between Seattle, Vancouver and Portland.

Yeah. Actually, even though I’ve lived most of my life within a day trip from Canada, never been to Canada.

Oh, it’s lovely. I’m originally from Georgia, so when I moved to Seattle, I was like, oh, I’m so excited to go to Canada for the first time. So, yeah, Canada. Vancouver, especially is beautiful.

Okay, so we talked about the Genesis project, which I’m really looking forward to, playing with Mjoralb, Three, Knox, Pycascades, and any of these. We could have jumped in and done like a deep talk about. So I’m actually just going to open it up to people listening. If there’s aspects of any of this that you want us to get back together again and talk about, then let us know or have questions.

And if anybody is interested in this stuff, where should they go to find out more about you or more about what you’re doing?

The best way to find me is to go to my website, which is via Co so DS that has things to my blog, my Twitter, where you can tweet at me. If you want to do that. You can also email me if that’s what you’re into. I also have calendar set up, specific. So calendar is like this scheduling service for those who don’t know. I have one set up specifically for open source stuff. So if you want to talk to me about anything related to open source or Python, you can use that actually have it turned off for the rest of the year so I can have a vacation but I will be turning it back on in January so people can do that.

Find me tweeted me, email me. Find me on GitHub. Yeah, just via codes.

All right. Awesome. Well, thanks so much for coming on and yeah, have a good Christmas.

Yeah, you too. Thank you so much.

Thanks to Digitalocia for sponsoring grab your $100 credit at testandcode.com DigitalOcean that link is also in the show notes at testandcode.com 59 thanks to Thea for talking with me today and thank you thank you for listening for sharing the show with friends and colleagues, for pitching in with the cost of the show through Patreon, for giving the show a rating on itunes and for checking out Digital Ocean through the link in the show notes. Thanks again to Marco for the audio help. He and Digital Ocean are the reasons you are hearing more of me lately. That’s all for now. Go out and test something you.