virtualenv supports six shells: bash, csh, fish, xonsh, cmd, posh. Each handles prompts slightly differently. Although the virtualenv custom prompt behavior should be the same across shells, Brian Skinn noticed inconsistencies. He set out to fix those inconsistencies. That was the start of an adventure in open source collaboration, shell prompt internals, difficult test problems, and continuous integration quirks.
Transcript for episode 130 of the Test & Code Podcast
This transcript starts as an auto generated transcript.
PRs welcome if you want to help fix any errors.
00:00:02 Virtual M supports six shells Bash, Seashell, Fish, Conch, CMD, and Pox. Each handles prompt slightly differently, although Virtual M prompt behavior should be the same across all shells. Brian Skin noticed some inconsistencies. He set out to fix those inconsistencies. That was the start of an adventure in open source collaboration shall prompt internals difficult test problems and continuous integration works. This is his story. This episode of Test and Code is brought to you by Talk Python Training. Do you want to get better at Python? Now is an excellent time to take an online course, whether you’re just learning Python or need to go deep into things like APIs and async our friends at Talk Python Training have a top notch course for you. Visit Talk. Python. Fm Test to find your next level that’s Talk. Python. Fm Test and also stick around until the end of the show for a chance to win a free course.
00:01:14 Welcome to Test and Code.
00:01:24 I am thrilled today on Test and Code to have the first Brian I’ve ever interviewed. I think I went and looked I was like, I can’t believe that. I don’t think there’s any other Brian’s that I’ve talked to. So welcome, Brian.
00:01:35 Thank you. Glad to be here.
00:01:37 Brian Skin, right?
00:01:38 Yes, that’s right.
00:01:39 And could you introduce yourself to everybody?
00:01:41 Sure.
00:01:43 So I am not a developer by trade. I’m actually a chemical engineer. Right now I’m doing research with electrochemical engineering processes, but that is more detail than we need to get into. I’ve been programming for a long time. I think technically the first I did was the original Basic on an IBM PC in the late 80s. That didn’t grab me too much. But then in high school, I got a Ti 82 and realized that I could automate my algebra and calculus homework. And that was it from there.
00:02:18 So I had one C plus plus class in College. That was about it, and it was pretty much all Excel. But in College, I had an internship where my responsibility was not chemical engineering directly, but the company that I was working for had this big archive of Excel sheets that they had these big furnaces that they use to grow crystals for optics. And they wanted to use this data. They had to improve their furnace programs. And so I actually wrote a full fledged VBA GUI driven data analysis app to try to do this for them. And that’s really where the majority of my programming chops started out.
00:02:58 Fast forward, remind me what VBA is.
00:03:00 Visual Basic. The Excel Office VBA.
00:03:04 What does the A stand for?
00:03:05 Visual Basic for applications.
00:03:07 Okay, thanks.
00:03:08 Yeah. So all this was in the context of a chemical engineering bachelor’s, and then I went on to graduate school and took a quantum chemistry class. That was interesting, but that kind of lay there for a while. And then I actually tried to implement there was an analysis that I was trying to do that. I tried to implement a full fledged object oriented tool starting out in BBA. That was frustrating. And actually what I was working on, there was a professor who expressed some interest in it. So I said, okay, VBA is not going to work for this. So I started looking around, saw Python was in common use was open source, was easy to get to, and that was 2014, and I’ve been doing Python on the side and some for work ever since.
00:03:50 Nice. Yeah.
00:03:52 So I do it a little bit at work for some data analysis. But then I have a bunch of projects up on GitHub and a few on Pi, all of them at least the recent ones. I started out on unit tests from using pytest now. Awesome book, by the way. Got a lot of good stuff out of that. And so from there expanded into partly, eventually what we’re going to talk about today in terms of a handful of contributions to open source projects, various things.
00:04:22 I love Python. I’m enjoying what I’m doing with it, and it makes a great compliment to the engineering work I’m doing on a daily basis.
00:04:30 Yeah, I think I started paying attention. I mean, I was using virtual environments, I guess, when I started writing tests and stuff and definitely didn’t start using virtual environments when I started using Python, because I started a long time ago, I can’t even remember around 20. 00, 20. 01, 20. 02, something like that.
00:04:48 And I can’t even remember when I learned about virtual environments. But there was somewhere in the Python two, seven or 26 time frame, I’m sure.
00:04:57 And then Python three bundles, V en V. Yeah. And I thought this story was over. We didn’t really care about Virtual M anymore because it was bundled with Python. But we do, right? So Virtual M exists still as a project separate outside of Python, but before the main thing I noticed that was different was the prompts were different.
00:05:22 Yes.
00:05:23 But there’s other stuff different, too, like it’s faster. So personally, on my personal projects or personal use, I’ve switched to the virtual end mostly for the speed purposes, because it’s just really fast.
00:05:40 High PyCharm is the Python IDE for professional developers. High Charm’s huge collection of tools out of the box includes an integrated debugger and test runner, Python profiler, a built in terminal, integration with version control and built in database tools, remote development capabilities with remote interpreters, an integrated SSH terminal, and integration with Docker and Vagrant. In addition to Python, PyCharm provides first class support for various Python web development frameworks, specific template languages, JavaScript, Coffee script TypeScript, HTML, CSS, AngularJS NodeJS, and more. High Term integrates with a Python notebook and has interactive Python console and supports Anaconda as well as multiple scientific packages including Map, Lib, and NumPy. Try out all of these times, saving features of PyCharm Pro for four months with the link Test And Code Compai PyCharm, make sure your editor is working with you to save you time. Use PyCharm.
00:06:41 So how did you get into thinking about and helping with the virtual and project?
00:06:47 So it was basically the same.
00:06:50 The prompt was for me. Also what got me looking closely at the source. I definitely also didn’t just as a Sidebar, I definitely didn’t start using virtual environments either. I was like, I don’t even know what this is because actually one of the projects that I have on pipes GitHub too, I went to with Python three nine coming out. I went to try to rerun my test suite on it, and I don’t even have a requirements file or an environment set up on that folder. So I was like, oh, okay, I’m not going to test that until I get a virtual environment on it.
00:07:19 But ultimately, what first jumped out at me from Virtual and that got me looking at the source was the fact that I do a lot of work on Python, both on Linux and on Windows, and virtual ends. I don’t actually haven’t looked that closely at the end, so I don’t know if it has the same sort of thing, but virtual ENS provides explicit support for six different shells or foreign Linux bash, CSH, fish, and conch. It’s a Python driven shell, and then CMD and PowerShell and Windows. And so I was working in CMD and Windows and Bash on Linux, and I noticed that one of the things that again, I don’t know if Env does it, but virtual. And if you just say Python M virtual ends, it will create the virtual environment for you. And then when you activate it, it modifies the prompt and it gives you a little indicator that you’re in a virtual environment, but you have the option to pass the virtual prompt argument, which lets you customize that prompt. And I noticed that on Windows, when I would do that, virtual length would add a space between what I supplied per n project name close windows. Cmd would insert a space between what I supplied the prompt and the rest of my prompt, whereas bash would just jam the old prompt and my prefix together. It was just a discrepancy in the behavior. So I raised that as an issue on the repo.
00:08:46 And originally I was going to do what would have made you happy? I was going to revise Bash to automatically add that space in there. Like, yeah, I like that a lot better. That’s the way I use it. But then downstream in the discussion in the issue, I think it was Antony Satilly that pointed out that that’s actually a less flexible, a less general behavior, because if somebody wants to instead of surrounding the prefix with Parenza in a space, if they just want two equal signs on either side and have that, but it directly up against the old prompt, then they wouldn’t be able to do that if the space were forcibly added between him and myself. And Bernard, who is the maintainer that I was working with, was like, yeah, the more general behavior is not to insert the space. And so that was the direction, the decision that was made on that, which takes an extra character every time you instantiate the virtual environment. But it’s more general. And that was the way that decision was made.
00:09:43 Okay. Yeah. So I just have to live with it. But I mean, to be honest, I’m using it. I usually set up a little macro within or a function within Bash to create my virtual environments so that I can pass in all the flags that I like. And I just added that extra stuff to the prompt that I want, and now I don’t care. Works fine.
00:10:05 Yeah.
00:10:06 You wrap it in Tooling and you edit it once and you’re good to go for sure.
00:10:10 So notably now one of the differences between V and V and virtual hem is the difference of the space, I guess, for what it’s worth.
00:10:20 So again, this is where the story really I started peeling back the hood and realized it was even more complex than just a space or not a space.
00:10:30 Because when I look deeper, there’s kind of three things. One, when you just invoke virtual. And so it’s a little bit different now with Renine, where you have that fancy dot argument where it takes the name of the folder and stuff that into the prefix, which is really cool. But at the time, you either had the default where it just put Env in parentheses, or the name of whatever the name of the virtual environment folder you provided, they put that name in parentheses as the prefix. Or you could supply the prompt, then it would use the custom thing you supplied supposed to. And then kind of the third factor is there’s an environment variable that it pays attention to. At activation time, you go to activate the environment virtual and disable prompt separated by underscores. And if that is set and non null, then it won’t modify the prompt when you activate the virtual environment. So if for some reason you want that prompt modification to be silenced, you can set this flag and it will, in theory, at least not change the prompt on you.
00:11:29 Okay, so this is really cool, and I didn’t know about this, and I’m going to start using it.
00:11:34 Oh, really?
00:11:34 So thanks for letting me know.
00:11:36 Cool. I have not found a use for it yet. I could see it being valuable in certain automation circumstances, but I haven’t had any for it personally.
00:11:43 For me, it’s valuable when I’m using. I’m always using a virtual environment, but if I’m, for instance, grabbing a code sequence an example and dropping it in some documentation, I don’t necessarily have to care about showing that what the name of my virtual environment is.
00:12:01 Okay. Examples. Yeah.
00:12:02 Cool. I also think if I could change the world.
00:12:06 Also, the default of the name of the environment seems bizarre. Now when most people just for the most part, use a consistent name, they either use Env or Venv or Venv or something like that for all of their virtual environments.
00:12:22 Yeah, exactly.
00:12:23 So naming it, that seems bizarre. And that’s why I think the 39 edition of the Dot option that names the virtual environment after the other directory that it’s in the parent of the EMV.
00:12:36 No, that definitely makes more sense. I don’t know. I haven’t looked for it, but there may well already be an issue on the repo.
00:12:42 I doubt we’re going to change it.
00:12:45 I mean, there’s also there’s another camp of people that like to put all of their virtual environments in another directory, not in the directory they’re working.
00:12:53 Oh, sure.
00:12:54 Put them in some stored directory somewhere.
00:12:57 That’s what like Pipex and a lot of the other tools do is they have a common repository that they put them in.
00:13:02 Yeah. In that case, it makes sense to have them all have different directory names and different names and stuff.
00:13:07 So simplicity in generality definitely is the good principle to go by.
00:13:13 But you cared enough about this to go and work on it.
00:13:16 Yeah. So it was something that I saw that was like, well, so this is where the rest of the story really gets complicated. So we’ve got these three standard behaviors, and you’ve got these six shells that virtual and addresses. When I started looking at the behavior of the shells, all but two of them did not follow the nominal behaviors.
00:13:38 So Bash and Fish obeyed all three.
00:13:42 Once we decided that we were not going to insert the space after the prompt argument, Bash and Fish, we’re good to go. Csh ignored virtual and disabled prompt. It used square brackets instead of horrendous, and instead of replacing the bracketed default argument. When you use prompt, it would take the prompt content and stick it into those square brackets instead of completely replacing it. Cons ignored both the dashprompt and the disabled prompt. Windows CMD had the default prefix behavior, but also an ignored disabled prompt and PowerShell. Also, like conch ignored prompt, but it did obey. So there’s this complicated mix of inconsistent behaviors. I was like, all right, so there’s a little bit of perhaps an OCD. So I was like, this is messed up. I would like to see it all squared away. And so I went in and I was like, all right, here’s what needs to be fixed. And I put that into the issue. And they’re like, great, yes, please reconcile all of this. And I looked into the test suite and realized that part of the reason why this was possible, this inconsistency between the shells, is because there were no tests for this activation prompt modification on activation behavior at all. It’s like, oh, this piece is missing. I can add this. This will be good.
00:15:00 I’m a very material contribution to all this.
00:15:02 Yes.
00:15:03 And so it was just like going into it. If somebody had said, you’re going to get fascinated by virtual Lem’s prompt activation behavior, I would have given you a very strange look.
00:15:14 But once I started looking at like, okay, this makes sense, and I think I know how to fix it.
00:15:18 Sure, I’ll do this. Why not? And so then I started digging into both the I learned about all the different syntax of these different shells and got familiar with the Pi test fixtures and the existing tests and everything, and Dove into it with Bernard’s help to get it in place.
00:15:34 So to test this, do you have to launch all of these different shells? Then let’s face it, your code is going to have errors, even code written by a kickass developer such as yourself. And when bad things happen, it’s nice to know Honey Badger has your back. Honey Badger makes you a DevOps hero by combining error monitoring, uptime monitoring, and Crown monitoring into a single easy to use platform, all for way less than you’re probably paying now. Honey Badger monitors and sends error alerts in real time with all the context needed to see what’s causing the error and where it’s hiding in your code so you can quickly fix it and get on with your day. The included up time and Crown monitoring also lets you know when your external services are having issues or your background jobs go AWOL or silently fail. Go to Honeybadger. Io and discover how Starr, Josh and Ben created a 100% bootstrap monitoring solution. Why is this important? Self funding means they only answer to you, the developer, and not a venture capital. Overlord test and code listeners get 30% off for six months. Simply mention the test and code podcast when signing up and they’ll apply the discount to your account. No credit card required.
00:16:44 So to test this, do you have to launch all of these different shelves then?
00:16:48 Basically, yes.
00:16:49 Okay.
00:16:50 My test is running inside of a Python interpreter and all of the activate scripts are outside of Python. They’re shell native scripts.
00:16:58 Okay.
00:16:59 And so absolutely yes. So there was inevitably some sub process calls involved. And that was the first big question. Like, okay, I want to look at what the prompt looks like. How do I do that? And so the first thing framed it so that I was making it Python call or run out of the subprocess standard library module. And the first thing I tried to do was just to naively, as it turned out, thinking that the prompt renders into either standard out or standard error. I don’t know how it gets to the terminal, but it does not route through standard out or standard error.
00:17:37 Even if you do subprocess call with shell equals true, you can do Echo and blank commands and whatever all you want you do not get any rendered prompt into your standard out capture or standard error capture at all.
00:17:51 I think that makes sense because you don’t really want that for piping things together and stuff like that.
00:17:56 Sure, it makes sense in retrospect. It was like okay, that doesn’t work. So what I ended up having to do is for each shell, and the way Bernard wanted me to do it was to kind of harmonize everything in kind of an inheritance hierarchy where there was a parent class, shell info, or shell test or whatever I called it. And then for each shell that got subclassed and all the test logic and the defaults were in the parent and then anything that needed overwriting was in the children. And so in those test methods I would construct, I would use the temp path factory fixture by test really nice to spawn a new temporary directory. And then for each shell I would create a script that what I ended up having to do was in a shell specific way. I would Echo the contents of the environment variable that defined the prompt template, and then I would Echo that to standard out and then inspect that before and after environment activation back in pytest. So I ended up can’t remember exactly why I ended up. I couldn’t capture it using the subprocessor has the capture STD out and capture STD error methods that didn’t work. So I ended up having to just redirect my Echo commands to an output file and then back in pytest, I would read that file scrape, pick it apart into lines, and then inspect it that way.
00:19:18 Oh wow. Okay, that was probably more work than you were thinking this would be.
00:19:24 That was considerably more work than I was thinking. Yes, so much of it, and there were so many. So once I figured out that framework, it worked fine. The shell equals true was critical. A lot of the existing environment activation tests were set up to use shell equals false, which worked well for those. But like for Bash, for example, the PS one environment variable that holds the shell template if you don’t use shell equals true, that environment variable simply doesn’t exist. So I tried for probably an hour to make either the old test work with shell equals true or my tests work with shell equals false and it was just not happening. So I had them separate. But once I got that figured out, it was pretty solid, except for the fact that I discovered all sorts of really weird things about the shells and actually about the CI environment on Azure DevOps, which is where the CI was running. And oh my gosh, some of these were really hard to debug. I think CSH was probably the biggest headache with Bash. If you want to see what the prompt format string is, you just Echo dollar sign PS one and it spits it out for you. Csh doesn’t do that.
00:20:38 I think you can’t Echo environment variables, it doesn’t print them correctly. So I actually had to call set and then pass that through rep.
00:20:49 I think prompt is the variables, so I had to just call set the entire list of environment variables, pass it through grep, pick out the one that I wanted, and then strip out the leaders so that all I had was the format string. It was a small piece of a nightmare, and for a non interactive shell, it simply doesn’t set the prompt environment variable, which really threw me because it would work fine when I was testing it on my server. It’s like I instantiate CSH and I run it, and because it’s local, even run through subprocess, it counts as an interactive shell for whatever reason. But on Azure DevOps it was a noninteractive shell, and so it was an inconsistent starting state between local development and remote test development. So I finally figured that out and added a I have to prepopulate the environment with the prompt variable at the outset of the test.
00:21:46 Colorizes by default uses the antsy color codes to colorize by default. And so it’s all this open bracket Ampersand percent sign text code semicolon stuff that it inserts everywhere. And so when I tried to do so, in theory, when you activate the prompt, if it’s just plain text, you have the old prompt format string and then your prefix just tacks in behind it in the format string. Well, with Fish, that didn’t work because there were all these formatting codes that I didn’t want to mess with. So I had to play with it until I found a weaker assert as kind of a compromise solution.
00:22:20 But then again, Azure DevOps for the win. Again, not their fault. But Azure DevOps doesn’t set a terminal type, and it’s Ubuntu VMs. Which okay, no big deal. It shouldn’t affect anything. But locally, even if you set terminal equal to null, that behaves differently than this particular unset terminal. And oh my gosh, so I would just kept failing. I kept failing. And the assert message was in the end, it was very clear what was going on. Fish uses a string interpolation when it puts together the modified prompt. Okay, and so the first the two argument string interpolation. The first argument is the colorization codes, and the second argument is the actual prefix string. And so the color codes go before the parentheses, and the environment name goes between the parentheses. But when term is unset, it strips out the ante codes and passes an empty string. And so the string interpolation was taking what was supposed to be the second argument and interpreting it as the first argument. So instead of getting color codes per n en close per end, I was getting en open Paren close parent.
00:23:27 Why is it putting it in the wrong place?
00:23:30 There’s two things. Why is it putting it in the first thing place and then, oh my gosh, I think I posted an issue on the Fish repo, and they’re like, oh, yeah, it will do this.
00:23:42 Thank you so much. And it was painful and exciting all at the same time. So, again, obvious in retrospect, but it was this incredible debugging issue. So simple on the surface, but, oh, my gosh, all these really weird quirks were quite interesting to dig into.
00:24:00 And then you listed one of the shells as Posh. I’ve never even heard of this. Who’s Posh?
00:24:06 It’s an abbreviation or PowerShell.
00:24:08 Okay.
00:24:09 I had seen it before I looked into the test suite, but yeah, so I just absorbed that abbreviation.
00:24:14 Okay. And in your normal day to day life, which shell do you normally use?
00:24:19 Cmd on Windows and Bash on Linux.
00:24:21 Okay, so for CMD, I’ve noticed that my prompt for my virtual environment shows up on top of the other prompt instead of next to it. Am I doing something wrong?
00:24:33 Top of it, like stacked vertically?
00:24:35 Yeah, like it’s on a line above.
00:24:38 That sounds like a bug.
00:24:39 Okay, that’s odd.
00:24:41 I haven’t observed that on my Windows.
00:24:42 Maybe there’s something odd that I’ve set up in my environment. Okay, well, are you done with this now?
00:24:49 Yeah, this was like middle 2019 maybe.
00:24:52 Okay, yeah.
00:24:53 So this all got closed up and merged. So part of why virtual is now so fast on Windows, it was always reasonably fast on Bash, but on Windows is because Bernard went back through and completely reimplemented the whole thing.
00:25:09 Souped to nuts the entire virtual lens, and I think he revamped the whole test suite as well. So nothing. I do not want anyone to think that I bear any ill will on this, but the prompt consistency test did not survive the transformation, the rewrite, and to a certain extent, it kind of doesn’t matter, because once all those activation scripts got sculpted together the right way, as long as no one changes it, the tests really aren’t needed. And they were fairly slow. So I think looking at where he went with it, I am comfortable with them departing.
00:25:42 Okay, so are those tests then still around, or are they part of the project?
00:25:47 It’s a history of the project. Actually, earlier today I was talking through a bit what I had done with one of my friends. I was actually looking into the tests, and as far as I can tell, it’s only in the history of the repo. It’s the code is not there currently.
00:26:01 Okay, well, I mean, I guess part of the issue would be that you can’t run them on any single machine.
00:26:07 Sure. So the virtual and the test suite already had some OS specific flags.
00:26:12 Okay.
00:26:13 I don’t think it was put into a fixture. It was just a standard way of sniffing it. And so when I put the tests together, basically it only would run deposits shells on Linux CI instances. It would only run the Windows shells on Windows instances.
00:26:29 Okay.
00:26:30 With just standard skips where it wasn’t applicable.
00:26:33 Okay, well, I have to bug Burnett and say, hey, what’s up with the disappearing tests, man?
00:26:39 With the amount of work he did to pull it over, I think he just tweeted about he has a talk where he goes through all the process of the rewrite. So given the amount of work he did on that, and I benefit usually from the speed on Windows, so I don’t want to give him too hard of a time.
00:26:52 Okay, so was your contribution before the rewrite or after it was just before the rewrite.
00:26:58 Maybe a few months before the rewrite.
00:26:59 Okay, have you checked after the rewrite? Does the prompt still work correctly?
00:27:03 It works on Windows and on Bash. I have not actually gone back through and checked the other shelves, but that would be something to do just to see.
00:27:13 And also the CI and the Azure DevOps sort of stuff. Is that something you were familiar with ahead of time, or do you learn this for this project?
00:27:20 I was relatively familiar with the basics of Travis CI.
00:27:25 This was the first time I had really used the Azure DevOps very much. I didn’t have to change the config for that at all. All of the specific skips were done in the test code itself. I’ve since done a deeper dive into Azure DevOps. Actually, that’s where I learned I have a list of things that I’m trying to learn and apply to my projects to tighten them up and learning Azure DevOps was one of those. So a few months ago, I dug deep into the YAML and pretty comfortable with the Azure DevOps. Yaml now, but at the time it was pretty new.
00:28:00 Okay, so Virtual Environment, Virtual and Build processor test process is running on Azure pipelines right now.
00:28:07 Yeah, I think it’s all Azure only. Okay. Yeah.
00:28:11 Hasn’t switched to GitHub actions now.
00:28:14 So I’ve also I started off on Travis. I’ve learned Azure, and I actually put a little bot together that I use GitHub Actions as the kind of the automation for it. Oh, no, I could take it back. It looks like virtual and does have a little bit of GitHub actions going on. And so I’ve done enough with Travis and Azure DevOps and GitHub Actions, I think to have a good feel of the differences between them. And I might be being unfair to Travis because I haven’t gone back and really dug into what it can do, but it seems like if I had to rank them in terms of the range of capabilities, I feel like Travis is slightly easier than GitHub actions to get a bare bones CI going. Github Actions is pretty good, but you have to write a bit more and a bit more complex YAML in order to get there. And then Azure DevOps is by far the most complex of the three. To get started on. But in terms of the complexity of what you can do kind of goes correspondingly. And so Travis and Get actions, are they top out a lot sooner compared to the DevOps.
00:29:22 This definitely highlights the need for these types of CI systems to be able to if you’re testing out multiple operating systems, it’s not realistic to do that outside of a CI system.
00:29:33 Absolutely.
00:29:34 Yeah. So for me personally, I definitely I bounce back and forth between Windows and Linux quite a bit. And so I kind of get some scattershot testing on both platforms. And so that helps some. Definitely. I really like the CI being able to do rigorously across all three of the majors, the Linux and the Mac OS and the Windows. So I have a DBN server at home that I actually run some of just for fun because I’m a nerd, I run some quantum chemical calculator. I’ve got some long running calculations that I have set up on that, but I also use it for programming. And so when I really have my detailed talks matrix where I’m testing across Python three five to three nine, and a fairly diverse matrix of versions of my Dependencies that I only do on Linux, because that way I could just fire and forget. And I’m not chewing up a lot. I’m not heating up and slowing down our personal laptop.
00:30:29 So I don’t know, it’s probably my workflow is kind of scattered. It could probably be improved. But given that I’m not working on the clock for anything, just doing it for fun, it works.
00:30:39 Yeah. Well, that’s a pretty interesting story there, Brian. It’s a lot of work.
00:30:43 Yeah. It was really great to work with Bernard on it. He was very kind with me when I did not two or three times didn’t understand what he was asking me to do, but it was a very, very positive open source contribution experience for me. I’m really glad.
00:30:57 That’s wonderful.
00:30:58 Yeah.
00:30:58 That’s great. Do you have anything in the future that you’re thinking about getting sinking your teeth into?
00:31:03 Oh, my. I think since it’s on the side, I have so many things that I would like to do that I have so little time for. I think one of the biggest things that I want to get back to is I have one of my side projects called Pentically.
00:31:20 It’s a regular generator, and so it’s aimed mostly at scientific data, where if you have an application that instead of generating its output as a structured like a gamble JSON or a structured binary format or something like that, a number of them will just report their output to standard out, and then you capture it to a text file and you have to scrape your data out of there. And so say you’ve got a two dimensional matrix that’s in this pretext output. So if it’s a big matrix, you may have a list of six columns with 100 items and then another below that, six columns with 100 items. Below that, six more columns and 100 items. And I had a project where I was trying to scrape some of that, and it’s just that the writing, all of the reggae to grab those blocks and pick them apart and reconstruct the matrix out of it was really getting old to write by hand. And so one of the use cases of Pent uses pipe parsing, and it has its own internal little mini language that you can use to define the structure of, for example, this column or data or this matrix data that’s broken up into these columns, and you provide these mini language patterns to the Pent parser. And on the fly, it generates the regex that it needs to go through there and parse it and pull out all that data into a cleanly formatted. It initially starts out as nested lists a string because it’s parsing it as text. I don’t want to make any assumptions about number format or anything like that. And then from there, it’s just a direct. You can basically do NumPy as array data, nested lists, comma, dtype, equals whatever. And then boom, you’ve got a NumPy array ready to go, or Pandas or whatever. And so that the core functionality is there, but there are a number of refinements that I want to get in there. And so the docs are sort of mostly done. There are some things I want to finish. That’s what I want to, I think is my next priority, to try to fully fledge out the docks and get some of the next set of features in there.
00:33:19 Okay.
00:33:20 That I want in.
00:33:21 That’s cool. Nice. So I love this story about just kind of trying to make things consistent across different platforms and carry enough to get in there and do it. So this is cool. So thanks.
00:33:32 Yeah, I would say one thing. There’s a lot of stuff out there about, and believe me, I felt it the nerves and the uncertainty about getting in there and you’re getting your first open source contribution in there. My goodness. Do not hesitate. The vast majority of maintainers are extremely nice people, and they would love to have your help and love to help you write something that helps the project out. Just be ready for them to come back and say, no, I want you to write this way because this fits the style of the project better. Or I want you to write this on a class based way instead of a function based way, because it’s going to integrate better with the rest of the test suite or the rest of the code. So come with your ideas, come eager to fix, but be ready to adapt to the requests of the maintainers when they come back to them, to you with that.
00:34:16 Yeah, I think that’s a good thing. Contributing to an open source project.
00:34:20 Those are some of the extra things you get is all that stuff that’s hard to really label but I feel like I’ve become a better software developer on the job as well because of the interactions I’ve had with people with open source projects.
00:34:35 Yeah. Communication and collaboration are not automatic are not necessarily easy. They take practice.
00:34:40 Yeah. Also paying attention. I mean, like Burnett and some other project maintainers that are really good at this. It’s a skill and I’m grateful that they have it. And being able to encourage people to contribute but encourage to consistent style and making sure the tests are there and things like that, it’s a hard thing to do because you’re asking somebody’s volunteering their time and you’re saying that’s not good enough. You need to do more.
00:35:07 How to do that? Tactfully so it’s good.
00:35:10 It’s a balancing act.
00:35:11 Well, thanks, Brian, for sharing this story with us. It was a fun trip that we took with you so talk to you later.
00:35:17 Thank you very much for having me on. I appreciate it.
00:35:22 Thank you, Brian. Thanks for fixing the prompt consistency across shelves and for telling us this great story on the podcast. Thank you Patreon supporters for continuing to support the show. Join them by going to testandcode.com support. Thank you PyCharm for sponsoring the show. Try PyCharm yourself at testandcode.com PyCharm thank you honey Badger for sponsoring check them out at Honeybadger. Io and mention test and code and get 30% off. Thank you talk pythonraining for sponsoring check them out at talk. Python. Fmtest to level up your skills. Enter to win a free course by joining this show’s mailing list at testingco. Comsubscribe. All of those links as well as links mentioned in the show are in the show notes at testing Code.com 130 that’s all for now. Now go out and test something.